Question 202766 in Html Question posted in
Some basic documentation can be found here.

Html – How to request/execute a command on a web page without using a web browser?

I’m trying to get the commentary of a football match from the William Hill website.
To get them I need the child elements of id="box_commentaries" to appear.

enter image description here

To make them appear I need to simulate a click on class="_commentary".

enter image description here

There are two ways to do this: by clicking directly or calling this function in the console.

enter image description here

My question is how can I do this without having to open a web browser? I don’t want to open a web browser because I’m going to be getting comments from dozens of matches simultaneously every minute and I don’t want to consume too much memory since I need to do more things.

The function I used was this:

(function() {
  setTimeout(function() {
    document.getElementsByClassName('_commentary')[0].click();
  }, 3000);
})();

To go directly to the page where the scoreboard is, just follow this link:

f’https://sports.whcdn.net/scoreboards/app/football/index.html?eventId={match_id}&sport=football&locale=en-gb&streamingAvailable=false&showSuggestions=true&expandDetails=true&showStreaming=false&referrer=https%3A%2F%2Fsports.williamhill.com’

You just need to change the match_id by the id of any live match

enter image description here

Ex: https://sports.whcdn.net/scoreboards/app/football/index.html?eventId=**28174073**&sport=football&locale=en-gb&streamingAvailable=false&showSuggestions=true&expandDetails=true&showStreaming=false&referrer=https%3A%2F%2Fsports.williamhill.com

P.S. I know how to use Selenium and I’ve written code with it that was very slow. Now I’m looking for other ways to make this request

Tags:

2

Answers


  1. 0

    Consider using https://npmjs.org/package/jsdom

    You need to setup it so it loads scripts and such, and it can emulate a browser without running a headless/head browser. Still there might be a chance it won’t work if the site has a lot of js that do stuff normally you shouldn’t.

    Login or Signup to reply.
  2. 0

    To my knowledge, you can either use Puppeteer or Selenium for this. Selenium is ideally suited for the test automation aspect. But in this case, you can use it for your use case. In puppeteer you can specify the above headless: true or false like below :

      let browser = await puppeteer.launch({headless: true});
  const page = await browser.newPage();
  await page.goto('YOUR_SITE');

    Similarly, in Selenium you can do the same as below :

        WebDriverManager.chromedriver().setup();
    ChromeOptions chromeOptions = new ChromeOptions();
    chromeOptions.addArguments("--headless");
    driver = webdriver.Chrome(executable_path="./chromedriver",options=chrome_options)
    url = "your url here"
    driver.get(url)

    Hope this helps.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search