i’ve succesfully scraped data from this site:
https://football.goaloo2.com/subleague/2022-2023/40/261/?selectround=23
Now after some hints and some basic tutorials i’m trying to scrape it using hidden API…but on this site i got something different from others i was able to scrape. This is my process: with chrome i do inspection/network and then Fetch/XHR, reloadong the page i got what i thought it would be the call (even if the preview is different from other scraped sites, there’s no three to navigate); than i copy as curl(cmd) and after pasting in converting site to python i obtained this:
import requests
params = {
'sclassId': '40',
'subSclassId': '261',
'matchSeason': '2022-2023',
'round': '23',
'flesh': '0.6807624444063407',
}
response = requests.get('https://football.goaloo2.com/ajax/LeagueOddsAjax', params=params)
odds = response.json()
I got this error: JSONDecodeError: Expecting value
I suspect data is not embedded in a json object, any idea on how to proceede if it is doable in python?
2
Answers
The returned object is not decodable JSON. You can take out the raw text and manipulate it into the JSON format. Then manually load it using json.loads() function. Note that, the web scrapping is not always standartized and you should check the stability of the manipulation. A very quick and dirty transformation as follows;
It returns as follows;
The response isn’t in
json
format that’s why theresponse.json()
failswe can use regular expressions to extract the data
and eval to convert string to list