I am trying to use BeautifulSoup to access thesaurus.com in order to quickly find synonyms for certain words. However, the synonyms are in a list that has different ids and classes per word, and so the best thing I can…
I am trying to extract data from a complex data structure (example will follow). I am not sure what is the best method (e.g. Beautifullsoup) to parse the test and get the desired field in a table. I need for…
As a fun experiment, I decided to scrape data from Google shopping, it works perfectly on my local but on my server it doesn't work. Here is the code #Web driver file from selenium import webdriver from selenium.webdriver.chrome.options import Options…
I am working on a data cleaning problem wherein I have a task to remove HTML tags from string while keeping the content of text. Example text for cleanup is given below. I tried removing "pre" tags and somehow i…
I am fetching 'value' from 'key' in JSON but I do not know why I cannot fetch the target information. Code below import json import requests #Person's ID id=1194452 #Url info=requests.get(f'https://api.brokercheck.finra.org/search/individual/{id}?hl=true&includePrevious=true&sort=bc_lastname_sort+asc,bc_firstname_sort+asc,bc_middlename_sort+asc,score+desc&wt=json') #convert to JSON x=info.json() #Value print(x["firstName"]) The following is…
I am trying to get the price of one item on the website in the url below. However, I am finding some issues when looking at the source page of the website. The url is: https://www.cartier.com/en-gb/love-bracelet-small-model_cod25372685655708131.html#dept=EU_Love The part of the…
I want to compare the price of coconut on two websites. there are two stores (websites) called laughs and glomark. Now,I have two files main.py and comparison.py. I think the problem is in the Laughs price scrapping part. This cord…
I want to grab the string "AKA" from a listing site, but the find_all function fails to return any values. import requests from bs4 import BeautifulSoup # Set the URL you want to scrape url = 'https://classified.azcentral.com/azcentral-marketplace/category/Legals/Maricopa%20County' # Use requests…
I am trying to build a python webscraper with beautifulsoup4. If I run the code on my Macbook the script works, but if I let the script run on my homeserver (ubuntu vm) I get the following error msg (see…
I am trying to read the window.appCache from a glassdoor reviews site. url = "https://www.glassdoor.com/Reviews/Alteryx-Reviews-E351220.htm" html = requests.get(url, headers={'User-Agent': 'Mozilla/5.0'}) soup = BeautifulSoup(html.content,'html.parser') text = soup.findAll("script")[0].text This isolates the dict I need however when I tried to do json.loads() I…