This is part of the json file I have got as an output after running running a python script using the telethon API. [{"_": "Message", "id": 4589, "to_id": {"_": "PeerChannel", "channel_id": 1399858792}, "date": "2020-09-03T14:51:03+00:00", "message": "Looking for product managers /…
I am trying to scrape the title of every item on an ebay page. This is the page. I first tried to scrape the title of the first listing (lines 5-7 of my code) , and I was successful as…
So I have a list of URLs that I pull from a database, and I need to crawl and parse through the JSON response of each URL. Some URLs return null, while others return information that is sent to a…
I am trying to collect data from shoes on eBay. For every item I want to collect all data including the custom description to build up a database. I have collected all aspects such as price, shipping title etc. with…
My goal is to be able to read a certain nested deep within a ton of divs. The only issue is that they seem to be dependent on javascript, so I can't get them by just using driver.page_source as far…
I'm trying to web scraping many websites, but one of these is "Lime Crime" that is supported by Shopify (AFAIK). I'm using lxml library, but when I tried to use xpath to go to an element I got an empty…
I am trying to scrape the price of the coin from the link below using the following Python Script, however I am running into some issues. If you could identify where I am going wrong then that would be great!…
I am trying to retrieve the variable in json format to python. url = "https://limitededt.com/collections/footwear/products/adidas-originals-jonah-hill-superstar-fw7577" source = requests.get(url).text soup = BeautifulSoup(source, 'lxml') print(soup.findAll('script')[10]) The code above produces: <script>window.ShopifyAnalytics = window.ShopifyAnalytics || {}; window.ShopifyAnalytics.meta = window.ShopifyAnalytics.meta || {}; window.ShopifyAnalytics.meta.currency = 'SGD';…
Sorry to disturb you guys. This is bad question, seems what really confused me is how ItemPipeline works in scrapy. I'll close it and start a new question. Where should I bind the db/redis connection to on scrapy, Spider or…
I am new to programming and also new to pyhon. My intension is to built an ebay webscraper. I am trying to extract an list of links with the bs4 find_all() method, but no matter what I try, it returns…