I am looking for a method to get information of a "trend" regarding some hashtag/key word on Twitter. Let`s say I want to measure how often the hashtag/key word "Python" is tweeted in time. For instance, today, "Python" is tweeted…
I'm trying to scrape https://sportsbook.draftkings.com/event/179665865?category=odds&subcategory=player-props however my code wasn't working and when I troubleshot it I realized it was taking me to the original page at https://sportsbook.draftkings.com/event/179665865 The first link is a tab within the second link. Here is the…
I have written the following code to login to a website. So far it simply gets the webpage, accepts cookies, but when I try to login by clicking the login button, the page hangs and the login page never loads.…
I've made a node.js web scraper code that runs fine on my computer, however, when I deploy to my Google Cloud VM instance running Debian, it returns a timeout error for a specific website. I've tried many differnent setups for…
I'm working on an eBay Scraper, and I'm having some trouble with a simple "AttributeError: 'NoneType' object has no attribute 'text'"... Here is my code url = 'https://www.ebay.com/sch/i.html?_from=R40&_nkw=2017+patrick+mahomes+psa+10+auto&_sacat=0&rt=nc&LH_Sold=1&LH_Complete=1&_ipg=100' def get_data(url): r = requests.get(url) soup = bs(r.text, 'html.parser') return soup def…
I am creating web scraper using scrapy python. Here is my code import scrapy class BlogSpider(scrapy.Spider): name = 'blogspider' start_urls = [ 'https://perfumehut.com.pk/shop/', ] def parse(self, response): yield { 'product_link': response.css('a.product-image-link::attr("href")').get(), 'product_title': response.css('h3.product-title>a::text').get(), 'product_price': response.css('span.price > span > bdi::text').get(), }…
I am creating a data scraper using scrapy. I shell the product url using scrapy shell 'https://royalprint.pk/product/name-print-superhero-sweatshirt-011/' and then run this command In [43]: response.css('span.woocommerce-Price-currencySymbol::text').get() Out[43]: 'Rs' It only returns the currency symbol. Here is the source code product image…
I have to collect some data from Twitter via Search. Unfortunately, its not possible to get Twitter API as they have rejected my request many times. I have decided to use twint python module for this purpose. I have installed…
As it is clear GetOldTweets3 is a python library for accessing old tweets which is not easily feasible with libraries like Tweepy. But recently, there is an unsolved issue with it due to the new Twitter API: https://github.com/Mottl/GetOldTweets3/issues/98. The question…
I have a simple script to analyze sold data on eBay (baseball trading cards). It seems to be working fine for the first 4 pages but on the 5th page it simply does not load in the desired html content…