I'm building a Python web scraper that goes through an eBay search results page (In this case 'Gaming laptops') and grabs the title of each item for sale. I'm using BeautifulSoup to first grab the h1 tag where each title…
I'm trying to scrape data from this website, which has a table of game credits for different categories. There are a total of 24 categories that I want to make in to 24 columns. In the example "https://www.mobygames.com/developer/sheet/view/developerId,1/" there are…
I'm trying to fetch the Book Title and books embeded url link from an url, the html source content of the url looks like below, i have Just taken some little portion out of it to understand. The when link…
Goal: I want to make a web scraper in a Rails app that runs indefinitely and can be scaled. Current stack app is running on: ROR/Heroku/Redis/Postgres Idea: I was thinking of running a Sidekiq Job that runs every n minutes…
I need to pull the sku (CS060120) from this section of html (edit:) Using VBA tried getAttribute and getElementsbyTag(and Class)Name but I am not able to get the return I am looking for. Anyone know how to do this? <section…
I'm trying to Scrape a blog "https://blog.feedspot.com/ai_rss_feeds/" and crawl through all the links in it to look for Artificial Intelligence related information in each of the crawled links. The blog follows a pattern - It has multiple RSS Feeds and…
I am finally getting the hang of Python and have started using it on a daily basis at work. However, the learning curve is still steep and I have hit a roadblock in trying something new with a code I…
I am trying to scrape a public facebook page using facebook api, but every time i sent request it return HTTP Error 400: Bad Request I am putting correct access token, which is app id + "|" + app_secret. But…
I want to build a dashboard that enables any company to track the social media activity of its competitors. For example.. let's say you are a small business owner. You would be able to add the 'facebook/instagram page id's' of…
I'm using Screaming Frog as a way to extract data from a Json generated from an URL. The Json generated is this form : {"ville":[{"codePostal":"13009","ville":"VAUFREGE","popin":"ouverturePopin","zoneLivraison":"1300913982","url":""},{"codePostal":"13009","ville":"LES BAUMETTES","popin":"ouverturePopin","zoneLivraison":"1300913989","url":""},{"codePostal":"13009","ville":"MARSEILLE 9EME ARRON","popin":"ouverturePopin","zoneLivraison":"1300913209","url":""}]} I'm using this regex in Custom > Extraction in Screaming Frog as…