web-scraping Questions

Woocommerce – How to extract price from web page using Beautiful Soup?

December 14, 2023
luna28blue
2 Answers

import requests from bs4 import BeautifulSoup URL="https://shop.beobasta.rs/proizvod/smrznuti-spanac/" header={"User Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 OPR/105.0.0.0","Accept-Language":"en-US,en;q=0.9"} response=requests.get(URL,headers=header) soup=BeautifulSoup(response.text,'html.parser') price_element = soup.find("span",class_="woocommerce-Price-amount amount") print(price_element) I'm trying to extract price from this website but the only thing…

VIEW QUESTION

Image won't display in browser using html

December 11, 2023
Darth_Buttholio
4 Answers

I am trying to do something very simple and display an image I scraped off the web using HTML. For some reason, it's not working. <!DOCTYPE html> <html> <body> <h2>HTML Image</h2> <img src="headline1.jpg" alt="headline" width="500" height="333"></img> </body> </html> My project…

VIEW QUESTION

Web scraping scripts using selenium not running inside docker container

December 8, 2023
BicycleChrist
3 Answers

Have a couple web scraping scripts using selenium and Firefox I'm trying run inside of a docker container. I'm using Arch Linux, but I've made sure to closely follow the install directions on the official docker website specific to arch…

VIEW QUESTION

Selenium webscraping affected by environment (cloud, docker, computer)?

December 6, 2023
Geoffrey
2 Answers

I have a rather strange problem that I can't explain. Context: as part of a project, I created a webscraping script (Selenium + Beautifulsoup) to retrieve job offers in France on Linkedin for Data professions (Data Analyst, Data Engineer, Data…

VIEW QUESTION

Html – using rvest to extract lat lon

November 24, 2023
89_Simple
2 Answers

I am trying to get the lat lon of all stores from here https://www.wellcome.com.hk/en/our-store On inspecting, I can see that lat and lon are contained within div library(dplyr) library(rvest) url_company <- rvest::read_html("https://www.wellcome.com.hk/en/our-store") url_company %>% html_elements("div") %>% # extracted all the…

VIEW QUESTION

Seo – python googlesearch library – how to remove + OR %20 signs at backend for using googlesearch library?

November 20, 2023
John Moores
2 Answers

i am using googlesearch library in python for scraping google search results . although if i type normal words in chrome i get results..but while sending the search string thriugh python library googlesearch i am unable to get same result…

VIEW QUESTION

Scraping a website using PHP "Simple HTML Dom Parser" for data

November 18, 2023
Blackstone
2 Answers

I'm trying to use the PHP Simple DOM Parser to pull information from a website and it isn't working for me. Specifically i am trying to get the: 5-day prediction 1-month prediction values from the top of the page. i…

VIEW QUESTION

Html – How to get text of td next to its previous sibling th with BeautifulSoup?

November 18, 2023
Newbie
2 Answers

Using BeautifulSoup/Python, I like to get the td text, Med from the following screenshot, or the ETF Risk from this website. I tried the following code, but it didn't work. What should be corrected? Risk = soup.find('th', string="ETF Risk").find_sibling("td").text

VIEW QUESTION

Extract Daily Mutual Funds NAV data from AMFI Website and Store it in MongoDB

November 16, 2023
dewashya
2 Answers

I want to download Daily NAV (Net asset Value) from AMFI Website for all the schemes and store all of it in MongoDB. But with my current code it is taking too long almost 5 days to download and push…

VIEW QUESTION

Javascript – Navigation Timeout Exceeded scraping table in Puppeteer

November 14, 2023
Evan
2 Answers

I am trying to scrape the very first name on a table from a website that presents a basketball team and that team's player's names and statistics. When I do so the Navigation Timeout is exceeded meaning the value was…

VIEW QUESTION