skip to Main Content

By use following code, I got the output
driver = webdriver.Chrome(‘/full/path/to/chromedriver’)

Product Name: USDA INSPECTED BEEF RIBEYE STEAK*

Product Price:

I am very confused that why product price is empty and it should be 9.98
here is the element "span class="price__value">$9.98</span"

import pandas as pd
from selenium import webdriver
import time
from selenium.webdriver.common.by import By


# Navigate to the wholefood product page
driver = webdriver.Chrome('/full/path/to/chromedriver')
url = 'https://wildforkfoods.com/products/usda-inspected-beef-ribeye-steak?region_id=990300&gclid=CjwKCAjwoIqhBhAGEiwArXT7K5X1qBn9wREml7UC643CTfsmyPI5ctxRMzRe3FA2To7CddZf43dnCBoC7lYQAvD_BwE&variant=9385081438244'
driver.get(url)

# Wait for the page to load
time.sleep(5)

# Extract the relevant product data
product_name = driver.find_element(By.XPATH, '//h1[text()="USDA Inspected Beef Ribeye Steak*"]')
product_price = driver.find_element(By.XPATH, '//span[@class="price__value"]')


# Print the product data
print('Product Name:', product_name.text)
print('Product Price:', product_price.text)

data = {'Product Name': [product_name.text], 'Product Price': [product_price.text]}
df = pd.DataFrame(data)

# Save the data to a csv file
df.to_csv('product_data.csv', index=False)

# Close the browser window
driver.quit()

I want to know the reason why I got empty and how can I improve it.

2

Answers


  1. Anything with character or symbols / etc, you can extract innerHtml like so :

    product_name = driver.find_element(By.XPATH, '//h1[text()="USDA Inspected Beef Ribeye Steak*"]')
    product_price = driver.find_element(By.XPATH, '//span[@class="price__value"]')
    
    product_price_innerHtml = product_price.get_attribute('innerHTML')
    

    Look at the result here

    Login or Signup to reply.
    1. Try with this XPath of span. I think your XPath is incorrect.

      product_price = driver.find_element(By.XPATH, ‘//*[@id="ProductForm"]/div[1]/div[2]/div/span[2]/span[2]/span[3]’)

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search