skip to Main Content

I am trying to learn how to scrape websites and therefore not using an API. I am trying to scrape eBay’s websites and my script runs for a few minutes but stops because of an error stating “AttributeError: ‘NoneType’ object has no attribute ‘text'”. I did my due diligence and search on Google/StackOverflow help but was unable to find any. The error points to the code below but I cannot figure out how to fix it.

total_sold_price = soup.find('span', {'class': 'vi-qtyS-hot-red'}).text

if total_sold_price:
    total_sold_price = soup.find('span', {'class': 'vi-qtyS-hot-red'}).text

else:
    try:
        total_sold_price = soup.find('a', {'class': 'vi-txt-underline'}).text
    except Exception as e:
        print(e)
        total_sold_price = ""

Entire code: https://pastebin.com/xS4bAwZK

Thanks in advance, I greatly appreciate it.

2

Answers


  1. The issue is soup.find('span', {'class': 'vi-qtyS-hot-red'}) returns None, and then you are trying to extract the text .text of None (hence the error message). There are a few ways you can handle it. All I did was move around some logic:

    Also, you have it store the text total_sold_price = soup.find('span', {'class': 'vi-qtyS-hot-red'}).text, and if it stores it, you have it do the exact something which is redundant.

    Lastly, the page is dynamic, so you might want to look into API or other ways to access the data.

    But for your code provided, maybe something like this instead:

    try:
        total_sold_price = soup.find('span', {'class': 'vi-qtyS-hot-red'}).text
    except Exception as e:
        try:
            total_sold_price = soup.find('a', {'class': 'vi-txt-underline'}).text
        except Exception as e:
                    print(e)
                    total_sold_price = ""
    
    Login or Signup to reply.
  2. Is it solved?
    Instead of {‘class’: ‘vi-qtyS-hot-red’}
    You can write = ‘vi-qtyS-hot-red’

    No need write ‘class’ and {}

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search