I am trying to learn how to scrape websites and therefore not using an API. I am trying to scrape eBay’s websites and my script runs for a few minutes but stops because of an error stating “AttributeError: ‘NoneType’ object has no attribute ‘text'”. I did my due diligence and search on Google/StackOverflow help but was unable to find any. The error points to the code below but I cannot figure out how to fix it.
total_sold_price = soup.find('span', {'class': 'vi-qtyS-hot-red'}).text
if total_sold_price:
total_sold_price = soup.find('span', {'class': 'vi-qtyS-hot-red'}).text
else:
try:
total_sold_price = soup.find('a', {'class': 'vi-txt-underline'}).text
except Exception as e:
print(e)
total_sold_price = ""
Entire code: https://pastebin.com/xS4bAwZK
Thanks in advance, I greatly appreciate it.
2
Answers
The issue is
soup.find('span', {'class': 'vi-qtyS-hot-red'})
returns None, and then you are trying to extract the text.text
of None (hence the error message). There are a few ways you can handle it. All I did was move around some logic:Also, you have it store the text
total_sold_price = soup.find('span', {'class': 'vi-qtyS-hot-red'}).text
, and if it stores it, you have it do the exact something which is redundant.Lastly, the page is dynamic, so you might want to look into API or other ways to access the data.
But for your code provided, maybe something like this instead:
Is it solved?
Instead of {‘class’: ‘vi-qtyS-hot-red’}
You can write = ‘vi-qtyS-hot-red’
No need write ‘class’ and {}