Scraping Ebay, working until I use it in sold items

There are several reasons why the output will be empty.

This is often because the site may think it is being accessed by a bot if requests is the default user-agent in the requests library is python-requests, this can be prevented by passing your actual User-Agent to the "headers". This seems to be a reason why you get a CAPTCHA.

The next step would be if User-Agent passing didn’t work would be to use rotate user-agent, for example, to switch between PC, mobile, and tablet, as well as between browsers e.g. Chrome, Firefox, Safari, Edge and so on.

Also if passing request headers is not enough. That’s when you can try using proxies (ideally residential) in combination with request headers.

An additional step is to use CAPTCHA solver, for example, 2captcha. It allows bypassing all possible CAPTCHAs depending on the target website.

Check the code using BeautifulSoup in online IDE.

from bs4 import BeautifulSoup
import requests, json, lxml

# https://requests.readthedocs.io/en/latest/user/quickstart/#custom-headers
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36",
}
   
params = {
    '_nkw': 'iphone_x',    # search query 
    'LH_Sold': '1',        # shows sold items
    '_pgn': 1              # page number
}

data = []
limit = 10                 # page limit (if needed)
while True:
    page = requests.get('https://www.ebay.es/sch/i.html', params=params, headers=headers, timeout=30)
    soup = BeautifulSoup(page.text, 'lxml')
    
    print(f"Extracting page: {params['_pgn']}")

    print("-" * 10)
    
    for products in soup.select(".s-item__info"):
        title = products.select_one(".s-item__title span").text
        price = products.select_one(".s-item__price").text
        
        data.append({
          "title" : title,
          "price" : price
        })

    if params['_pgn'] == limit:
       break
    if soup.select_one(".pagination__next"):
        params['_pgn'] += 1
    else:
        break

print(json.dumps(data, indent=2, ensure_ascii=False))

Example output:

[
    {
    "title": "Apple iPhone X 64 GB y 256 GB Grado A++ Desbloqueado - Excelente Estado Todos los Colores",
    "price": "234,52 EUR"
  },
  {
    "title": "Funda de silicona a prueba de golpes para iPhone 11 Pro Max 14Pro 8 7 SE 2022 colores",
    "price": "4,56 EUR"
  },
  {
    "title": "Apple iPhone X 64 GB 256 GB gris plateado sin contrato COMO NUEVO SIN MANCHA Wow ",
    "price": "377,00 EUR a 409,00 EUR"
  },
  {
    "title": "Funda transparente de silicona completa a prueba de golpes para iPhone 11 12 13 14 PRO MAX Mini X XR 8",
    "price": "1,13 EUR a 4,06 EUR"
  },
  {
    "title": "Apple iPhone X - 256 GB - Plateado (Desbloqueado) (Leer descripción) FA1065",
    "price": "163,88 EUR"
  },
  other results ...
]

Also you can using official eBay Finding API, has a limit of 5000 requests per day, or third-party API like Ebay Organic Results API from SerpApi. It’s a paid API with a free plan that handles blocks and parsing on their backend.

Example code with pagination:

from serpapi import EbaySearch
import json

params = {
    "api_key": "...",                 # serpapi key, https://serpapi.com/manage-api-key   
    "engine": "ebay",                 # search engine
    "ebay_domain": "ebay.es",         # ebay domain
    "_nkw": "iphone_x",               # search query
    "LH_Sold": "1",                   # shows sold items
    "_pgn": 1                         # page number
}

search = EbaySearch(params)           # where data extraction happens

page_num = 0

data = []

while True:
    results = search.get_dict()     # JSON -> Python dict

    if "error" in results:
        print(results["error"])
        break
    
    for organic_result in results.get("organic_results", []):
        title = organic_result.get("title")
        price = organic_result.get("price")

        data.append({
          "title" : title,
          "price" : price
        })
                    
    page_num += 1
    print(page_num)
    
    if "next" in results.get("pagination", {}):
        params['_pgn'] += 1
    else:
        break
    print(json.dumps(data, indent=2))

Output:

[
  {
    "title": "Apple iPhone X (10) Desbloqueado 64 GB/256 GB Gris espacial/Plateado - Excelente Estado",
    "price": {
      "raw": "297,34 EUR",
      "extracted": 297.34
    }
  },
  {
    "title": "Nuevo anuncioApple iPhone X - 64GB - Bianco (Sbloccato).",
    "price": {
      "raw": "340,00 EUR",
      "extracted": 340.0
    }
  },
  {
    "title": "Apple iPhone X - 256GB - Factory Unlocked - Good Condition",
    "price": {
      "raw": "230,80 EUR",
      "extracted": 230.8
    }
  },
  other results ...
]