Html Question posted in
Some basic documentation can be found here.

Html – Confused in how to do web scraping for the first time

There is a website like bonbast.com and I’m trying to get values but I’m just confused about how to do it. Values should be something like the output of "US Dollar" and "Euro".
My code:

import requests 
from bs4 import BeautifulSoup

r = requests.get("https://bonbast.com/")
soup = BeautifulSoup(r.content, "html.parser")

usd1 = soup.find(id="usd1")
print(usd1)

same_val = soup.find_all(class_="same_val")
print(same_val)

the output is blank and not showing the number of the currency. HTML page https://i.imgur.com/NrTaUY8.png

Tags:

2

Answers


  1. 0

    That page is filled by Javascript, that’s not somthing BS can scrape.

    You’ll need to use a browser like app like , here is an example:

    from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from selenium.common.exceptions import NoSuchElementException

options = webdriver.ChromeOptions()
options.add_argument("--disable-extensions")
options.add_argument("--no-sandbox")
options.add_argument("--disable-search-engine-choice-screen")
options.add_argument("--user-data-dir=selenium")
options.add_argument("--headless")

service = Service()
driver = webdriver.Chrome(service=service, options=options)
driver.get('https://bonbast.com/')

time.sleep(1)

try:
    trs = driver.find_element(By.CSS_SELECTOR, '#usd1')
    print(f"usd: {trs.text}")
except NoSuchElementException:
    print('Could not find usd1')
    exit()

driver.quit()

    With the result:

    (venv) ➜  python3 ../bonbast.py
usd: 62800
(venv) ➜
    Login or Signup to reply.
  2. 0

    From the Reference of this Repo – https://github.com/drstreet/Currency-Exchange-Rates-Scraper/blob/master/scrapper.py

    Focusing only on the necessary parts to retrieve exchange rates USD and Euro.

    import requests
from bs4 import BeautifulSoup
import re

def get_currency_data():
    url = 'https://www.bonbast.com'
    session = requests.Session()
    headers = {
        'User-Agent': 'Mozilla/5.0',
        'Referer': url,
        'Accept': 'application/json, text/javascript, */*; q=0.01',
    }

    response = session.get(url, headers=headers)
    soup = BeautifulSoup(response.text, 'html.parser')

    # Find the script containing the 'param' value
    scripts = soup.find_all('script')
    param = None
    for script in scripts:
        if '$.post' in script.text:
            match = re.search(r'param:s*"([^"]+)"', script.text)
            if match:
                param = match.group(1)
                break

    if param:
        post_url = url + '/json'
        post_data = {'param': param}

        # Make POST request to fetch the JSON data
        response = session.post(post_url, headers=headers, data=post_data)
        data = response.json()

        # Extract relevant currency data (USD and Euro)
        currencies = {}
        for key, value in data.items():
            if key == 'usd1':  # USD sell price
                currencies['USD_sell'] = value
            elif key == 'usd2':  # USD buy price
                currencies['USD_buy'] = value
            elif key == 'eur1':  # Euro sell price
                currencies['EUR_sell'] = value
            elif key == 'eur2':  # Euro buy price
                currencies['EUR_buy'] = value

        return currencies
    else:
        return {'error': 'Param not found'}

# Execute the function and print the result
result = get_currency_data()
print(result)

    output

    {'EUR_sell': '68500', 'EUR_buy': '68400', 'USD_sell': '62800', 'USD_buy': '62700'}
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search