skip to Main Content

HI i have put the below code together to look at the previous days high and low and then compare it to the day before to see what type of day it is with regards to the stock market.

I am having trouble though with it working consistently and can figure out why.

The error which comes back is below, after the code, however after about 5 minutes it then works, its very strange and I cant figure out what the issue is.

I’m using visual studio and I must admit this is the first time I have used python so there may be something obvious but I would really appreciate the advice from those who know more.

import yfinance as yf
import pandas as pd
from datetime import datetime, timedelta
import requests
from bs4 import BeautifulSoup

# Function to fetch S&P 500 tickers from Wikipedia
def fetch_sp500_tickers():
    url = "https://en.wikipedia.org/wiki/List_of_S%26P_500_companies"
    response = requests.get(url)
    soup = BeautifulSoup(response.text, 'html.parser')
    table = soup.find('table', {'class': 'wikitable'})
    tickers = []
    for row in table.find_all('tr')[1:]:
        ticker = row.find_all('td')[0].text.strip()
        tickers.append(ticker)
    return tickers

# List of known bank holidays
bank_holidays = [
    datetime(2025, 1, 1).date(),
    datetime(2025, 1, 20).date(),
    datetime(2025, 2, 17).date(),
    datetime(2025, 4, 18).date(),
    datetime(2025, 5, 26).date(),
    datetime(2025, 6, 19).date(),
    datetime(2025, 7, 4).date(),
    datetime(2025, 9, 1).date(),
    datetime(2025, 11, 27).date(),
    datetime(2025, 12, 25).date()
]

# Calculate last three unique market open days excluding today
today = datetime.now().date()
last_three_days = []
for i in range(1, 4):
    day = today - timedelta(days=i)
    while day.weekday() >= 5 or day in bank_holidays:
        day -= timedelta(days=1)
    last_three_days.append(day)

# Fetch S&P 500 tickers
tickers = fetch_sp500_tickers()

# Processing each ticker
for ticker in tickers:
    try:
        # Fetch data from the last 3 days
        start_date = last_three_days[2]
        end_date = today
        data_full = yf.download(ticker, start=start_date, end=end_date)

        # Ensure there are at least two days of data
        if len(data_full) < 2:
            print(f"{ticker} - Not enough data")
            continue

        # Get the high values for the two closest days to today
        high_day_1 = data_full['High'].iloc[-2] if len(data_full) >= 2 else None
        high_day_2 = data_full['High'].iloc[-1] if len(data_full) >= 1 else None

        # Get the low values for the two closest days to today
        low_day_1 = data_full['Low'].iloc[-2] if len(data_full) >= 2 else None
        low_day_2 = data_full['Low'].iloc[-1] if len(data_full) >= 1 else None

        # Calculate differences
        difference_high = float(high_day_2.iloc[0]) - float(high_day_1.iloc[0]) if high_day_1 is not None and high_day_2 is not None else None
        difference_low = float(low_day_2.iloc[0]) - float(low_day_1.iloc[0]) if low_day_1 is not None and low_day_2 is not None else None

        # Check condition
        if difference_high is not None and difference_low is not None:
            if difference_high < 0 and difference_low > 0:
                result = "Inside Day"
            elif difference_high > 0 and difference_low < 0:
                result = "Outside Day"
            elif difference_high > 0 and difference_low > 0:
                result = "2 Up"
            elif difference_high < 0 and difference_low < 0:
                result = "2 Down"
            else:
                result = "No Pattern"
            print(f"{ticker} - High difference: {difference_high:.2f}, Low difference: {difference_low:.2f}, Pattern: {result}")
        else:
            print(f"{ticker} - Insufficient data for pattern recognition")

    except Exception as e:
        print(f"{ticker} - Error: {e}") 

The error is:

1 Failed download:
[‘ABT’]: JSONDecodeError(‘Expecting value: line 1 column 1 (char 0)’)
ABT – Not enough data
[100%**] 1 of 1 completed

Ideally I would like to just run the script however the delay is making this very difficult to do.

2

Answers


  1. yfinance has rate limits to prevent abuse and ensure fair usage for all users, exceeding these limits may result in temporary or permanent bans from accessing Yahoo Finance data. I think that’s what’s happening.
    To prevent this when you are testing the code use fewer tickers to reduce the requests.

    One way to minimize the number of requests is to download data for multiple tickers in a single request.

    tickers = ['AAPL', 'MSFT', 'GOOGL']
    data = yf.download(tickers, start='2024-12-01', end='2024-12-15')
    

    This efficiently downloads historical data for multiple tickers in a single call and the downloaded data is returned as a pandas DataFrame.
    To access the data for individual tickers you can use multi-level indexing.

    apple_data = data['Adj Close']['AAPL']
    

    You can also save this DataFrame to a CSV file so until you need to update the data, you don’t have to download it everytime you run the code.

    data.to_csv('sp500_data.csv', index=False)   # data is a pandas DataFrame
    

    To load the CSV file

    file_path = "sp500_data.csv" 
    try:
        data = pd.read_csv(file_path) 
    except FileNotFoundError:
        print(f"Error: File not found: {file_path}")
    

    Note 1: Wikipedia SP500 list can be saved in a file so you don’t have to download it all the time

    import os
    
    def get_sp500_tickers():
        file_path = "tickers.txt"
        if not os.path.exists(file_path):
            tickers = fetch_sp500_tickers()   # if not exist download the tickers from Wikipedia
        
            try:
                with open(file_path, 'w') as file:   # save it to a file
                    for ticker in tickers:
                        file.write(str(ticker) + 'n') 
            except Exception as e:
                print(f"Error saving tickers list to file: {e}")
                
            return tickers
        else:
            try:
                with open(file_path, 'r') as file:   # if exists load it from the file
                    lines = file.readlines()
                    return [line.strip() for line in lines]  # strip newline characters
            except FileNotFoundError:
                print(f"Error: File not found: {file_path}")
                return []   
    

    Note 2: the way you use the try…except is not correct, you should use try…except…else

    try:
        # the code that you suspect might raise an exception
        data_full = yf.download(ticker, start=start_date, end=end_date)
    except Exception as e:
        # executed only if an exception occurs within the try block
        print(f"{ticker} - Error: {e}")
    else:
        # executed only if no exceptions occur within the try block
        if len(data_full) < 2:
            print(f"{ticker} - Not enough data")
            continue
    
        high_day_1 = data_full['High'].iloc[-2] if len(data_full) >= 2 else None
        high_day_2 = data_full['High'].iloc[-1] if len(data_full) >= 1 else None
        ...
    
    Login or Signup to reply.
  2. As Lewis mentioned above yfinance has rate limits to prevent abuse.
    What you are trying to achieve can be done without using the API.
    I have spent quite an amount of time working on alternatives. Please feel free to check out the python code on my github. It offers something similar to what you’re looking for and also has a UI for easy use.
    https://github.com/GerC97/Yahoo-Finance-Data-Python-UI

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search