skip to Main Content

I want to build a scrapper. And I wrote this code, it keeps on running into error. I would be grateful if anyone can help.

import requests
import pandas as pd

def fetch_json(api_link, timeout=10):
    try:
        response = requests.get(api_link, timeout=timeout)
        if response.status_code == 200:
            return response.json()
        else:
            print(f"Failed to fetch data from API for link: {api_link}")
            return None
    except requests.exceptions.Timeout:
        print(f"Timeout occurred while fetching data for link: {api_link}. Skipping...")
        return None
    except requests.exceptions.RequestException as e:
        print(f"Error occurred while fetching data for link {api_link}: {e}")
        return None

api_link = "https://apiv2.cricket.com.au/web/players/list?isActive=1&teamId=&jsconfig=eccn%3Atrue&format=json"

player_data= fetch_json(api_link)

name = []

if player_data:
    players = player_data['players']
    for player in players:
        id = "".join(players['id'])
        displayname = "".join(players['displayname'])
        name.append({
            "id" : player.get("id"),
            "displayname" : player.get("displayname")
        })
    print(f"Player Name Retrieved")
    
player_name = pd.DataFrame(name)

It shows error along the id column and I can’t seem to solve this problem.

2

Answers


  1. As mentioned in the comment, your code has some simple typos when referring to the values in the json data. For each player in the list of players use:

    player['id']
    player['displayName']

    name.append(
            {"id" : player['id'],
            "displayname" : player['displayName']
            })
    
    Login or Signup to reply.
  2. This is the sample data:

    {'id': 2786, 'firstName': 'Khaliq', 'lastName': 'Noori', 'middleName': 'Dad', 'displayName': 'Khaliq Dad', 'nationality': '', 'gender': '', 'teams': [{'id': 6, 'name': 'Afghanistan Men', 'isWomensTeam': False}], 'isActive': True}

    As you can see, you need to change from "players" and "displayname" to "players" and "displayName" respectively. Also, there is no need to use join here.

    if player_data:
        players = player_data['players']
        for player in players:
            id = player['id']
            displayname = player['displayName']
            name.append({
                "id": id,
                "displayname": displayname
            })
        print("Player Names Retrieved")
        
    player_name = pd.DataFrame(name)
    
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search