Question 242460 in Shopify Question posted in
The Shopify help center can be found here.

Python code with Shopify pagination stops midway

I have this code where I want to get the Id and the Inventory item Id of all my products.
The problem is that when it arrives at 2000 products aprox it stops iterating the pagination. I don’t know if it’s a cap of the API or my code isn’t ok.

import requests
import csv

API_KEY = "key"
PASSWORD = "password"
SHOP_HANDLE = "name"
API_VERSION = '2023-07'  # Actualiza la versión de la API si es necesario
CSV_FILE_PATH = 'product_inventory_item_ids.csv'

def get_all_products():
    page_info = None
    all_products = []

    while True:
        url = f"https://{SHOP_HANDLE}.myshopify.com/admin/api/{API_VERSION}/products.json?limit=250"
        if page_info:
            url += f"&page_info={page_info}"

        response = requests.get(url, auth=(API_KEY, PASSWORD))

        if response.status_code != 200:
            print(f'Error al obtener productos: {response.text}')
            break

        products_data = response.json()

        if not products_data['products']:
            break

        all_products.extend(products_data['products'])
        page_info = products_data.get('next_page_info')

        if not page_info:
            break

    return all_products

def main():
    with open(CSV_FILE_PATH, 'w', newline='') as csv_file:
        csv_writer = csv.writer(csv_file)
        csv_writer.writerow(['Product ID', 'Inventory Item ID'])  # Encabezados del archivo CSV
        
        for product in get_all_products():
            product_id = product['id']
            inventory_item_id = product['variants'][0]['inventory_item_id']

            csv_writer.writerow([product_id, inventory_item_id])

    print('Proceso completado. Los datos se han guardado en el archivo CSV.')

if __name__ == "__main__":
    main()

I managed to use the pagination to get those 2000 products but it stops there and I just don’t know why.

Tags:

2

Answers


  1. 0

    You know you can solve this much easier with a single Bulk Export query? Your bulk export result would be a list of all your products! As a bonus, each product would also dump the variant info you want, including its item inventory ID. Finally, you get this as a downloadable JSON file from Shopify!

    While paging is fun for the whole family and wholesome, bulk exports are far superior for these kinds of tasks.

    Login or Signup to reply.
  2. 0

    You need to throttle. This is how I do it

        def retrieve_all( query_provider, data_retriever):
        has_next_page = True
        edges = []
        cursor = None
        time_to_sleep = -1

        while has_next_page:
            if time_to_sleep > 0:
                time.sleep(time_to_sleep)
            query = query_provider(cursor)
            result_ = json.loads(shopify.GraphQL().execute(query))

            throttle = result_["extensions"]["cost"]
            current_points = int(throttle["throttleStatus"]["currentlyAvailable"]) - 200
            last_score = int(throttle["requestedQueryCost"])
            time_to_sleep = (last_score - current_points) / 100

            if 'errors' in result_:
                if result_['errors'][0]['extensions']['code'] == 'THROTTLED':
                    continue
                else:
                    raise Exception(result_['errors'][0]['message'])

            data = data_retriever(result_)
            has_next_page = data['pageInfo']['hasNextPage']
            if len(data['edges']) == 0 and has_next_page:
                raise Exception("You have permission problem going that far")

            edges = edges + data['edges']
            if len(edges) > 0:
                cursor = edges[-1]['cursor']
        return edges

    And you call this like

            def query_provider(cursor):
            after = ""
            if cursor:
                after = ''', after: "%s"''' % cursor
            with open("yourquery.graphql", 'r') as fp:
                return fp.read() % after

retrieve_all(query_provider, lambda s: s['data']['inventoryItems'])
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search