Shopify Question posted in
The Shopify help center can be found here.

How to query or export ALL Shopify customers or orders with rate limiting

Im curious to what would be the best way if i wanted to export all Customers or Orders that meet a certain criteria as json or csv etc., considering the rate limit and pagination factors. I am guessing it would be some kind of looping script that would query a max of 250 records and repeat that until the pagination is done. Is there any code out there that shows this, i feel like with a lot of apps this would be useful or needed to do certain things. It’s more or less the export feature in shopify, but doing it in your own app.

Tags:

2

Answers


  1. 0

    Shopify has a nice API module for Node.js. It offers easy pagination handling and rate limiting. From the perspective of app, you may decide to use some queue mechanism. As these processes may take longer to complete based on number of orders or customers, whenever a client requests a record export, push the job in a queue and notify once done.

    For fetching orders or customers using API while taking care of rate limiting, following code from Shopify API Node Docs should suffice.

    const Shopify = require("shopify-api-node");

const shopify = new Shopify({
  shopName: "your-shop-name",
  apiKey: "your-api-key",
  password: "your-app-password",
  autoLimit: true,
});

(async () => {
  let params = { limit: 250 };
  do {
    const products = await shopify.product.list(params);
    console.log(products);
    params = products.nextPageParameters;
  } while (params !== undefined);
})().catch(console.error);

    autoLimit – Optional – This option allows you to regulate the request
    rate in order to avoid hitting the rate limit. Requests are limited
    using the token bucket algorithm. Accepted values are a boolean or a
    plain JavaScript object. When using an object, the calls property and
    the interval property specify the refill rate and the bucketSize
    property the bucket size. For example { calls: 2, interval: 1000,
    bucketSize: 35 } specifies a limit of 2 requests per second with a
    burst of 35 requests. When set to true requests are limited as
    specified in the above example. Defaults to false.

    Once done, you can write them as csv or JSON or anything else as needed. For queues, you may use something like Bull or Agenda.

    Login or Signup to reply.
  2. 0

    The superior way to get the data you want is to use GraphQL and the Bulk Queries. With those you specify what you want in terms of Customer or Order data. Then you ask for the data with your query, and Shopify then packages it all up in a complete JSONL file. You it around and wait till they complete the package, at which point you get a download URL where you can have ALL the data.

    Paging is a crappy way to go about exporting data. It works sure, but it is slow, and subject to network errors as anyone who has worked much with Shopify will tell you.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search