Amazon Web Services Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – How to use nextToken to paginate?

Some context is needed for my question at the bottom of the post. There are two tables in the DynamoDB with the following schema.

type Post @model {
  id: ID!
  author: String!
  comments: [Comment] @hasMany
}

type Comment @model {
  id: ID!
  description: String!
  post: Post! @belongsTo
}

The Post table has 125 posts. The Comment table has 3000+ comments. Fetching all 125 Posts takes less than 5 seconds. Fetching all 3000+ comments with a limit of 1000 and paginating takes more than 10 seconds.

In each Post object, it contains a limit of 100 comments. Assuming each Post has 100+ comments, when we fetch all 125 posts, we have 1,250 of 3000+ comments within 5 seconds versus waiting greater than 10 seconds when we fetch all comments from the Comment table.

Code for fetching from Comment Table where it takes greater than 10 seconds.

    do {
      const result = await API.graphql(graphqlOperation(listComments, { limit: 1000, nextToken }));
      const tokens = result.data.listComments.items;
      allEvents.push(...tokens);
      nextToken = result.data.listComments.nextToken;
    } while (nextToken);

In order to speed up the time it takes for our app to get all the comments (the goal), I tried extracting all comment data from the Post objects since we get all Post objects much faster. The thinking was if we get 1250 comments from the Posts that quickly, we could just take the nextToken from the Post.comments.nextToken. (array of 100 comments is at Post.comments.items), and hit that same graphql function above. However when doing that, it returns the following pagination error.

 error in fetchAllCommentsAPI: {"data":{"listComments":null},"errors":[{"path":["listComments"],"data":null,"errorType":"IllegalArgument","errorInfo":null,"locations":[{"line":2,"column":3,"sourceName":null}],"message":"Invalid pagination token given."}]}

I thought it might be because we need to add a filter for the specific Post we are fetching for but that gave the same error.

Adding a filter to my nextToken graphql call causes an error:

const result = await API.graphql(graphqlOperation(listComments, { filter: { commentPostId: { eq: postId } }, nextToken })); // where commentPostId is the id of the Post.

I dont understand why I am getting that error, I would think the nextToken would take you to that filtered page.

Questions

  1. How can I use nextToken when fetching comments while filtering for a specific post?

  2. Is there a better way to get all comment data efficiently / is extracting from Post objects bad practice?

Tags:

2

Answers


  1. 0

    The token from a filtered query can only be used with the same filter parameters. When you get Post.comments.nextToken, it’s specific to the pagination of the relationship query in the Post table. And when you are trying to use this token with listComments fails because it’s a different query context altogether. You can try to implement something below to get comments for a specific post using Post table pagination : –

    const getPostWithComments = async (postId) => {
  let nextToken = null;
  let allComments = [];
  
  do {
    const result = await API.graphql(
      graphqlOperation(getPost, {
        id: postId,
        commentsLimit: 100,
        commentsNextToken: nextToken
      })
    );
    
    const post = result.data.getPost;
    allComments.push(...post.comments.items);
    nextToken = post.comments.nextToken;
  } while (nextToken);
  
  return allComments;
};
    1. Using Post.comments in the initial Post query to fetch comments is good since it’s faster and efficient (5s vs. 10s). as you are getting the data as part of your Post query anyway. But for subsequent data(comment), would recommend to implement pagination for loading additional comments when and only needed, maybe when users scroll or request for more.
    Login or Signup to reply.
  2. 0 
    const getCommentsForPost = async (postId) => {
  // This is assuming that the postId is not null
  let nextToken = null;
  let allComments = [];

  do {
    const result = await API.graphql(
    graphqlOperation(listComments, { filter: { 
       commentPostId: { eq: postId } 
    // No need for limits
      }, 
     nextToken 
     })
   );

const comments = result.data.listComments.items;
allComments.push(...comments);
nextToken = result.data.listComments.nextToken;
} while (nextToken);

return allComments;
};

    This should get the comments for a particular post without any error. Also could you clarify if you are getting the comments only when you view a single post details, or you are trying to get multiple posts comments.

    Either way, this function can still get the comments that you need

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search