Question 295947 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – AWS ListObjectsV2 runs indefinitely

I am using AWS ListObjectsV2 to iterate over S3 keys. There are more than 1,000 keys, so the output returns me IsTruncated=true as it should. However the ContinuationToken is empty which makes the loop in my code to run indefinitely.

In case of IsTruncated=true the ContinuationToken should never be Null/Empty.

Am I missing something or is it a bug on the AWS API?

Tags:

2

Answers


  1. 0

    I’m not sure of your particular issue, but this might help. Many AWS clients in the SDKs contain a paginator to help with paging operations. Have you tried using ListObjectsV2Paginator? This is an example of how it can be used in Go.

    // Create the Paginator for the ListObjectsV2 operation.
p := s3.NewListObjectsV2Paginator(client, params, func(o *s3.ListObjectsV2PaginatorOptions) {
    if v := int32(maxKeys); v != 0 {
        o.Limit = v
    }
})

// Iterate through the S3 object pages, printing each object returned.
var i int
log.Println("Objects:")
for p.HasMorePages() {
    i++

    // Next Page takes a new context for each page retrieval. This is where
    // you could add timeouts or deadlines.
    page, err := p.NextPage(context.TODO())
    if err != nil {
        log.Fatalf("failed to get page %v, %v", i, err)
    }

    // Log the objects found
    for _, obj := range page.Contents {
        fmt.Println("Object:", *obj.Key)
    }
}
    Login or Signup to reply.
  2. 0

    You should be looking at NextContinuationToken:

    NextContinuationToken is sent when isTruncated is true, which means there are more keys in the bucket that can be listed. The next list requests to Amazon S3 can be continued with this NextContinuationToken. NextContinuationToken is obfuscated and is not a real key

    ContinuationToken is a copy of what you sent along with the request. If you didn’t send a continuation token, it won’t get returned either.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search