Question 58175 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Getting Crc32MismatchException when using dynamodb enhanced client when we are using queryPaginator – Amazon Web Sevices

Bug description

We tried to queryPaginator using DynamoDbEnhancedClient but we got software.amazon.awssdk.core.exception.Crc32MismatchException.

software.amazon.awssdk.core.exception.Crc32MismatchException: Expected 4178815326 as the Crc32 checksum but the actual calculated checksum was 3264908315

Expected Behavior
I should be able to queryPaginator results

Current Behavior

Error logs:

“exception”:”software.amazon.awssdk.core.exception.Crc32MismatchException: Expected 4178815326 as the Crc32 checksum but the actual calculated checksum was 3264908315ntat software.amazon.awssdk.core.exception.Crc32MismatchException$BuilderImpl.build(Crc32MismatchException.java:88)ntat software.amazon.awssdk.core.internal.util.Crc32ChecksumValidatingInputStream.validateChecksum(Crc32ChecksumValidatingInputStream.java:62)ntat software.amazon.awssdk.core.internal.util.Crc32ChecksumValidatingInputStream.close(Crc32ChecksumValidatingInputStream.java:50)ntat java.base/java.io.FilterInputStream.close(FilterInputStream.java:179)ntat software.amazon.awssdk.utils.FunctionalUtils.lambda$safeRunnable$5(FunctionalUtils.java:124)ntat software.amazon.awssdk.utils.FunctionalUtils.invokeSafely(FunctionalUtils.java:140)ntat software.amazon.awssdk.protocols.json.internal.unmarshall.JsonResponseHandler.lambda$handle$2(JsonResponseHandler.java:87)ntat java.base/java.util.Optional.ifPresent(Optional.java:178)ntat software.amazon.awssdk.protocols.json.internal.unmarshall.JsonResponseHandler.handle(JsonResponseHandler.java:87)ntat software.amazon.awssdk.protocols.json.internal.unmarshall.JsonResponseHandler.handle(JsonResponseHandler.java:36)ntat software.amazon.awssdk.protocols.json.internal.unmarshall.AwsJsonResponseHandler.handle(AwsJsonResponseHandler.java:43)ntat software.amazon.awssdk.core.http.MetricCollectingHttpResponseHandler.lambda$handle$0(MetricCollectingHttpResponseHandler.java:52)ntat software.amazon.awssdk.core.internal.util.MetricUtils.measureDurationUnsafe(MetricUtils.java:63)ntat software.amazon.awssdk.core.http.MetricCollectingHttpResponseHandler.handle(MetricCollectingHttpResponseHandler.java:52)ntat software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle(AwsSyncClientHandler.java:95)ntat software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$resultTransformationResponseHandler$8(BaseClientHandler.java:262)ntat software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleSuccessResponse(CombinedResponseHandler.java:99)ntat software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:75)ntat software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:60)ntat software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:41)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:30)ntat software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:50)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:36)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)ntat software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)ntat software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)ntat software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:48)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:31)ntat software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)ntat software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)ntat software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)ntat software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:193)ntat software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)ntat software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:167)ntat software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:82)ntat software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:175)ntat software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:76)ntat software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)ntat software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:56)ntat software.amazon.awssdk.services.dynamodb.DefaultDynamoDbClient.query(DefaultDynamoDbClient.java:4046)ntat software.amazon.awssdk.services.dynamodb.paginators.QueryIterable$QueryResponseFetcher.nextPage(QueryIterable.java:134)ntat software.amazon.awssdk.services.dynamodb.paginators.QueryIterable$QueryResponseFetcher.nextPage(QueryIterable.java:125)ntat software.amazon.awssdk.core.pagination.sync.PaginatedResponsesIterator.next(PaginatedResponsesIterator.java:58)ntat java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)ntat java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)ntat java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)ntat java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)ntat java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)ntat java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)ntat java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:693)ntat

AWS Java SDK version used and Java Version used

2.17.27

java 17

Snipped code:

 AwsBasicCredentials credentials = AwsBasicCredentials.create(getAccessKey(), getSecretKey());

 DynamoDbClient.builder()
    .region(Region.of(getRegion()))                    
    .endpointOverride(URI.create(getEndpoint()))                    
    .credentialsProvider(StaticCredentialsProvider.create(credentials))
    .build();

Related same issue but we are unable to find the solution:

Getting Crc32MismatchException when using dynamodb enhanced client

https://github.com/aws/aws-sdk-java-v2/issues/3703

https://sdk.amazonaws.com/java/api/2.0.0/software/amazon/awssdk/core/exception/Crc32MismatchException.html

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    Possible Solution

    DynamoDbClient.builder().region(Region.of(getRegion())).endpointOverride(URI.create(getEndpoint())).credentialsProvider(StaticCredentialsProvider.create(credentials)).overrideConfiguration(o -> o.putHeader("Accept-Encoding", "gzip")).build();

  2. 0

    You can build an Enhanced Client using the below Java code. There is no need to use AwsBasicCredentials and hard code your credentials. Also – in your Java code, you are not using DynamoDbEnhancedClient.

        package com.example.dynamodb.enhanced;

// snippet-start:[dynamodb.java2.mapping.query.import]

import com.example.dynamodb.Customer;
import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbEnhancedClient;
import software.amazon.awssdk.enhanced.dynamodb.DynamoDbTable;
import software.amazon.awssdk.enhanced.dynamodb.Key;
import software.amazon.awssdk.enhanced.dynamodb.TableSchema;
import software.amazon.awssdk.enhanced.dynamodb.model.QueryConditional;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
import software.amazon.awssdk.services.dynamodb.model.DynamoDbException;

import java.util.Iterator;
// snippet-end:[dynamodb.java2.mapping.query.import]

/*
 * Before running this code example, create an Amazon DynamoDB table named Customer with these columns:
 *   - id - the id of the record that is the key
 *   - custName - the customer name
 *   - email - the email value
 *   - registrationDate - an instant value when the item was added to the table
 *
 * Also, ensure that you have set up your development environment, including your credentials.
 *
 * For information, see this documentation topic:
 *
 * https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
 */

public class EnhancedQueryRecords {

    public static void main(String[] args) {

        ProfileCredentialsProvider credentialsProvider = ProfileCredentialsProvider.create();
        Region region = Region.US_EAST_1;
        DynamoDbClient ddb = DynamoDbClient.builder()
            .region(region)
            .credentialsProvider(credentialsProvider)
            .build();

        queryTable(ddb );
        ddb.close();
    }

  public static void queryTable(DynamoDbClient ddb) {
    try {
        DynamoDbEnhancedClient enhancedClient = DynamoDbEnhancedClient.builder()
            .dynamoDbClient(ddb)
            .build();

        DynamoDbTable<Movies> movieTable = enhancedClient.table("Movies", TableSchema.fromBean(Movies.class));
        QueryConditional queryConditional = QueryConditional
            .keyEqualTo(Key.builder()
            .partitionValue(2013)
            .build());

        // Write out Movie information. 
        Iterator<Movies> results = movieTable .query(queryConditional).items().iterator();
        while (results.hasNext()) {
            Movies rec = results.next();
            System.out.println("The title of the movie is "+rec.getTitle());
            System.out.println("The movie information  is "+rec.getInfo());
        }

    } catch (DynamoDbException e) {
        System.err.println(e.getMessage());
        System.exit(1);
        }
    }

    In this example, the credentials are handled by using the shared credentials and config files that is documented here:

    Default credentials provider chain

    As far as paginated results and the Enhanced Client, look at the Java docs here:

    Interface DynamoDbTable

    Unlike the low level DynamoDB client that has a paginated method, the enhanced client does not – as shown in the Javadoc.

    The enhanced client returns your data in Java objects.

    You can find this complete example here:

    Get started with DynamoDB tables, items, and queries using an AWS SDK

    You can read more about the Enhanced Client in the Java V2 Developer Guide here:

    Mapping items in DynamoDB tables

    I have provided everything you need to successfully create Java apps using the Enhanced Client.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search