Question 245593 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – BadRequest when getting presigned object with token and credential values changing

I have a presigned S3 URL that returns a JSON file. I need to use this URL in my Spring Boot application to retrieve the file, so I’m using WebClient.

Here is my code:

@RestController
public class Demo {

    private final WebClient client;

    @Autowired
    public Demo(WebClient.Builder webClientBuilder) {
        this.client = webClientBuilder
                .baseUrl("https://<base_url>.s3.amazonaws.com")
                .build();
    }

    @GetMapping("/data")
    public String processS3URL() {

        return client.get()
                .uri(uriBuilder -> uriBuilder
                        .path("/<urlPath>.json")
                        .queryParam("X-Amz-Security-Token", <securityToken>)
                        .queryParam("X-Amz-Algorithm", "AWS4-HMAC-SHA256")
                        .queryParam("X-Amz-Date", "20230913T103914Z")
                        .queryParam("X-Amz-SignedHeaders", "host")
                        .queryParam("X-Amz-Expires", "172800")
                        .queryParam("X-Amz-Credential", <credential>)
                        .queryParam("X-Amz-Signature", <signature>)
                        .build())
                .retrieve()
                .bodyToMono(String.class)
                .block();
    }
}

However, I’m encountering an issue:

java.lang.Exception: org.springframework.web.reactive.function.client.WebClientResponseException$BadRequest: 400 Bad Request from GET {givenURL}

Upon careful inspection, I noticed that the token and credential values have been altered by WebClient. Specifically, the token and credential values in the error URL differ from the original values I provided.

For instance, a segment of the token in the URL, returned by the error, appears as "HYVG6%252F202309" instead of the original "HYVG6%2F202309". The same issue occurs with the signature.

Why is this happening? How can I retrieve the expected result using WebClient?

Alternatively, are there other methods to retrieve data from a presigned URL?

Tags:

2

Answers


  1. 0

    Two things I would do to investigate –

    1. Check the URI in your browser. Does it give you a response?
    2. Where is the signature being generated?

    All that said, you may just want to use the AWS SDK S3 library to generate the URL and invoke it directly instead of building it from scratch: https://docs.aws.amazon.com/AmazonS3/latest/userguide/example_s3_Scenario_PresignedUrl_section.html

    Login or Signup to reply.
  2. 0

    any other ways to retrieve the data from presigned URL

    Yes, you should ideally be using the AWS SDK for Java to get presigned objects – and as much as possible for any other interaction with AWS – as the SDK handles all of the low-level detail for you.

    There’s an AWS provided example here of how to do so.

    However, if for some reason the SDK is not an option, or if you’re just curious to understand the issue, the remainder of this answer addresses that.

    For instance, a segment of the token in the URL, returned by the error, appears as "HYVG6%252F202309" instead of the original "HYVG6%2F202309".

    Your token and credential values are being double encoded.

    Using the example you’ve provided, your token has already been URL encoded – HYVG6%2F202309. The presence of %2F proves this, as in ASCII, the / character is represented as 2F in hex. Encoding HYVG6/202309 will result in the above value, with / being encoded as %2F.

    The error throws HYVG6%252F202309 as the token value. When you see “%252F”, it means that your “%” character is being encoded again. In ASCII, the % character is represented as 25 in hex. So %2F becomes %252F after the second round of encoding. Encoding HYVG6%2F202309 will result in the above value, with the % being encoded as %25.

    An easy way to fix this would be to first decode the values before passing them to the uriBuilder. This way, when the uriBuilder does its encoding, it will return your values to their original form.

    Or alternatively, you can also change the encoding mode of the WebClient by creating a DefaultUriBuilderFactory with the EncodingMode set to NONE & setting it as the desired factory for the WebClient.Builder.

    As per the source code, the default encoding mode is TEMPLATE_AND_VALUES which encodes the URI template as well as the template values (your token and credential values).

    EncodingMode.NONE will result in no encoding being applied, and the request being sent successfully.

    This should work:

    @Autowired
public Demo(WebClient.Builder webClientBuilder) {
    DefaultUriBuilderFactory factory = new DefaultUriBuilderFactory();
    factory.setEncodingMode(DefaultUriBuilderFactory.EncodingMode.NONE);

    this.client = webClientBuilder
        .baseUrl("https://<base_url>.s3.amazonaws.com")
        .uriBuilderFactory(factory)
        .build();
}
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search