skip to Main Content

I’m trying to migrate Several spring boot services to EKS and they can’t retrieve aws credentials from credentials chain and pods are failing with following error: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain

These are what I’ve tried so far:

I’m using Web identity token from AWS STS for credentials retrieval.

@Bean
public AWSCredentialsProvider awsCredentialsProvider() {
    if (System.getenv("AWS_WEB_IDENTITY_TOKEN_FILE") != null) {
        return WebIdentityTokenCredentialsProvider.builder().build();
    }
    return new DefaultAWSCredentialsProviderChain();
}

@Bean
public SqsClient sqsClient(AWSCredentialsProvider awsCredentialsProvider) {
    return SqsClient
            .builder()
            .credentialsProvider(() -> (AwsCredentials) awsCredentialsProvider.getCredentials())
            .region(Region.EU_WEST_1).build();
}

@Bean
public SnsClient snsClient(AWSCredentialsProvider awsCredentialsProvider) {
    return SnsClient
            .builder()
            .credentialsProvider(() -> (AwsCredentials) awsCredentialsProvider.getCredentials())
            .region(Region.EU_WEST_1).build();
}

The services also have aws-java-sdk-sts maven dependency packaged.

IAM role for the services is also fine and AWS_WEB_IDENTITY_TOKEN_FILE is a also automatically created within pod after each Jenkins build based on K8s manifest file.

From pod I can make GET and POST request to SNS and SQS without any problem.

3

Answers


  1. Chosen as BEST ANSWER

    Problem was fixed.

    Main issue was conflicting AWS SDK BOM version with individual models. Also previous version of BOM I was using wasn't supporting AWS SDK v2.x .

    These are the main take aways from the issue:

    1. AWS SDK authenticate services using credentials provider chain . The default credential provider chain of the AWS SDK for Java 2.x searches for credentials in your environment using a predefined sequence.

      1.1 As of AWS SDK for Java 2.x Web identity token from AWS STS is within default provider chain.

      1.2 As long as using v2 of the SDK and having the STS dependency makes explicit configuration of Web identity token redundant.

      1.3 Make sure candidate service is using AWS SDK v2 as it’ll reduce the configuration code to minimum.

    If a candidate service using AWS SDK v1 following configuration should be added as Web identity token isn’t in default provider chain for v1.

    @Bean
    public AWSCredentialsProvider awsCredentialsProvider() {
        if (System.getenv("AWS_WEB_IDENTITY_TOKEN_FILE") != null) {
            return WebIdentityTokenCredentialsProvider.builder().build();
        }    
        return new DefaultAWSCredentialsProviderChain();
    }
    

    Last but not least try to use try to use latest AWS SDK BOM dependency . (currently all modules have the same version, but this may not always be the case)


  2. You should have roleArn, sessionname and token details in the identity token cred provider build.

    Try this

    return  WebIdentityTokenCredentialsProvider.builder()
            .roleArn(System.getenv("AWS_ROLE_ARN"))
            .roleSessionName(System.getenv("AWS_ROLE_SESSION_NAME"))
            .webIdentityTokenFile(System.getenv("AWS_WEB_IDENTITY_TOKEN_FILE"))
            .build();
    

    than just returning as return WebIdentityTokenCredentialsProvider.builder().build();

    Login or Signup to reply.
  3. You can try to create the file:

    • Windows: C:Users[username].awsconfig
    • Mac: /Users/[username]/.aws/config
    • Linux: /home/[username]/.aws/config

    and add an AWS credential to it.
    Ex:

    [default]
    aws_access_key_id = key_value
    aws_secret_access_key = secret_value
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search