Question 95673 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

AWS IAM Role for Service Account with EBS CSI driver: could not create volume in EC2: NoCredentialProviders: no valid providers in chain – Amazon web services

I have successfully installed the AWS EBS CSI driver to my EKS cluster.

This is meant to be using the "IAM Role for Service Account" technique.

I am trying to utilise the checkout example app that AWS have given here
The pod will not come up (pending) and the PVC is showing this:

Name:          ebs-claim
Namespace:     test
StorageClass:  ebs-sc
Status:        Pending
Volume:        
Labels:        app=ebs-claim
               com.mylabel.contact=dl-myteam.dlonp1
Annotations:   volume.beta.kubernetes.io/storage-provisioner: ebs.csi.aws.com
               volume.kubernetes.io/selected-node: ip-10-232-100-115.ec2.internal
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      
Access Modes:  
VolumeMode:    Filesystem
Used By:       meme-ebs
Events:
  Type     Reason              Age               From                                                                                      Message
  ----     ------              ----              ----                                                                                      -------
  Warning  ProvisioningFailed  27s               persistentvolume-controller                                                               storageclass.storage.k8s.io "ebs-sc" not found
  Normal   Provisioning        8s (x4 over 25s)  ebs.csi.aws.com_ebs-csi-controller-6dfdb77cdf-fbsbz_1760973c-09bb-43ab-b005-ffcd818447fc  External provisioner is provisioning volume for claim "test/ebs-claim"
  Warning  ProvisioningFailed  5s (x4 over 22s)  ebs.csi.aws.com_ebs-csi-controller-6dfdb77cdf-fbsbz_1760973c-09bb-43ab-b005-ffcd818447fc  failed to provision volume with StorageClass "ebs-sc": rpc error: code = Internal desc = Could not create volume "pvc-05efbff8-9506-4003-9bab-e1ce4719bc1c": could not create volume in EC2: NoCredentialProviders: no valid providers in chain
caused by: EnvAccessKeyNotFound: failed to find credentials in the environment.
SharedCredsLoad: failed to load profile, .
EC2RoleRequestError: no EC2 instance role found
caused by: EC2MetadataError: failed to make EC2Metadata request

Similar to an issue I saw here, but had no answers.

Can anyone suggest things to try? Seems like the IAM role is not wired thru to the API that mounts the volume on EC2?

Tags:

2

Answers


  1. 0

    Looks like an issue with the service account that your efs csi driver is using. For example, make sure it’s using the right role with the right trust policy for your EKS cluster. For example check the right annotation below:

    apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/name: aws-efs-csi-driver
  name: efs-csi-controller-sa
  namespace: kube-system
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::111122223333:role/AmazonEKS_EFS_CSI_DriverRole

    And the role that you are using has the right trust policy

    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::111122223333:oidc-provider/oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub": "system:serviceaccount:kube-system:efs-csi-controller-sa"
        }
      }
    }
  ]
}

    The instructions here are pretty clear. (It worked for me)

    Login or Signup to reply.
  2. 0

    Ensure that the created ServiceAccounts have the correct IRSA annotations.

    If you are using the helm chart, and doing an upgrade from an older version, double check the location of the IRSA ServiceAccount annotation (they may have changed, had me stumped for a bit as to why things didn’t work).

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search