skip to Main Content

I’m trying to access S3 from AWS lambda and I keep getting an access denied. The lambda code has been deployed, and I run an application on my PC that invokes said lambda. The lambda is meant to read 1 files from S3, modify it, and write it to a different location in S3.

I have run this code without the lambda i.e. the logic that goes into the lambda can reside on my PC. That works i.e. I run the code on my PC. The code fetches the file from S3, modifies it, and saves it, successfully. However, the problem occurs when this same logic is deployed as a lambda. I therefore believe it to be some configuration problem, but I have no idea what.

Here’s my inline s3 policy for the lambda execution role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::MY_BUCKET/*"
        }
    ]
}

Here’s the policty configuration for the S3 bucket:

{
    "Version": "2012-10-17",
    "Id": "PolicyNumber",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::ACCOUNT_ID:role/web_functions"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::MY_BUCKET/*"
        }
    ]
}

Note that I’m using "s3:*" for now. I’ve been trying various combinations and s3:* is only being used until I can get something to work.

Additionally (if it matters):

  • The runtime is Go 1.x
  • Arch: x86_64
  • There is no VPC configured
  • For the lambda execution role, there are 2 other policies. 1 is the default policy added at function creation time for logging. The other was added when I enabled X-Ray monitoring.
  • My S3 bucket is not publicly accessible.
  • The app on my PC runs using an IAM role with admin permissions. This is different from the lambda execution role.

I’ve looked at other posts answering this question and I think I’ve covered everything, but something is still wrong. Please let me know if you find any problem I missed.

Any help will be greatly appreciated

Edit: As requested, I’m sharing a portion of the code and the error message:

func getS3Client(region string) *s3.Client {
    //region is a string (not aws.String()): "us-east-2"
    client := s3.New(s3.Options{
        Region: region,
    })
    return client
}

func getFile(bucket string, filename string) {
        client := getS3Client(region)
        object, err := client.GetObject(context.TODO(), &s3.GetObjectInput{
            Bucket: aws.String(bucket),
            Key:    aws.String(filename),
        })
        if err != nil {
            return nil, errors.New(fmt.Sprintf("Error reading s3 object: %v (bucket=%s, filename=%s)n", err, bucket, filename))
    }   
}

And here’s the error as reported by golang:

{"errorMessage":"Error reading s3 object: operation error S3: GetObject, https response error StatusCode: 403, RequestID: QEWV1F12YHCT862X, HostID: knxgns9O+oCTJ9G2p+hXNEyZJqjeTup/25Ab6Ih6jOrN/FAZvH7gjxKb0JEp3rGQMVyzqdgu30s=, api error AccessDenied: Access Denied (bucket=MY_BUCKET, filename=DCIM/061121/DSC01033.JPG)n","errorType":"errorString"}

And here’s a sample log from testing the lambda within the AWS console:

START RequestId: 33fe5480-bbfc-42e6-bce8-e5d3589d1187 Version: $LATEST
bucket: MY_BUCKET, filename: DCIM/061121/DSC01423.JPG, region: us-east-2
Error reading s3 object: operation error S3: GetObject, https response error StatusCode: 403, RequestID: KDCZ6XBVSNHX6HE1, HostID: FG1nTNI9MAwh2HhwBfRCE0Xj355RefzsSy1ludhM52rls6yzu+/ElgCeB27Q4i6enjPlX8HaGQ8=, api error AccessDenied: Access Denied (bucket=MY_BUCKET, filename=DCIM/061121/DSC01423.JPG)
: errorString
null
END RequestId: 33fe5480-bbfc-42e6-bce8-e5d3589d1187
REPORT RequestId: 33fe5480-bbfc-42e6-bce8-e5d3589d1187  Duration: 132.27 ms Billed Duration: 133 ms Memory Size: 512 MB Max Memory Used: 39 MB  Init Duration: 98.39 ms 

2

Answers


  1. Chosen as BEST ANSWER

    Solved

    The permissions were fine. I didn't even need the permissions on the S3 bucket (it works with/without those permissions). The error is with the following line of code:

    client := s3.New()
    

    Apparently this call doesn't inherit things like service role values. However, the following works:

    cfg, _ := config.LoadDefaultConfig(context.TODO())
    client := s3.NewFromConfig(cfg)
    

    Since I was calling the service from a lamba and not my local PC, I used s3.New() since I didn't have a credentials file, and I wasn't explicitly setting any environment variables. I wrongly assumed that s3.New() will use the role configured for the lambda. s3.NewFromConfig() does that correctly.

    Hope this saves someone else a bunch of time.


  2. I am assuming that role attached to lambda function is arn:aws:iam::ACCOUNT_ID:role/web_functions
    According to me Lambda function is missing s3:ListBucket permission & that’s the reaseon it showing 403 error.
    I would suggest to try below policy. Its having same permission you gave along with s3:ListBucket permission.

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "VisualEditor0",
          "Effect": "Allow",
          "Action": [
            "s3:GetObject",
            "s3:PutObject"
          ],
          "Resource": "arn:aws:s3:::MY_BUCKET/*"
        },
        {
          "Sid": "VisualEditor1",
          "Effect": "Allow",
          "Action": "s3:ListBucket"
          "Resource": "*"
        }
      ]
    }
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search