Amazon Web Services Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – Lambda layer's contents don’t match the S3 object after deploying with Terraform

I’m uploading layer.zip to an S3 bucket and then creating the Lambda layer manually by attaching the layer.zip from S3. When I deploy using Terraform, both the S3 object and the Lambda layer version are updated, but the contents of the Lambda layer do not match the layer.zip file in S3. However, when I manually create the Lambda layer by attaching the layer.zip from S3, it works correctly. Can you help me understand why the Lambda layer’s contents don’t match the S3 object after deploying with Terraform?

terraform configuration and logs is something like below.

resource "aws_s3_object" "lambda_layer" {
  bucket              = "cps-request-handler-${var.environment}"
  key                 = "layers.zip"
  source              = "../../../layers.zip"
  etag                = filemd5("../../../layers.zip")
}

resource "aws_lambda_layer_version" "lambda_layer" {
  layer_name          = "cps-request-handler-python"
  s3_bucket           = "cps-request-handler-${var.environment}"
  s3_key              = "layers.zip"
  compatible_runtimes = ["python3.9"]
  description         = "cps-lambda-layer"
  source_code_hash    = filemd5("../../../layers.zip")
}

LOGS-

# module.cps-request-handler-kmg-platform-dev.aws_lambda_layer_version.lambda_layer must be replaced
-/+ resource "aws_lambda_layer_version" "lambda_layer" {
      ~ arn                         = "arn:aws:lambda:ap-southeast-2:24:layer:cps-request-handler-python:54" -> (known after apply)
      ~ code_sha256                 = "vanSi466wd7U=" -> (known after apply)
      - compatible_architectures    = [] -> null
      ~ created_date                = "2024-07-08T03:46:08.644+0000" -> (known after apply)
      ~ id                          = "arn:aws:lambda:ap-sou2:layer:cps-request-handler-python:54" -> (known after apply)
      ~ layer_arn                   = "arn:aws:lambda:ap-south2:layer:cps-request-handler-python" -> (known after apply)
      + signing_job_arn             = (known after apply)
      + signing_profile_version_arn = (known after apply)
      ~ source_code_hash            = "b2b5e003591a1dfa31ec684e" -> "112932d48ce97717736487" # forces replacement
      ~ source_code_size            = 61591646 -> (known after apply)
      ~ version                     = "54" -> (known after apply)
        # (7 unchanged attributes hidden)
    }

  # module.cps-confluent-request-handler-kmg-platform-dev.aws_s3_object.lambda_layer will be updated in-place
  ~ resource "aws_s3_object" "lambda_layer" {
      ~ etag                          = "e1f4a9f539372def76d5307d9" -> "112932d48ce97717736487"
        id                            = "layers.zip"
        tags                          = {}
      ~ version_id                    = "3HU.4GNHsj.iASru5O" -> (known after apply)
        # (23 unchanged attributes hidden)
    }

when I manually create the Lambda layer by attaching the layer.zip from S3, it works correctly. I expect the samething to work in terraform

Tags:

2

Answers


  1. 0

    In your current configuration there is no information for Terraform to infer that aws_s3_object.lambda_layer must be created or updated before aws_lambda_layer_version.lambda_layer, and so it’s possible that Terraform will make both of these requests concurrently and the Lambda layer might be created before the S3 object has been updated, causing the layer to capture an older version of that object.

    You can explain the correct order to Terraform by making the aws_lambda_layer_version configuration refer to the aws_s3_object values:

    resource "aws_s3_object" "lambda_layer" {
  bucket              = "cps-request-handler-${var.environment}"
  key                 = "layers.zip"
  source              = "../../../layers.zip"
  etag                = filemd5("../../../layers.zip")
}

resource "aws_lambda_layer_version" "lambda_layer" {
  layer_name          = "cps-request-handler-python"
  s3_bucket           = aws_s3_object.lambda_layer.bucket
  s3_key              = aws_s3_object.lambda_layer.key
  compatible_runtimes = ["python3.9"]
  description         = "cps-lambda-layer"
  source_code_hash    = aws_s3_object.lambda_layer.etag
}

    Terraform uses the references between objects to understand the dependencies. In this example aws_lambda_layer_version.lambda_layer‘s configuration refers to aws_s3_object.lambda_layer and so Terraform will infer that any actions needed for the S3 object must be complete before starting the actions related to the layer version.

    This structure also has the advantage for you that you don’t need to duplicate the same information in both resources: you can update the bucket name and object key in aws_s3_object.lambda_layer only in future and the layer version resource will automatically follow those changes.

    Login or Signup to reply.
  2. 0

    First, you should confirm the problem actually exists. Within the lambda console, you can view the lambda’s SHA256 BASE64 encoded hash. You can observe it before and after. As well, this SHA256 BASE64 encoded hash appears in CloudTrail when a lambda is created/updated.

    Having said that and looking deeper, the contents of the zip file could be identical, but if it’s zipped by different zipping software or even at different times, the SHA256 will almost certainly be different. Some zipping solutions (Such as Terraform’s Archive provider) can produce exact ZIP files over and over, but others cannot.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search