Amazon Web Services Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – Exclude directory from terraform archive file for lambda function?

I have a lambda function defined via a terraform file. When I terraform apply it, I get an error that the resultant .zip file is larger than the maximum size allowable for a lambda file. Initially this was confusing as the two expected components (the .py defining the function and the .tf defining the infrastructure) are a few KB each, much less than the 50 MB limit. unzip -l lambda_function_add_user_post.zip displays the following the following information:


    Length      Date    Time    Name
   ---------  ---------- -----   ----
    15144690  01-01-2049 00:00   .terraform/providers/registry.terraform.io/hashicorp/archive/2.4.2/darwin_arm64/terraform-provider-archive_v2.4.2_x5
   444405970  01-01-2049 00:00   .terraform/providers/registry.terraform.io/hashicorp/aws/5.35.0/darwin_arm64/terraform-provider-aws_v5.35.0_x5
         213  01-01-2049 00:00   .terraform.tfstate.lock.info
        2466  01-01-2049 00:00   lambda-addUserPost.tf
         891  01-01-2049 00:00   lambdaFunctions.py
    99589452  01-01-2049 00:00   lambda_function_add_user_post.zip

First question… why does the unzip -l list the .zip file itself? Is it somehow recursively nested? How is this even possible?

Second question: The two darwin_arm64 files are clearly causing the size problem. Its probably simpler to simply hard code their full paths into the excludes, but it would be useful in the future to understand how to exclude whole directories.

The fileset documentation online indicates I must use this function, but I am unable to apply it correctly. Here is my attempt:

data "archive_file" "lambda_function_add_user_post" {
  type          = "zip"
  source_dir    = "${path.module}"
  output_path   = "${path.module}/lambda_function_add_user_post.zip"
    
  excludes = 
  [
       fileset("${path.module}/.terraform", "**"),
       ".terraform.lock.hcl",
       "terraform.tfstate",
       "terraform.tfstate.backup",
       "venv",
       "venv/*"
  ]
}

This yields the following:

Error: Invalid expression
│
│ on lambda-addUserPost.tf line 85, in data "archive_file" "lambda_function_add_user_post":
│ 85: excludes =
│ 86: [
│
│ Expected the start of an expression, but found an invalid expression token.

If I comment out the fileset line then it works (but I get an error re: the .zip being too large instead).

How can I write this correctly?

Tags:

2

Answers


  1. 0

    Why not just specify lambdaFunctions.py as a source for archive.

    data "archive_file" "lambda_function_add_user_post" {
  type        = "zip"
  source_file = "${path.module}/lambdaFunctions.py"
  output_path = "${path.module}/lambda_function_add_user_post.zip"
}

    This should produce zip with only single file.

    why does the unzip -l list the .zip file itself?

    Probably you have achieved this file too on second run.

    Update:
    For you example correct .tf code look like that:

    data "archive_file" "lambda_function_add_user_post" {
  type        = "zip"
  source_dir  = "${path.module}"
  output_path = "${path.module}/lambda_function_add_user_post.zip"
  excludes    = concat(
    tolist(fileset("${path.module}/.terraform", "**")),
    tolist([
      ".terraform.lock.hcl",
      "terraform.tfstate",
      "terraform.tfstate.backup",
      "venv",
      "venv/*",
      "*.zip",
    ]))
}
    Login or Signup to reply.
  2. 0

    What I would do is create an additional directory, put the python file there, and point the data source for archiving to the directory. The way you are currently doing things means that you are also including the provider binaries, which can be very large in size. It shouldn’t be included in the zip file for the Lambda function because Lambda does not need it. In doing so, your excludes does not have to worry about provider binaries, so I would suggest moving the Lambda source to a subdirectory, e.g., source, and then you can update your code to the following:

    data "archive_file" "lambda_function_add_user_post" {
  type          = "zip"
  source_dir    = "${path.module}/source"
  output_path   = "${path.module}/lambda_function_add_user_post.zip"
}

    If you need to exclude venv (or anything else that’s fluff) that should be much easier using this approach.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search