Amazon web services – Setting up a vault agent sidecar in an ECS service via terraform to pass env variables stored in the vault

I hope I can help you with this solution.
Reviewing the problem, I see that it may be rooted in invalid ARN syntax in the terraform configuration for the task definition you have in ECS. There’s a possibility that a variable that should contain a valid ARN is either poorly defined or empty. To correct the ARNs and ensure that all variables and therefore syntax are correct, let’s go step by step:

Error: creating ECS Task Definition (test): operation error ECS: RegisterTaskDefinition, https response error StatusCode: 400, RequestID: ..., ClientException: Invalid arn syntax.

Can you check the following:

1. Invalid ARNs in task_role_arn and execution_role_arn:

   • Verify the Variables: Ensure that var.ecs_task_role_arn and var.ecs_task_execution_role_arn contain valid ARNs and are correctly assigned.

   • Correct Format: ARNs should follow this format:

     arn:aws:iam::<AWS_ACCOUNT_ID>:role/<ROLE_NAME>
     

     For example:

     arn:aws:iam::123456789012:role/MyECSRole
     

   • Empty or Null Values: Check very carefully that these variables are not empty or undefined. What Terraform does is pass null values, causing those ARN syntax errors. It could be in these files variables.tf and terraform.tfvars files to confirm that these variables have assigned values.

2. Syntax Errors in the container_definitions JSON:

   • JSON Validation: Use a tool like JSONLint You could use a tool like JSONLint to validate the syntax. I know sometimes this solution is very obvious, but it doesn’t hurt to do it and verify that your JSON is correct in container_definitions.

   • Incorrect Properties: For instance, within your container definitions, you’re using "networkMode": "awsvpc",This is a task-level property, I don’t understand why you have it at the level of individual containers. In my opinion, this should be removed from your container definition.

3. Incorrect Configuration of environmentFiles:

   • Using environmentFiles: I see that this parameter is expecting you to reference an environment file stored in S3, and the value must absolutely be the ARN of the S3 object.

   • Correction: To load environment variables from a file, keep in mind that you need to provide the ARN of that S3 object. I’ll give you an example:

     "environmentFiles": [
       {
         "value": "arn:aws:s3:::bucket_name/path/to/env_vars",
         "type": "s3"
       }
     ],
     

   • Alternative: Thinking of an alternative for this, whether you’re mounting the file from a volume, I advise you to consider using the secrets parameter or loading the variables within the application

4. Using Volumes with Fargate:

   • Fargate Restrictions: Remember well that Fargate does not support docker_volume_configuration. For this, you can use or should use EFS (Elastic File System) volumes instead.

   • EFS Volume Configuration:

     volume {
       name = "vault-secrets"
       efs_volume_configuration {
         file_system_id = aws_efs_file_system.my_efs.id
       }
     }
     

   • Mount Points in Containers:

     Carefully review that the mountPoints in the containers correspond to the defined volumes and are correctly configured for Fargate.

5. Reviewing IAM Roles:

   • Necessary Permissions: It doesn’t hurt to review the necessary permissions you have for the referenced IAM roles, these are required for ECS tasks. I know it’s obvious, but check it, as this may prevent access to other resources like EFS and S3

   • Existence of Roles: The same goes for the specific roles; you must ensure they exist in the AWS account.

6. Undefined or Misreferenced Variables:

   • Confirm Variables: Inspect that your variables used in the configuration (${var.*}) are defined, and if they are, ensure that they are correctly referenced.

   • Output Variables for Debugging:

     To do a more accurate debugging, you could add the following to the Terraform configuration to get the values of the variables:

     output "ecs_task_role_arn" {
       value = var.ecs_task_role_arn
     }
     
     output "ecs_task_execution_role_arn" {
       value = var.ecs_task_execution_role_arn
     }
     

     And then you run terraform plan to see what values they have and to give you the assurance that those values are correct.

7. Simplify for Debugging:

   • Test with Minimal Configuration: Now, to make it more minimalist, reduce the task definition to the most essential and verify if the error persists. After that, start adding components one by one, isolating the problem.

Summary and Recommendations:

So in summary of all this, the "Invalid arn syntax" error is almost always due to a malformed ARN or null values passed in variables that should have those values. The recommendation I would give you is that the first thing you do is verify the variables ecs_task_role_arn and ecs_task_execution_role_arn to ensure they have valid ARNs and are well-formatted. Apart from this, carefully review your container_definitions

configuration to correct syntax errors or properties in locations where they shouldn’t be, that are incorrectly placed. Lastly, adjust your volumes and mount points so that these are fully compatible with Fargate.

I hope this brings you to a solution or closer to one. If you have more questions or need more details, please ask. Best regards.