Amazon web services - packer post-processor: failed to push image into ECR

MacUsers
October 27, 2023
219 views
0 votes
2 Answers

I tried looking for an answer to this in Packer community-forum without any joy. so, I am wondering if anyone can give me a hand here to understand why it’s being failed to push the image to ECR.

Below is my very simple packer HCL file, with two stepes under post-processors block to tag and push:

packer {
  required_plugins {
    docker = {
      source  = "github.com/hashicorp/docker"
      version = "~> 1"
    }
  }
}
#
locals {
  timestamp = regex_replace(timestamp(), "[- TZ:]", "")
}
#
variable "img_name" {
  type    = string
}
variable "repo" {
  type    = string
}
#
source "docker" "al2023" {
  image  = "public.ecr.aws/amazonlinux/amazonlinux:2023"
  commit = true
}
#
build {
  name    = var.img_name
  sources = ["source.docker.al2023"]

  post-processors {
    post-processor "docker-tag" {
      repository = var.repo
      tags       = ["latest", local.timestamp]
    }

    post-processor "docker-push" {
      ecr_login           = true
      keep_input_artifact = false
      login_server        = var.repo
    }
  }
}

then I run packer as null_resource through terraform:

resource "aws_ecr_repository" "this" {
  name                 = "${var.app_name}build"
  image_tag_mutability = "IMMUTABLE"

  encryption_configuration {
    encryption_type = "KMS"
    kms_key         = var.kms_key_arn
  }

  image_scanning_configuration {
    scan_on_push = true
  }

  tags = merge(
    var.extra_tags,
    { Name = "${var.app_name}build" }
  )
}
#
resource "null_resource" "packer" {
  depends_on = [aws_ecr_repository_policy.this]

  triggers = {
    img_name = local.img_name
  }

  provisioner "local-exec" {
    working_dir = "${path.module}/packer_build"
    command     = <<EOF
packer init ${path.module}/my_build.pkr.hcl && 
PACKER_LOG=1 packer build 
  -var img_name=${local.img_name} 
  -var repo=${aws_ecr_repository.this.repository_url} 
  ${path.module}/my_build.pkr.hcl

if [ $? -eq 0 ]; then
  printf "n[+]  Packer build SUCCEEDED!!n"
else
  printf "n[-]  Packer build FAILED!!!n" >&2
  exit 1
fi
EOF
  }
}

which fails with:

│ ': exit status 1. Output: r-push): 79a6254fc7b1: Retrying in 7 seconds
│     znpexc-ibc-aut-20231009070233.docker.al2023 (docker-push):
│ 76ec60dbc4a7: Retrying in 6 seconds
│     znpexc-ibc-aut-20231009070233.docker.al2023 (docker-push):
│ 79a6254fc7b1: Retrying in 6 seconds
| ....
| ....
│     znpexc-ibc-aut-20231009070233.docker.al2023 (docker-push):
│ 79a6254fc7b1: Retrying in 1 second
│     znpexc-ibc-aut-20231009070233.docker.al2023 (docker-push):
│ EOF
│     znpexc-ibc-aut-20231009070233.docker.al2023 (docker-push):
│ Logging out...
│ 2023/10/09 07:03:36 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin:
│ 2023/10/09 07:03:36 Executing: /usr/bin/docker [--config
│ /tmp/packer1870521035 logout
│ 99xxxxxxxxxx.dkr.ecr.eu-west-2.amazonaws.com/znpexc-ibc-autbuild]
│     znpexc-ibc-aut-20231009070233.docker.al2023 (docker-push):
│ Removing login credentials for
│ 99xxxxxxxxxx.dkr.ecr.eu-west-2.amazonaws.com
│     znpexc-ibc-aut-20231009070233.docker.al2023 (docker-push):
│ Removing temporary Docker configuration directory
│ 2023/10/09 07:03:36 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin:
│ 2023/10/09 07:03:36 error: Bad exit status: 1
│ 2023/10/09 07:03:36 [INFO] (telemetry) ending docker-push
│
│ * Post-processor failed: Bad exit status: 1
│ ==> Wait completed after 1 minute 951 milliseconds
│ 2023/10/09 07:03:36 machine readable: error-count []string{"1"}
│ ==> Some builds didn't complete successfully and had errors:
│ 2023/10/09 07:03:36 machine readable:
│ znpexc-ibc-aut-20231009070233.docker.al2023,error []string{"1 error(s)
│ occurred:nn* Post-processor failed: Bad exit status: 1"}
│
│ * Post-processor failed: Bad exit status: 1
│ ==> Builds finished but no artifacts were created.
│ Build 'znpexc-ibc-aut-20231009070233.docker.al2023' errored after 1
│ minute 951 milliseconds: 1 error(s) occurred:
│
│ * Post-processor failed: Bad exit status: 1
│
│ ==> Wait completed after 1 minute 951 milliseconds
│
│ ==> Some builds didn't complete successfully and had errors:
│ --> znpexc-ibc-aut-20231009070233.docker.al2023: 1 error(s) occurred:
│
│ * Post-processor failed: Bad exit status: 1
│
│ ==> Builds finished but no artifacts were created.
│ 2023/10/09 07:03:36 [INFO] (telemetry) Finalizing.
│ 2023/10/09 07:03:36 waiting for all plugin processes to complete...
│ 2023/10/09 07:03:36
│ /home/santanu/.config/packer/plugins/github.com/hashicorp/docker/packer-plugin-docker_v1.0.8_x5.0_linux_arm64:
│ plugin process exited
│ 2023/10/09 07:03:36
│ /home/santanu/.config/packer/plugins/github.com/hashicorp/docker/packer-plugin-docker_v1.0.8_x5.0_linux_arm64:
│ plugin process exited
│ 2023/10/09 07:03:36
│ /home/santanu/.config/packer/plugins/github.com/hashicorp/docker/packer-plugin-docker_v1.0.8_x5.0_linux_arm64:
│ plugin process exited
│
│   [-]  Packer build FAILED!!!

Any idea what am I doing wrong here?
It works perfectly okay with source type amazon-ebs, using assume_role{} configuration in place but I don’t see anything simialr for docker build-type.

I’m trying to find out if there is anything missing in the packer HCL or issue with my ECR repo. Below is the policy that attached to the repo, if that helps…

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowPullPush",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::99xxxxxxxxxx:root"
        ]
      },
      "Action": [
        "ecr:BatchCheckLayerAvailability",
        "ecr:BatchGetImage",
        "ecr:CompleteLayerUpload",
        "ecr:GetDownloadUrlForLayer",
        "ecr:InitiateLayerUpload",
        "ecr:ListImages",
        "ecr:PutImage",
        "ecr:UploadLayerPart"
      ]
    }
  ]
}

as requiest by @Matthew Schuchard, this is what I see when packer runs by itself:

==> znpexc-ibc-aut-20231009203420.docker.al2023: Creating a temporary directory for sharing data...
==> znpexc-ibc-aut-20231009203420.docker.al2023: Pulling Docker image: public.ecr.aws/amazonlinux/amazonlinux:2023
2023/10/10 14:56:35 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:35 Set Packer temp dir to /home/santanu/.config/packer/tmp3020070013
2023/10/10 14:56:35 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:35 Executing: /usr/bin/docker [pull public.ecr.aws/amazonlinux/amazonlinux:2023]
    znpexc-ibc-aut-20231009203420.docker.al2023: 2023: Pulling from amazonlinux/amazonlinux
    znpexc-ibc-aut-20231009203420.docker.al2023: Digest: sha256:f11c9d2a038d1b658dac60a51222bfb7a5d3e04f919093528581f6ce41906b37
    znpexc-ibc-aut-20231009203420.docker.al2023: Status: Image is up to date for public.ecr.aws/amazonlinux/amazonlinux:2023
    znpexc-ibc-aut-20231009203420.docker.al2023: public.ecr.aws/amazonlinux/amazonlinux:2023
==> znpexc-ibc-aut-20231009203420.docker.al2023: Starting docker container...
    znpexc-ibc-aut-20231009203420.docker.al2023: Run command: docker run -v /home/santanu/.config/packer/tmp3020070013:/packer-files -d -i -t --entrypoint=/bin/sh -- public.ecr.aws/amazonlinux/amazonlinux:2023
2023/10/10 14:56:36 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:36 Starting container with args: [run -v /home/santanu/.config/packer/tmp3020070013:/packer-files -d -i -t --entrypoint=/bin/sh -- public.ecr.aws/amazonlinux/amazonlinux:2023]
2023/10/10 14:56:36 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:36 Waiting for container to finish starting
    znpexc-ibc-aut-20231009203420.docker.al2023: Container ID: 6bdcbfb60fd73d7bc260511ddb4fce2810d1092bc666103afcd2a9203daebcff
==> znpexc-ibc-aut-20231009203420.docker.al2023: Using docker communicator to connect: 172.17.0.2
2023/10/10 14:56:36 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:36 Running the provision hook
==> znpexc-ibc-aut-20231009203420.docker.al2023: Committing the container
2023/10/10 14:56:36 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:36 Committing container with args: [commit 6bdcbfb60fd73d7bc260511ddb4fce2810d1092bc666103afcd2a9203daebcff]
    znpexc-ibc-aut-20231009203420.docker.al2023: Image ID: sha256:dd281fedd51e82ecaa731fecaf627ff4befd462467a59433368379457b05f6f4
==> znpexc-ibc-aut-20231009203420.docker.al2023: Killing the container: 6bdcbfb60fd73d7bc260511ddb4fce2810d1092bc666103afcd2a9203daebcff
==> znpexc-ibc-aut-20231009203420.docker.al2023: Running post-processor:  (type docker-tag)
2023/10/10 14:56:37 [INFO] (telemetry) ending docker.al2023
2023/10/10 14:56:37 [INFO] (telemetry) Starting post-processor docker-tag
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-tag): Tagging image: sha256:dd281fedd51e82ecaa731fecaf627ff4befd462467a59433368379457b05f6f4
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-tag): Repository: <sensitive>:latest
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-tag): Tagging image: sha256:dd281fedd51e82ecaa731fecaf627ff4befd462467a59433368379457b05f6f4
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-tag): Repository: <sensitive>:20231010145634
2023/10/10 14:56:37 [INFO] (telemetry) ending docker-tag
2023/10/10 14:56:37 Flagging to keep original artifact from post-processor 'docker-tag'
2023/10/10 14:56:37 [INFO] (telemetry) Starting post-processor docker-push
==> znpexc-ibc-aut-20231009203420.docker.al2023: Running post-processor:  (type docker-push)
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Creating temporary Docker configuration directory
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Fetching ECR credentials...
2023/10/10 14:56:37 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:37 Getting ECR token for account: 998380306071 in eu-west-2..
2023/10/10 14:56:37 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:37 [INFO] AWS Auth provider used: "SharedCredentialsProvider"
2023/10/10 14:56:37 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:37 Found region eu-west-2
2023/10/10 14:56:37 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:37 [INFO] AWS authentication used: "SharedCredentialsProvider"
2023/10/10 14:56:37 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:37 Successfully got login for ECR: <sensitive>
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Logging in...
2023/10/10 14:56:37 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:37 Executing: /usr/bin/docker [--config /tmp/packer296049404 login -u AWS --password-stdin <sensitive>]
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Login Succeeded
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): WARNING! Your password will be stored unencrypted in /tmp/packer296049404/config.json.
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Configure a credential helper to remove this warning. See
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): https://docs.docker.com/engine/reference/commandline/login/#credentials-store
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Pushing: <sensitive>:20231010145634
2023/10/10 14:56:37 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:56:37 Executing: /usr/bin/docker [--config /tmp/packer296049404 push <sensitive>:20231010145634]
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): The push refers to repository [<sensitive>]
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): 5392f2132bbf: Preparing
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): 79a6254fc7b1: Preparing
    .... <goes for a min or so> ....
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): 5392f2132bbf: Retrying in 1 second
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): 79a6254fc7b1: Retrying in 1 second
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): EOF
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Logging out...
2023/10/10 14:57:28 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:57:28 Executing: /usr/bin/docker [--config /tmp/packer296049404 logout <sensitive>]
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Removing login credentials for 998380306071.dkr.ecr.eu-west-2.amazonaws.com
    znpexc-ibc-aut-20231009203420.docker.al2023 (docker-push): Removing temporary Docker configuration directory
2023/10/10 14:57:28 [INFO] (telemetry) ending docker-push

* Post-processor failed: Bad exit status: 1
Build 'znpexc-ibc-aut-20231009203420.docker.al2023' errored after 53 seconds 42 milliseconds: 1 error(s) occurred:

* Post-processor failed: Bad exit status: 1

==> Wait completed after 53 seconds 43 milliseconds
2023/10/10 14:57:28 packer-plugin-docker_v1.0.8_x5.0_linux_arm64 plugin: 2023/10/10 14:57:28 error: Bad exit status: 1
==> Wait completed after 53 seconds 43 milliseconds
2023/10/10 14:57:28 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2023/10/10 14:57:28 machine readable: znpexc-ibc-aut-20231009203420.docker.al2023,error []string{"1 error(s) occurred:nn* Post-processor failed: Bad exit status: 1"}

* Post-processor failed: Bad exit status: 1
==> Builds finished but no artifacts were created.
2023/10/10 14:57:28 [INFO] (telemetry) Finalizing.

==> Some builds didn't complete successfully and had errors:
--> znpexc-ibc-aut-20231009203420.docker.al2023: 1 error(s) occurred:

* Post-processor failed: Bad exit status: 1

==> Builds finished but no artifacts were created.
2023/10/10 14:57:28 waiting for all plugin processes to complete...
2023/10/10 14:57:28 /home/santanu/.config/packer/plugins/github.com/hashicorp/docker/packer-plugin-docker_v1.0.8_x5.0_linux_arm64: plugin process exited
2023/10/10 14:57:28 /home/santanu/.config/packer/plugins/github.com/hashicorp/docker/packer-plugin-docker_v1.0.8_x5.0_linux_arm64: plugin process exited
2023/10/10 14:57:28 /home/santanu/.config/packer/plugins/github.com/hashicorp/docker/packer-plugin-docker_v1.0.8_x5.0_linux_arm64: plugin process exited

-S

Answers

Chosen as BEST ANSWER
- MacUsers
- October 27, 2023 at 12:31 am
- 0 votes
0
This post gave me the asnswer: How to Push to Account Using ASSUME_ROLE in AWS ECR

I had to add aws_profile to "docker-push", like this:
```
    post-processor "docker-push" {
      aws_profile         = var.aws_profile
      ecr_login           = true
      keep_input_artifact = false
      login_server        = var.img_repo
      only                = ["docker.ub2204"]
    }
```
where var.aws_profile is the name of the profile in ~/.aws/config, which represents the role that terraform runs with. I can now push the image to ECR without any issue.

Thanks all for helping me out.

(Edit)

- NgcB236nh
- October 10, 2023 at 4:20 pm
- 0 votes
0
I am able to run your Packer file, the syntax is correct.

The problem in here is the Principal IAM Policy

you should allow user that terraform use to run, not root
```
"Principal": {
    "AWS": [
        "arn:aws:iam::account-id:user/push-pull-user-1",
        "arn:aws:iam::account-id:user/push-pull-user-2"
    ]
}
```
Reference: https://docs.aws.amazon.com/AmazonECR/latest/userguide/repository-policy-examples.html
Login or Signup to reply.

Please signup or login to give your own answer.

Click here to cancel reply.

Amazon web services – packer post-processor: failed to push image into ECR

Answers