Question 340432 in Docker Question posted in
The official documentation can be found here.

Docker pull command fails in the middle of a pull operation

I have a kubernetes cluster in Azure. In the cluster I exec into a pod and try to pull a docker image from a private Azure container registry (ACR) named xxx. Before the pulls, I logged in via docker login using a service principal that has pull access to that private ACR.

When I pull a small image, it works ok:

runner@arc-runner-set-nkb6b-runner-dtcnv:~$ docker pull xxx.azurecr.io/hello-world
Using default tag: latest
latest: Pulling from hello-world
1b930d010525: Pull complete
Digest: sha256:92c7f9c92844bbbb5d0a101b22f7c2a7949e40f8ea90c8b3bc396879d95e899a
Status: Downloaded newer image for xxx.azurecr.io/hello-world:latest
xxx.azurecr.io/hello-world:latest
runner@arc-runner-set-nkb6b-runner-dtcnv:~$ docker images
REPOSITORY                                               TAG       IMAGE ID       CREATED       SIZE
xxx.azurecr.io/hello-world   latest    fce289e99eb9   5 years ago   1.84kB

But, when I attempt to pull a larger image (the one below weights 1.24GB) from the same ACR, the pull breaks in the middle. It downloads some of the layers and then fail with pull access denied error, like below:

runner@arc-runner-set-nkb6b-runner-dtcnv:~$ docker pull xxx.azurecr.io/infrastructure-github-actions-image:latest
latest: Pulling from infrastructure-github-actions-image
31bd5f451a84: Pull complete
c775818794e7: Pull complete
5cf300f11d84: Pull complete
eb81f54b01af: Pull complete
f3a184abd17b: Downloading
dc1168a60e35: Downloading
5d3d464daf1d: Download complete
9a8c0a4ab53e: Download complete
0c1136627dd1: Download complete
pull access denied for xxx.azurecr.io/infrastructure-github-actions-image, repository does not exist or may require 'docker login': denied: {"errors":[{"code":"DENIED","message":"requested access to the resource is denied"}]}

Other observations:

  • I can successfully pull the xxx.azurecr.io/infrastructure-github-actions-image:latest image locally (on my local laptop)
  • On the k8s pod I can successfully pull any image (big or small) from the docker hub:
runner@arc-runner-set-gs4q4-runner-vggj5:~$ docker images
REPOSITORY                                               TAG                  IMAGE ID       CREATED        SIZE
cimg/android                                             2024.01.1-browsers   4bf6cdd305c8   3 weeks ago    7.3GB
openjdk                                                  21                   079114de2be1   4 months ago   504MB
xxx.azurecr.io/hello-world   6d3df195e8           fce289e99eb9   5 years ago    1.84kB

I’m out of ideas why the operation fails in the middle. Anyone stumbled upon such a problem or have any suggestions on what else I can do to debug this?

Thank you.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    Ok, solved. To anyone that stumbles upon the same thing let me explain where was the problem:

    The docker pull command I was running from a container requires docker daemon to be present. Docker daemon was provided by another container collocated within the same pod named docker:dind (where dind means docker in docker), it's from here: (https://hub.docker.com/_/docker).

    The problem was the most recent docker:dind distribution 25.0.3. I downgraded dind to docker:24-dind and the pull works correctly now.

    I didn't have time yet to do deeper into the problem, but I hope I will.

    Thanks anyone who tried to help.


  2. 0

    Opinionated but empirical answer (the same happened to me).

    The message usually comes when you put the wrong image name. Please check if your image has the correct name and tag on the Azure ECR repository. You probably use shell autocompletion with the wrong history added on the first machine; on another machine, it’s fine. Recheck all symbols carefully or try to run the same command from another shell/terminal (probably a bad symbol somewhere).

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search