Question 85225 in Docker Question posted in
The official documentation can be found here.

How to use locally built image in docker-compose file correctly?

I have multiple services that needs to use the same locally built image.

So I created a helper service which would build the image and all other services would depend on the builder service to ensure that image is built first.
Here’s docker-compose.yml with only 2 services but in actual, I have around 20 services.

version: "3.5"

services:
    api:
        image: test-image      # using the image built below
        networks:
            - mylab
        depends_on:
            - tests-container-builder
        volumes:
            - ./tests:/tests
        command: pytest -v -p no:cacheprovider /tests/api.py

    license-chrome:
        image: test-image
        networks:
            - mylab
        depends_on:
            - tests-container-builder
        volumes:
            - ./tests:/tests
        command: sh -c "/tests/health-check-script.sh && pytest -v -p no:cacheprovider -n=1 /tests/license.py"

    tests-container-builder:
        build:
            context: .
            dockerfile: ./tests/Dockerfile
            target: test-env
        image: test-image             # this service builds the image

But when I build and run the container using

docker-compose up --build api; 
docker-compose up --scale license-chrome

I’m getting error like

ERROR: for mylab_visual_ner-chrome_1  no such image: sha256:ceff9945d8722c89331696aa33d88f84703e322fbb64c1ebaa8f83c6e19a4cfa: No such image: sha256:ceff9945d8722c89331696aa33d88f84703e322fbb64c1ebaa8f83c6e19a4cfa

ERROR: for visual_ner-chrome  no such image: sha256:ceff9945d8722c89331696aa33d88f84703e322fbb64c1ebaa8f83c6e19a4cfa: No such image: sha256:ceff9945d8722c89331696aa33d88f84703e322fbb64c1ebaa8f83c6e19a4cfa
ERROR: The image for the service you're trying to recreate has been removed. If you continue, volume data could be lost. Consider backing up your data before continuing.

Continue with the new image? [yN]

It works fine if I remove the test-container-builder service and use build command inside all services.

Is it considered an anti-pattern to use a different service to build image?
What is the proper way to do this ?

Tags:

2

Answers


  1. 0

    I’d consider an absolutely-best-practice Compose setup to only contain long-running containers, and not containers you don’t expect to actually run. In your case the "builder" container doesn’t actually do anything and I wouldn’t include it in the Compose setup. Conversely, build:ing a literally identical image from the same source code is all but free; Docker needs to scan the build context to determine that the two images are identical, but in the end you will just get two different names for the same physical image.

    Taking @TheFool’s suggestion to use YAML anchors to avoid repeating the build setup, I might write:

    version: '3.8'
services:
  api:
    build: &build-definition
      context: .
      dockerfile: ./tests/Dockerfile
      target: test-env # (can you avoid this?)
    command: pytest -v -p no:cacheprovider /tests/api.py
  license-chrome:
    build: *build-definition
    # (does the health check belong in an ENTRYPOINT wrapper in the image?)
    command: sh -c "/tests/health-check-script.sh && pytest -v -p no:cacheprovider -n=1 /tests/license.py"

    I’ve omitted unnecessary networks:, since Compose creates a perfectly usable network named default for you, and volumes:, since your code should generally be built into the image.

    If you want to explicitly build the image and include it in the Compose setup (maybe you want to push it to a registry) then you need to manually docker-compose build the builder image, not any of the containers you’re going to run. The Compose setup can look like

    version: '3.8'
services:
  api:
    image: registry.example.com/my-name/test-image:${TAG:-latest}
    command: pytest -v -p no:cacheprovider /tests/api.py
  license-chrome:
    image: registry.example.com/my-name/test-image:${TAG:-latest}
    command: sh -c "/tests/health-check-script.sh && pytest -v -p no:cacheprovider -n=1 /tests/license.py"
  tests-container-builder:
    image: registry.example.com/my-name/test-image:${TAG:-latest}
    build:
      context: .
      dockerfile: ./tests/Dockerfile
      target: test-env
    restart: "no"
    command: exit 0

    Compared to my previous Compose setup and your question, I make it explicit that the container should just exit immediately and not restart, and I use a registry-qualified name. depends_on: only affects the order in which containers start starting and doesn’t include any dependencies on image builds.

    If you’re using this setup, you need to docker-compose build the builder "container" before you run the rest:

    export TAG=20211124                          # often useful to avoid ...:latest
docker-compose build test-container-builder  # not "api"
# docker-compose push test-container-builder # if required
docker-compose up -d                         # could just launch specific containers
    Login or Signup to reply.
  2. 0

    This is maybe not a real answer, but I have recreated your scenario, and it surprisingly works. I did expect it to fail if you only want to build the container that hasn’t a build key in the config.

    docker system prune -af       # ensure we dont get confused with pre build images or chaching
docker-compose up --build two # try to run only service two

    Compose file:

    version: "3.9"
volumes:
  data: null
services:
  one:
    image: foo
    build:
      context: ./
      dockerfile: Dockerfile
      target: stage1
    command: sh -c 'sleep 5 && echo "one done"'
  two:
    image: foo
    command: sh -c 'sleep 5 && echo "two done"'
    volumes:
      - data:/stuff
      - ./test:/test
    depends_on:
      - one

    Dockerfile

    FROM busybox as stage1
RUN echo "Hello World"
FROM busybox
CMD sleep infinity

    The output I am getting is the following. It may make sense because when reading the definition of the –build flag, Build images before starting containers., it sounds a bit like it doesn’t care about what service you want to start. It just builds all images.

    docker-compose up --build two
Creating network "test_default" with the default driver
Building one
[+] Building 3.3s (6/6) FINISHED                                                                       
 => [internal] load build definition from Dockerfile                                              0.1s
 => => transferring dockerfile: 121B                                                              0.0s
 => [internal] load .dockerignore                                                                 0.1s
 => => transferring context: 2B                                                                   0.0s
 => [internal] load metadata for docker.io/library/busybox:latest                                 2.3s
 => [stage1 1/2] FROM docker.io/library/busybox@sha256:b5cfd4befc119a590ca1a81d6bb0fa1fb19f1fbeb  0.4s
 => => resolve docker.io/library/busybox@sha256:b5cfd4befc119a590ca1a81d6bb0fa1fb19f1fbebd0397f2  0.0s
 => => sha256:b5cfd4befc119a590ca1a81d6bb0fa1fb19f1fbebd0397f25fae164abe1e8a6a 2.29kB / 2.29kB    0.0s
 => => sha256:50e44504ea4f19f141118a8a8868e6c5bb9856efa33f2183f5ccea7ac62aacc9 527B / 527B        0.0s
 => => sha256:ffe9d497c32414b1c5cdad8178a85602ee72453082da2463f1dede592ac7d5af 1.46kB / 1.46kB    0.0s
 => => sha256:3cb635b06aa273034d7080e0242e4b6628c59347d6ddefff019bfd82f45aa7 772.78kB / 772.78kB  0.3s
 => => extracting sha256:3cb635b06aa273034d7080e0242e4b6628c59347d6ddefff019bfd82f45aa7d5         0.1s
 => [stage1 2/2] RUN echo "Hello World"                                                           0.4s
 => exporting to image                                                                            0.0s
 => => exporting layers                                                                           0.0s
 => => writing image sha256:81893d72f6be148599854469098e9f8c80a73e28acd5991a1757033440d34e5c      0.0s
 => => naming to docker.io/library/foo                                                            0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Creating test_one_1 ... done
Creating test_two_1 ... done
Attaching to test_two_1
two_1  | two done
test_two_1 exited with code 0
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search