Question 339117 in Docker Question posted in
The official documentation can be found here.

Docker – Jenkins BUILD_TAG changes when executing parallel builds

Addendum to the question:

I have now found out that for some reason the BUILD_TAG seems to be "faulty" ONLY for the docker-compose command. Also, it doesn’t matter if the pipelines are running in parallel. The BUILD_TAG for the docker-compose command is always exactly one BUILD_TAG behind the currently running pipeline or the pipeline that was executed last.

Let’s assume I start the merging pipeline. It receives the BUILD_TAG Merging-1 from Jenkins. If I echo this BUILD_TAG before and after the docker-compose command, I get Merging-1 in both cases.

Let’s now assume that I start the testing pipeline, which receives the BUILD_TAG Testing-1. If I now output this BUILD_TAG via echo before and after the docker-compose command, I get the output Testing-1 in both cases. However, the docker-compose command gives me a Recreating Merging-1 as output.

Original Question:

I have a behaviour in my Jenkins pipelines that I can’t explain.

I have two pipelines – a testing pipeline and a merging pipeline – which are (or at least should be) independent of each other. Both containers use the same docker-compose.yml (albeit each in their own workspace) to create a database container and a Rails container.

To keep the naming of the containers unique, I work with the environment variable BUILD_TAG from Jenkins. Accordingly, my docker-compose.yml looks like this:

version: '3.8'
services:
  rails:
    build:
      context: /var/lib/jenkins/dockerfiles/rails_docker/
    container_name: ${BUILD_TAG}_rails
    image: jenkins/rails
    networks:
      - jenkins
    tty: true
  database_host:
    container_name: ${BUILD_TAG}_maria_test_db
    image: mariadb:10.4
    networks:
      - jenkins
    restart: always
networks:
  jenkins:

Now the following behaviour occurs. If my merging pipeline is currently running and I start my testing pipeline in parallel, the docker-compose command of the testing pipeline sends a terminate signal to the containers of the merging pipeline. It apparently does this because the docker-compose command (suddenly) assumes that the BUILD_TAG in the testing pipeline corresponds to the BUILD_TAG of the merging pipeline running in parallel. Incidentally, this behaviour can also be reversed (i.e. first start the testing pipeline and then the merging pipeline).

These are the outputs of the Jenkins consoles when running in parallel.
Merging pipeline:

Build tag is jenkins-FooBar-FooBar.base-Merging-240
No Docker container found with name: jenkins-FooBar-FooBar.base-Merging-240_maria_test_db
[Pipeline] echo
Build tag rails is jenkins-FooBar-FooBar.base-Merging-240
[Pipeline] echo
Build tag maria DB is jenkins-FooBar-FooBar.base-Merging-240
[Pipeline] sh
+ docker-compose -f /var/lib/jenkins/workspace/FooBar/FooBar.base/Merging/docker/rails_docker/docker-compose.yml up -d --force-recreate --build
Building rails
Step 1/18 :
Step 2/18 : 
Step 3/18 :
Step 4/18 : 
Step 5/18 : 
Step 6/18 : 
Step 7/18 : 
Step 8/18 : 
Step 9/18 : 
Step 10/18 : 
Step 11/18 : 
Step 12/18 : 
Step 13/18 : 
Step 14/18 : 
Step 15/18 : 
Step 16/18 : 
Step 17/18 : 
Step 18/18 :

Successfully built 85b48dcc91c0
Successfully tagged jenkins/rails:latest
Creating jenkins-FooBar-FooBar.base-Merging-240_rails ... 
Creating jenkins-FooBar-FooBar.base-Merging-240_maria_test_db ...

Testing pipeline:

Build tag is jenkins-FooBar-FooBar.base-Testing-593

Build tag rails is jenkins-FooBar-FooBar.base-Testing-593
[Pipeline] echo
Build tag maria DB is jenkins-FooBar-FooBar.base-Testing-593
[Pipeline] sh
+ docker-compose -f /var/lib/jenkins/workspace/FooBar/FooBar.base/Testing/docker/rails_docker/docker-compose.yml up -d --force-recreate --build
Building rails
Step 1/18 :
Step 2/18 : 
Step 3/18 : 
Step 4/18 : 
Step 5/18 : 
Step 6/18 :
Step 7/18 : 
Step 8/18 : 
Step 9/18 : 
Step 10/18 : 
Step 11/18 : 
Step 12/18 : 
Step 13/18 : 
Step 14/18 : 
Step 15/18 : 
Step 16/18 : 
Step 17/18 : 
Step 18/18 :

Successfully built 85b48dcc91c0
Successfully tagged jenkins/rails:latest
Recreating jenkins-FooBar-FooBar.base-Merging-240_rails ... 
Recreating jenkins-FooBar-FooBar.base-Merging-240_maria_test_db ...

Is this behaviour known in any way? Am I missing something obvious? Thank you in advance for your help.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    thank you very much for your answer. Firstly, changing the docker-compose command from

    sh "docker-compose -f ${WORKSPACE}/docker/rails_docker/docker-compose.yml up -d --force-recreate --build"

    to

    sh "docker-compose -p ${BUILD_TAG} -f ${WORKSPACE}/docker/rails_docker/docker-compose.yml up -d --force-recreate --build" has indeed fixed the problem.

    However, I still don't understand why it works this way.

    I understood from the documentation you linked to that the COMPOSE_PROJECT_NAME variable is particularly relevant if, for example, I have not specified a container name. But I have it in my docker-compose.yml and it is also used (unfortunately with the BUILD_TAG of the last build).

    In addition, although the docker-compose.ymls are syntactically identical, they are each located in their own directories. The docker-compose.yml for the testing pipeline is in the WORKSPACE of the testing pipeline and that of the merging pipeline is in the WORKSPACE of the merging pipeline.

    Can you explain in more detail why your solution works?

    Thank you for your answer anyway.


  2. 0

    Docker compose determines which resources to manage by the compose project name, which is by default name (basename) of the directory where the config file is located, instead of names of the individual containers. As the default compose project name (in above case rails_docker) is used in both pipelines, docker compose now identifies the projects from the different pipelines as the same project and will replace the containers when there are changes to the configuration.

    To use different projects for testing and merging pipelines, set the compose project name by using -p flag with docker compose commands or by setting COMPOSE_PROJECT_NAME environment variable. See docker compose docs for details.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search