Question posted in
The official documentation can be found here.

how to orchestrate subsets of docker compose services

I am using a bunch of docker containers to run a webserver + database + whatever else.

Currently I have each service defined in it’s own docker-compose.yml, file like…

project-root/docker/
  /base/docker-compose.yml 
  /postgres/docker-compose.yml
  /web-server/docker-compose.yml
  /web-app/docker-compose.yml
  /rabbitmq/docker-compose.yml
  /reverse-proxy/docker-compose.yml

…etc. Depending on the context (ie local devlopment vs production), I might need to start one or more of the services at once. For local development I might want only postgres and web-server, but not reverse-proxy. Whereas in prod I might want all services running.

I’ve been using some different scripts, similar to…

# ./scripts/start-local.sh

docker compose /
  -f ./docker/base/docker-compose/yml /
  -f ./docker/service-a/docker-compose.yml /
  -f ./docker/sevice-b/docker-compose.yml /
  up

…to start the different sub-sets of services, but it feels messy. Often I get orphaned container messages, and other times --remove-orphans will kill a service that I’d rather have kept running.

Ideally I’d like to be able to start some subset of services, and then add/remove other services as needed without docker considering it a different "project" (and triggering the orphaned container stuff).

Is there a better approach to handling this sort of "run subset of all services" use case using the docker cli?

Tags:

2

Answers


  1. 0

    The optimal method for managing multiple Docker services across different environments is to use a base Docker Compose file along with environment-specific override files. This ensures flexibility, maintainability, and consistency in your deployments.

    Login or Signup to reply.
  2. 0

    Here are couple of options:

    Option 1: COMPOSE_FILE

    Set COMPOSE_FILE variable to include the docker-compose files that you want (e.g. based on whether you’re on dev/staging/prod).

    I like to define a base file for core services which I always want, plus additional files for logical pieces which I may or may not want.

    Example:

    COMPOSE_FILE=compose.yaml:compose.dev.yaml:compose.mailcatcher.yaml:compose.traefik.yaml

    I set this variable by sourcing a bash script (e.g. source docker.env).
    After that I can simply run docker compose up, etc.

    Option 2: Use profiles

    Use profiles (this is a newer compose feature). This allows you to keep all of your services in a single docker-compose file (though you may need a prod file to override some details). In each service, you can define a service to be part of one or more profiles:

    services:
  frontend:
    image: frontend
    profiles: [core]

  phpmyadmin:
    image: phpmyadmin
    depends_on: [db]
    profiles: [dev]

  backend:
    image: backend
    profiles: [core]

    Then you can start services for one or more profiles like this:

       docker compose --profile core up   # start one profile
   docker compose --profile core --profile dev up  # start multiple profiles

    But to make it easy for myself, I set the COMPOSE_PROFILES variable in my script (sourcing it like above)

    export COMPOSE_PROFILES="core dev"

    so that I can simply do

       docker compose up

    and it knows which profiles I want.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search