Question 256661 in PostgreSQL Question posted in
The official documentation can be found here.

Make Docker Compose Container wait for container PostgreSQL and restore

I have a docker compose with one container PostgreSQL and another one with my App. I need to restore two databases into the PostgreSQL and then make the App start.

Currently I have tried this docker compose:

version: "3.7"
services:
  dataspace_postgres:
    container_name: custompostgres
    platform: linux/amd64
    build:
      context: ./
      dockerfile: Dockerfile.CustomPostgres
    networks:
      - dataspace
    environment:
      PGUSER: postgres
      POSTGRES_DB: postgres
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: password
    ports:
      - 5434:5432
    volumes:
      - disk-postgres:/var/lib/postgresql/data
      - ./dumps/psql/managementreporting.sql:/managementreporting.sql
      - ./dumps/psql/idams_new.sql:/idams_new.sql
      - ./dumps/init.sh:/docker-entrypoint-initdb.d/init.sh
    healthcheck:
      test: ["CMD-SHELL", "sh -c 'pg_isready -U postgres -d managementreporing'"]
      interval: 10s
      timeout: 120s
      retries: 10

  dataspace_consolidatedreportingunit:
    container_name: consolidatedreportingunit
    platform: linux/amd64
    build:
      context: ./
      dockerfile: Dockerfile.ReportingTool
    networks:
      - dataspace
    ports:
      - 8080:80
      - 8081:443
    depends_on:
      dataspace_postgres:
        condition: service_healthy

networks:
  dataspace:
    driver: bridge

volumes:
  disk-postgres:
    driver: local

And my init.sh file:

pg_restore -v -c -d managementreporting managementreporting.sql
pg_restore -v -c -d idams_new idams_new.sql

Currently on startup it fails with the following error no matter what i tried:

pg_restore: connecting to database for restore custompostgres
| 2023-09-29 14:27:03.007 UTC [64] FATAL: database
"managementreporting" does not exist custompostgres |
pg_restore: error: connection to server on socket
"/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: database
"managementreporting" does not exist custompostgres exited with code 1

For me the only way to restore the databases is to mount them directly into /docker-entrypoint-initdb.d/ (for dumps) but then I have no way to make the other container wait the postgesql to finish the restore.

Any ideas?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I managed to solve both of the issues, of restoring databases and waiting for the restores to complete using just simple methods.

    First of all in the Docker-Compose, under volumes, you can restore the databases if you have backup them using plain mode by simply injecting them into docker-entrypoint-initdb.d folder, but using this method you cannot check when the restore actually completes, see below:

    volumes:
  - disk-postgres:/var/lib/postgresql/data
  - ../PostgreSQL/dumps/plain/managementreporting.sql:/docker-entrypoint-initdb.d/managementreporting.sql
  - ../PostgreSQL/dumps/plain/idams_new.sql:/docker-entrypoint-initdb.d/idams_new.sql

    Instead, we use custom format backups and we inject them into the root folder, so then we need a script to restore them and also to make a flag when the restore completes, see below:

    volumes:
  - disk-postgres:/var/lib/postgresql/data
  - ../PostgreSQL/dumps/psql/managementreporting.sql:/managementreporting.sql
  - ../PostgreSQL/dumps/psql/idams_new.sql:/idams_new.sql
  - ../PostgreSQL/dumps/init.sh:/docker-entrypoint-initdb.d/init.sh
healthcheck:
  test: ["CMD-SHELL", "test -f /tmp/healthcheck.txt"]
  interval: 10s
  timeout: 120s
  retries: 10
  start_period: 60s

    And the cmd-shell init.sh script for the restore and flag, see below

    #Restore Consolidated Reporting Unit Database
createdb managementreporting #use it to avoid locale error on restore instead of pg_restore -C
pg_restore -v -U postgres -d managementreporting --no-owner /managementreporting.sql # -d should be postgres
#Restore Production BluePrint Idams New Database
createdb idams_new #use it to avoid locale error on restore instead of pg_restore -C
pg_restore -v -U postgres -d idams_new --no-owner /idams_new.sql # -d should be postgres
#Create a file to use healtcheck and ensure that PostgreSQL init has finished as during this action can accept connections
touch /tmp/healthcheck.txt #same file name at docker-compose healtcheck

    In the init.sh script we use createdb to create the database instead of -C switch into the pg_restore as this produces problems when the database to be restored has different collation than default postgres database. (Also found that the -C switch expects to find the database already existing instead of creating the database, strange, most provably because expects to drop it first).

    In addition, we use --no-owner switch into pg_restore in order to ignore previous owner from the server where we took the backup because this owner may not exist in the server which we use for restore.

    In the end, we create a file (empty file) just to make it available for healtcheck in docker-compose and flag the container as healthy and then start the second container with the application. Finally do not pay so much attention into start_period: 60s, I placed this just there because I am sure the databases will not be restored before 60s and therefore there is no need to make tcp check calls before 60s.


  2. 0

    First of all, I see that init.sh is being mounted in the directory /docker-entrypoint-initdb.d/, but the paths to the dump files are relative. The dumps are being mounted in the root folder /. Also, I think you meant for the -C or --create flag on pg_restore (not lowercase -c or --clean) to create the database. The fixed init.sh script would look like this:

    pg_restore -v --create -d managementreporting /managementreporting.sql
pg_restore -v --create -d idams_new /idams_new.sql

    Another thing you can try (if the healthcheck isn’t working) is that
    you can insert a "wait for" service in between the app and the database. For example:

    services:
  db:
    image: postgres
  wait:
    image: willwill/wait-for-it
    command: -t 60 db:5432
    depends_on:
      - db
  app:
    image: some/image
    depends_on:
      - wait

    The "app" service will wait for the "wait" service to be ready, and the "wait" service will wait for a socket to be open on port 5432 on the db service.

    Here’s a blog post that demonstrates it (albeit flyway instead of an app, but same idea.)

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search