Question posted in
The official documentation can be found here.

(psycopg2.OperationalError) FATAL: the database system is starting up, docker + redis + celery

Am trying to run Postgres, Celery, Redis, and the project itself in docker, but it seems the database is not connecting at all. It tries to connect and fails.

Below is the error it throws after running docker :

travel_dev_container |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 656, in __connect
travel_dev_container |     connection = pool._invoke_creator(self)
travel_dev_container |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
travel_dev_container |     return dialect.connect(*cargs, **cparams)
travel_dev_container |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 493, in connect
travel_dev_container |     return self.dbapi.connect(*cargs, **cparams)
travel_dev_container |   File "/usr/local/lib/python3.7/site-packages/psycopg2/__init__.py", line 127, in connect
travel_dev_container |     conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
travel_dev_container | psycopg2.OperationalError: FATAL:  the database system is starting up

Am wondering what I have configured wrong, below is my docker-compose.yml file :

version: "3.7"

services:

  redis:
    container_name: redis_huxy_tour_dev_container
    image: redis
    ports:
      - "6379:6379"

  db:
    environment:
      POSTGRES_USER: test
      POSTGRES_PASSWORD: test
      POSTGRES_DB: test
    image: postgres:latest
    networks:
      - default
    ports:
      - 5405:5432
    restart: always
    volumes:
      - ./postgres-data:/var/lib/postgresql/data

  flask_service:
    container_name: travel_dev_container
    restart: always
    image: flask
    build:
      context: ./
      dockerfile: Dockerfile
    depends_on:
        - redis
        - db
    ports:
      - "5000:5000"
    volumes:
      - ./:/app
    environment:
        - FLASK_DEBUG=1
        - FLASK_APP=run.py
        - DATABASE_URL=postgresql://test:test@db/test
        - REDIS_URL=redis://redis:6379/0
        - WEATHER_API_KEY=1d4ce67223a53a013fc03ead36137396
        - SECRET_KEY=jfdjhfl

then, my Dockerfile :

FROM python:3.7

RUN mkdir app

COPY . /app/

WORKDIR /app

RUN chmod +x entrypoint.sh

# Install the Python libraries
RUN pip3 install --no-cache-dir -r requirements.txt

EXPOSE 5000

#CMD ["./entrypoint.sh"]
CMD ["bash", "entrypoint.sh"]

finally, the shell script file entrypoint.sh:

#Run migrations
flask db upgrade

# Run Celery worker
celery -A app.tasks.weather_tasks.celery worker -l info &

# Run Celery beat
celery -A app.tasks.weather_tasks.celery beat -l info &

python run.py

What could be the reason of this error

Tags:

2

Answers


  1. 0

    You need to use a wrapper to test for the database to be ready before running the flask app. Docker Docs titled "Control startup and shutdown order in Compose" has an example script:

    #!/bin/sh
# wait-for-postgres.sh

set -e
  
host="$1"
shift
cmd="$@"
  
until PGPASSWORD=$POSTGRES_PASSWORD psql -h "$host" -U "postgres" -c 'q'; do
  >&2 echo "Postgres is unavailable - sleeping"
  sleep 1
done
  
>&2 echo "Postgres is up - executing command"
exec $cmd

    Make this script executable chmod +x wait-for-postgres.sh
    and change the flask startup command to be:

    command: ["./wait-for-postgres.sh", "db", "python", "app.py"]

    In your case, you would add the command in the flask_service section of the docker-compose.yml:

      flask_service:
  ...
    command: ["./wait-for-postgres.sh", "db", "bash", "entrypoint.sh"]
    Login or Signup to reply.
  2. 0

    You can use healthcheck dependencies on your containers.

    On your flask_application container, under dependencies, use:

    depends_on:
      db:
        condition: service_healthy

    Now you must implement a way to determine when postgres is healthy.
    To do that, you can run a healthcheck test command on your db container:

    db:
      healthcheck:
          test: ["CMD", "pg_isready", "-q"]

    In case you are getting some error like FATAL: role "root" does not exist, make use of this healthcheck command instead, and ensure your environment variables are set accordingly:

    db:
    healthcheck:
        test: [ "CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}" ]

    Your flask_application container will now not start until postgres is ready

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search