Question 186092 in Redis Question posted in
The official documentation can be found here.

Redis – dbbackup with docker/celery/celery-beat not working: [Errno 2] No such file or directory: 'pg_dump'

I am setting up a backup to use dbbackup. However, I am receiving an error when backing up my data. There is a similar question where the person was able to resolve it, however, the answer doesn’t show how. here

My dbbackup version is django-dbbackup==4.0.2

Please find below my dockerfile:

  database:
    build:
      context: .
      dockerfile: pg-Dockerfile
    expose:
      - "5432"
    restart: always
    volumes:
      - .:/myworkdir
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: password
      POSTGRES_DB: faranga_db
  redis:
    image: redis
    restart: on-failure
    volumes:
      - .:/myworkdir
    expose:
      - "6379"
  celery:
    build: .
    restart: on-failure
    command: bash -c "sleep 10; celery -A project worker -l info"
    volumes:
      - .:/myworkdir
    env_file:
      - .env
    depends_on:
      - database
      - redis
  beat:
    build: .
    restart: on-failure
    command: bash -c "sleep 10; celery -A project beat -l info --pidfile=/tmp/celeryd.pid"
    volumes:
      - .:/myworkdir
    env_file:
      - .env
    depends_on:
      - database
      - redis

my celery task:

@app.task
def database_backup():
    management.call_command('dbbackup')

# media backup works just fine
@app.task
def media_backup():
    management.call_command('mediabackup')

DB backup settings

# django db backup https://django-dbbackup.readthedocs.io/en/master/installation.html
DBBACKUP_STORAGE = 'django.core.files.storage.FileSystemStorage'
DBBACKUP_STORAGE_OPTIONS = {'location': '~/myworkdir/backups/db/'}


def backup_filename(databasename, servername, datetime, extension, content_type):
    pass


DBBACKUP_FILENAME_TEMPLATE = backup_filename
DBBACKUP_CONNECTOR = "dbbackup.db.postgresql.PgDumpBinaryConnector"

Error stack trace:

[2023-02-09 14:44:00,052: ERROR/ForkPoolWorker-6] CommandConnectorError: Error running:  pg_dump --dbname=postgresql://postgres:password@database:5432/faranga_db --format=custom 
faranga-celery-1     | [Errno 2] No such file or directory: 'pg_dump'
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/dbbackup/utils.py", line 120, in wrapper
faranga-celery-1     |     func(*args, **kwargs)
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/dbbackup/management/commands/dbbackup.py", line 93, in handle
faranga-celery-1     |     self._save_new_backup(database)
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/dbbackup/management/commands/dbbackup.py", line 106, in _save_new_backup
faranga-celery-1     |     outputfile = self.connector.create_dump()
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/dbbackup/db/base.py", line 92, in create_dump
faranga-celery-1     |     return self._create_dump()
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/dbbackup/db/postgresql.py", line 112, in _create_dump
faranga-celery-1     |     stdout, stderr = self.run_command(cmd, env=self.dump_env)
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/dbbackup/db/base.py", line 180, in run_command
faranga-celery-1     |     raise exceptions.CommandConnectorError(
faranga-celery-1     | 
faranga-celery-1     | [2023-02-09 14:44:00,080: ERROR/ForkPoolWorker-6] Task administration.tasks.database_backup[601e33a6-0eef-42c0-a355-cb7d33d7ebaa] raised unexpected: CommandConnectorError("Error running:  pg_dump --dbname=postgresql://postgres:password@database:5432/faranga_db --format=custom n[Errno 2] No such file or directory: 'pg_dump'")
faranga-celery-1     | Traceback (most recent call last):
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/dbbackup/db/base.py", line 165, in run_command
faranga-celery-1     |     process = Popen(
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/sentry_sdk/integrations/stdlib.py", line 201, in sentry_patched_popen_init
faranga-celery-1     |     rv = old_popen_init(self, *a, **kw)
faranga-celery-1     |   File "/usr/lib/python3.10/subprocess.py", line 969, in __init__
faranga-celery-1     |     self._execute_child(args, executable, preexec_fn, close_fds,
faranga-celery-1     |   File "/usr/lib/python3.10/subprocess.py", line 1845, in _execute_child
faranga-celery-1     |     raise child_exception_type(errno_num, err_msg, err_filename)
faranga-celery-1     | FileNotFoundError: [Errno 2] No such file or directory: 'pg_dump'
faranga-celery-1     | 
faranga-celery-1     | During handling of the above exception, another exception occurred:
faranga-celery-1     | 
faranga-celery-1     | Traceback (most recent call last):
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/celery/app/trace.py", line 451, in trace_task
faranga-celery-1     |     R = retval = fun(*args, **kwargs)
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/sentry_sdk/integrations/celery.py", line 207, in _inner
faranga-celery-1     |     reraise(*exc_info)
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/sentry_sdk/_compat.py", line 56, in reraise
faranga-celery-1     |     raise value
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/sentry_sdk/integrations/celery.py", line 202, in _inner
faranga-celery-1     |     return f(*args, **kwargs)
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/celery/app/trace.py", line 734, in __protected_call__
faranga-celery-1     |     return self.run(*args, **kwargs)
faranga-celery-1     |   File "/myworkdir/administration/tasks.py", line 8, in database_backup
faranga-celery-1     |     management.call_command('dbbackup')
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/django/core/management/__init__.py", line 181, in call_command
faranga-celery-1     |     return command.execute(*args, **defaults)
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 398, in execute
faranga-celery-1     |     output = self.handle(*args, **options)
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/dbbackup/utils.py", line 120, in wrapper
faranga-celery-1     |     func(*args, **kwargs)
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/dbbackup/management/commands/dbbackup.py", line 93, in handle
faranga-celery-1     |     self._save_new_backup(database)
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/dbbackup/management/commands/dbbackup.py", line 106, in _save_new_backup
faranga-celery-1     |     outputfile = self.connector.create_dump()
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/dbbackup/db/base.py", line 92, in create_dump
faranga-celery-1     |     return self._create_dump()
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/dbbackup/db/postgresql.py", line 112, in _create_dump
faranga-celery-1     |     stdout, stderr = self.run_command(cmd, env=self.dump_env)
faranga-celery-1     |   File "/usr/local/lib/python3.10/dist-packages/dbbackup/db/base.py", line 180, in run_command
faranga-celery-1     |     raise exceptions.CommandConnectorError(
faranga-celery-1     | dbbackup.db.exceptions.CommandConnectorError: Error running:  pg_dump --dbname=postgresql://postgres:password@database:5432/faranga_db --format=custom 
faranga-celery-1     | [Errno 2] No such file or directory: 'pg_dump'
Tags:

2

Answers


  1. 0

    If you´re running this command inside your project container, you don´t have access to the postgres binaries to dump and restore your database. Try access your postgres container and call this command in shell.

    You could get you database container id from docker ps and access your container shell with docker exec -it [your_container_id] bash . Then try to call pg_dump from inside of it. This should probably work

    Login or Signup to reply.
  2. 0

    Maybe a bit late to answer, I had met this issue before and solved it by adding pg_dump related packages into the django container image (also the celery worker image) in which the dbbackup command is going to execute.

    In my case, to avoid always connecting to internet to install the packages when re-build the image, I downloaded the needed debs to local, so my Dockerfile is like:

    FROM python:3.9.16

...

COPY --chown=1001 . /opt/backend/
RUN chmod +x /opt/backend/deploy/*.deb
RUN apt-get install -y /opt/backend/deploy/*.deb
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search