Question 235520 in Docker Question posted in
The official documentation can be found here.

Docker – Error response from daemon: failed to create task for container

I have a django app. That also has redis, celery and flower.

Everything is working on my local machine.

But When I am trying to dockerize it The redis and django app is starting. But celery and flower is failing to start.

It is giving me this error while starting celery:

Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "celery -A core worker -P eventlet --autoscale=10,1 -l INFO": executable file not found in $PATH: unknown

Dockerfile

# For more information, please refer to https://aka.ms/vscode-docker-python
FROM python:bullseye

EXPOSE 8000

RUN apt update && apt upgrade -y
# && apk install cron iputils-ping sudo nano -y

# Install pip requirements
COPY requirements.txt .
RUN python -m pip install -r requirements.txt
RUN rm requirements.txt

WORKDIR /app
COPY ./src /app
RUN mkdir "log"

# Set the environment variables
ENV PYTHONUNBUFFERED=1
ENV DJANGO_SETTINGS_MODULE=core.settings


# Creates a non-root user with an explicit UID and adds permission to access the /app folder
# For more info, please refer to https://aka.ms/vscode-docker-python-configure-containers
RUN adduser -u 5678 --disabled-password --gecos "" appuser && chown -R appuser /app
# RUN echo 'appuser ALL=(ALL) NOPASSWD: ALL' >  /etc/sudoers.d/appuser
USER appuser


# During debugging, this entry point will be overridden. For more information, please refer to https://aka.ms/vscode-docker-python-debug
ENTRYPOINT ["sh", "entrypoint.sh"]
CMD ['gunicorn', 'core.wsgi:application', '--bind', '0.0.0.0:8000']

entrypoint.sh

#!/bin/sh

echo "Apply database migrations"
python manage.py migrate

# Start server
echo "Starting server"
# run the container CMD
exec "$@"

docker-compose.yml

version: "3.8"

services:
  mailgrass_backend:
    container_name: mailgrass_backend
    restart: unless-stopped
    ports:
      - "8000:8000"
    volumes:
      - npm:/app
      - ./.env:/app/.env:ro
    networks:
      - npm
    env_file: .env
    depends_on:
      - redis
    build:
      context: .
      dockerfile: ./Dockerfile
  redis:
    container_name: redis
    image: redis:7.0-alpine
    restart: unless-stopped
    env_file: .env
    ports:
      - "6379:6379"
    command:
      - 'redis-server'
    networks:
      - npm
  celery:
    container_name: celery
    restart: unless-stopped
    env_file: .env
    volumes:
      - npm:/app
      - ./.env:/app/.env:ro
    build:
      context: .
      dockerfile: ./Dockerfile
    networks:
      - npm
    depends_on:
      - redis
    entrypoint:
      - "celery -A core worker -P eventlet --autoscale=10,1 -l INFO"
  flower:
    container_name: flower
    restart: unless-stopped
    ports:
      - "5555:5555"
    env_file: .env
    volumes:
      - npm:/app
      - ./.env:/app/.env:ro
    build:
      context: .
      dockerfile: ./Dockerfile
    networks:
      - npm
    depends_on:
      - redis
      - celery
    entrypoint:
      - "celery -b redis://redis:6379 flower"
volumes:
  npm:
  postgres:
networks:
  npm:

requirements.txt

amqp==5.1.1
asgiref==3.7.2
attrs==23.1.0
billiard==4.1.0
black==23.7.0
celery==5.3.1
certifi==2023.7.22
cffi==1.15.1
charset-normalizer==3.2.0
click==8.1.6
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.3.0
cron-descriptor==1.4.0
cryptography==41.0.3
defusedxml==0.7.1
dj-crontab==0.8.0
dj-rest-auth==4.0.1
Django==4.2.4
django-allauth==0.54.0
django-annoying==0.10.6
django-celery-beat==2.5.0
django-cleanup==8.0.0
django-cors-headers==4.2.0
django-debug-toolbar==4.2.0
django-filter==23.2
django-phonenumber-field==7.1.0
django-timezone-field==5.1
djangorestframework==3.14.0
djangorestframework-simplejwt==5.2.2
dnspython==2.4.1
drf-spectacular==0.26.4
email-validator==2.0.0.post2
eventlet==0.33.3
flower==2.0.1
greenlet==2.0.2
humanize==4.7.0
idna==3.4
inflection==0.5.1
isort==5.12.0
jsonschema==4.18.6
jsonschema-specifications==2023.7.1
kombu==5.3.1
mailchecker==5.0.9
Markdown==3.4.4
mypy-extensions==1.0.0
oauthlib==3.2.2
packaging==23.1
pathspec==0.11.2
phonenumberslite==8.13.18
Pillow==10.0.0
platformdirs==3.10.0
prometheus-client==0.17.1
prompt-toolkit==3.0.39
pycparser==2.21
PyJWT==2.8.0
pyotp==2.9.0
python-crontab==3.0.0
python-dateutil==2.8.2
python-dotenv==1.0.0
python3-openid==3.2.0
pytz==2023.3
PyYAML==6.0.1
redis==4.6.0
referencing==0.30.2
requests==2.31.0
requests-oauthlib==1.3.1
rpds-py==0.9.2
six==1.16.0
sqlparse==0.4.4
tornado==6.3.2
tzdata==2023.3
uritemplate==4.1.1
urllib3==2.0.4
vine==5.0.0
wcwidth==0.2.6
whitenoise==6.5.0
gunicorn

What am I doing wrong here?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I have found the problem in my docker-compose.yml.

    I had defined entrypoint like this:

    entrypoint:
      - "celery -A core worker -P eventlet --autoscale=10,1 -l INFO"

    But the correct way to define this in one line is this:

    entrypoint: "celery -A core worker -P eventlet --autoscale=10,1 -l INFO"

  2. 0

    A Unix command is made of a list of words. In Docker and in Compose, there are two syntaxes to specify commands, one that uses a bare string and one that takes an explicit list of words.

    In your Compose file, you’re using the list syntax

    entrypoint:
  - "celery -A core worker ..."

    and so Compose believes you’re providing it with the complete list of shell words. There is no splitting, expansion, or any other processing that happens here. It looks for a single executable file like /usr/bin/celery -A core worker ... including spaces in the filename, and when it doesn’t find it you get that error.

    You’ve already figured out that using a simple string will work here. Note that Compose has its own string-splitting rules that are similar but not identical to what a POSIX shell will do.

    entrypoint: celery -A core worker ...

    You can also split this into words yourself.

    entrypoint:
  - celery
  - -A
  - core
  - worker
  ...

    Your Dockerfile uses the entrypoint-wrapper pattern, which means you probably don’t want to override the entrypoint: in the Compose file; this causes the wrapper script to not run. If you override command: instead, it will get passed as arguments to the entrypoint script, and the exec "$@" line at the end of that script will run that command.

    The Dockerfile also supports "shell command string" vs. "list of words" mode. The "list of words" must be a syntactically valid JSON array. If it’s not, Docker interprets it as a command string and wraps it in /bin/sh -c. Your CMD line has single quotes and not double quotes, so it’s not valid JSON; if you tried to run the image without overriding the command you’d probably get an error like [gunicorn not found. Make sure to use double quotes here.

    I’d also consider a substantial cleanup of the Compose file. There’s no reason to mount a named volume over containers’ /app directories, and this can prevent Docker from seeing code changes. Compose provides a network named default which is fine for most use cases. You don’t need to specify container_name:, and you can use a shorter form of build: if you have all default options.

    services:
  celery:
    restart: unless-stopped
    env_file: .env
    build: .
    depends_on:
      - redis
    entrypoint: "celery -A core worker -P eventlet --autoscale=10,1 -l INFO"
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search