I have created a project github repository that has Docker, Django, React, Redis, Celery:
https://github.com/axilaris/dockerize-django-react-celery-redis. My goal is to get celery working correctly with the logo appearing and background task log prints.
This is based from this Docker – React and Django example tutorial code:
https://github.com/dotja/authentication_app_react_django_rest
And trying to use Docker – Celery & Redis from this tutorial code:
https://github.com/veryacademy/docker-mastery-with-django/tree/main/Part-4%20Django%20Postgres%20Redis%20and%20Celery <– Part 4 tutorial for Celery & Redis
Here is the docker-compose.yaml for the redis & celery part:
# Redis
redis:
image: redis:alpine
container_name: redis
ports:
- "6379:6379"
# celery
celery:
restart: always
build:
context: ./backend
command: celery -A backend worker -l DEBUG
volumes:
- .:/django
container_name: celery
depends_on:
- redis
- backend
here is how I implemented the background tasks in user_api/tasks.py (https://github.com/axilaris/dockerize-django-react-celery-redis/blob/main/backend/user_api/tasks.py)
in backend/user_api/tasks.py
from __future__ import absolute_import, unicode_literals
from celery import shared_task
import logging
@shared_task
def add(x, y):
logging.debug("XXX add")
return x + y
my project should work very simply by running:
docker-compose build
docker-compose up
However, there is no celery logo (1) and I dont see the logs print on the background process (2). I think it is executing as result.ready returns. But I want these 2 for completeness. (eventhough result.ready maybe be working with celery executing background task)
It did not print this celery logo. (you can check the full log prints for docker-compose up – https://gist.github.com/axilaris/a2776fc8f39e53bbc93e0d7a4e0ba0f5):
celery | -------------- celery@b755a7cdba8d v5.3.6 (emerald-rush)
celery | --- ***** -----
celery | -- ******* ---- Linux-6.6.12-linuxkit-aarch64-with 2024-03-02 20:48:06
celery | - *** --- * ---
celery | - ** ---------- [config]
celery | - ** ---------- .> app: core:0xffff9bbd7550
celery | - ** ---------- .> transport: redis://redis:6379//
celery | - ** ---------- .> results: redis://redis:6379/
celery | - *** --- * --- .> concurrency: 10 (prefork)
celery | -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
celery | --- ***** -----
celery | -------------- [queues]
celery | .> celery exchange=celery(direct) key=celery
and here is how I execute the background task, but there is no log prints in django logs on the background task. (you can find the full log prints https://gist.github.com/axilaris/a2776fc8f39e53bbc93e0d7a4e0ba0f5)
>>> % docker exec -it backend_container sh
>>> /app # python manage.py shell
>>> Python 3.9.18 (main, Jan 27 2024, 07:18:02)
>>> [GCC 13.2.1 20231014] on linux
>>> Type "help", "copyright", "credits" or "license" for more information.
>>> (InteractiveConsole)
>>> from user_api.tasks import add
>>> result = add.delay(2, 2)
>>> result.ready
<bound method AsyncResult.ready of <AsyncResult: 9046dd90-f44d-4eba-9881-acc0fbc4278a>>
My goal is to see that celery logo print above and to verify in the django logs that the background process is executed just like in (https://youtu.be/oBQxFn1CDno?si=58ZRfLZeuCC8fz01&t=1204 at 20:04).
2
Answers
Not sure if you’re looking for logo such Redis ASCII logo shown below in your terminal.
In any case, here’s what I went through.
The full logs in your GitHub Gist shows that Celery is functioning correctly, but it doesn’t display the Celery logo because it’s a background process.
Celery Functionality:
Missing Celery Logo:
Celery itself doesn’t have a graphical user interface (GUI) that displays a logo. It operates in the background, handling tasks asynchronously.
Background Task Logs:
The logs show successful execution of background tasks is likely because result.ready returns.
Debug Logs:
I see you enabled debugging mode in your
docker-compose.yaml
file and recommend to check the celery logsdocker-compose logs celery --tail=1000 -f
in order to check any missing logs.Please let me know what you expect to see such as svg logo or ASCII logs if something I’m missing.
I noticed that on some machines i could only start celery if i provide also name of worker, scheduller, queue and concurrency, try to run celery as follows:
also don’t forget to configure celery at start like this:
probably
CELERY_BROKER_URL
&&CELERY_RESULT_BACKEND
you should set from env variables, in my case i set them in settings file.Also it might be helpful to attach you image by id and try to start celery from cli.