Docker – TemplateDoesNotExist when rendering an email message with Celery in Django

I am developing an API for a forum website and have a classed-based view that sends an email message to a user in order to make his email confirmed. I have a celery task, that sends email and in the task I render data to string (email template). But when I call the task in my APIView, I get error that template does not exist. However, when I call it just as a regular function, without delay method it works as expected. Also it seems that there is no problem with Celery itself because I can run other tasks successfully. Here is the code of the view.

class RequestEmailToConfirmAPIView(GenericAPIView):

    permission_classes = [IsAuthenticated, EmailIsNotConfirmed]
    success_message = 'The message has just been delivered.'
    serializer_class = DummySerializer

    def get(self, request):
        current_url = get_current_site(request, path='email-confirmation-result') 

        user = request.user

        token = EmailConfirmationToken.objects.create(user=user)

        send_confirmation_email.delay(
            template_name='email/confirm_email.txt', current_url=current_url,
            email=user.email, token_id=token.id, user_id=user.id
        )

        return Response(data={"message": self.success_message}, status=status.HTTP_201_CREATED)

note: get_current_site function returns the current url including protocol (http) in order to create the url for a user to click and confirm his email

the code of send_confirmation_email task:

@shared_task
def send_confirmation_email(template_name: str, current_url: str, email: str, token_id: int, user_id: int):
    """
    Отправляет письмо для подтверждения определенных действий.
    """
    data = {
        'current_site': str(current_url),
        'token_id': str(token_id),
        'user_id': str(user_id)
    }

    # message = get_template(template_name).render(data)

    message = render_to_string(template_name, context=data)
    send_mail(
        subject='Пожалуйста, подтвердите почту',
        message=message,
        from_email='[email protected]',
        recipient_list=[email],
        fail_silently=True
    )

here is the folder structure (the API view and the task is placed in forum package):

├── backend
│   ├── admin_info.txt
│   ├── celery_app.py
│   ├── core
│   │   ├── asgi.py
│   │   ├── __init__.py
│   │   ├── settings.py
│   │   ├── urls.py
│   │   └── wsgi.py
│   ├── Dockerfile
│   ├── fixture.json
│   ├── forum
│   │   ├── admin.py
│   │   ├── apps.py
│   │   ├── helpers.py
│   │   ├── __init__.py
│   │   ├── logic.py
│   │   ├── management
│   │   │   ├── commands
│   │   │   │   ├── createdata.py
│   │   │   │   └── __init__.py
│   │   │   └── __init__.py
│   │   ├── models.py
│   │   ├── permissions.py
│   │   ├── serializers.py
│   │   ├── urls.py
│   │   ├── utils.py
│   │   ├── validators.py
│   │   └── views.py
│   ├── manage.py
│   ├── middleware.py
│   ├── requirements.txt
│   ├── templates
│   │   └── email
│   │       ├── confirm_email.txt
│   │       ├── password_reset_email.html
│   │       ├── password_reset_email.txt
│   │       └── restore_account.txt
│   └── web.env
├── celerybeat-schedule
├── docker-compose.yml
├── nginx
│   ├── Dockerfile
│   ├── nginx.conf
│   └── proxy_params
├── README.md
└── setup.cfg

here is celery_app config:

from celery import Celery
from django.conf import settings

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'core.settings')

app = Celery('backend.core')
app.config_from_object('django.conf:settings')
app.conf.broker_url = settings.CELERY_BROKER_URL
app.autodiscover_tasks()

and docker-compose.yml:

services:

  # PostgreSQL
  db:
    image: postgres:14.1-alpine
    restart: always
    env_file:
      - backend/web.env
    ports:
      - '5432:5432'
    volumes:
      - db:/var/lib/postgresql/data

  # Backend Django
  backend:
    build:
      context: ./backend
    command:
      bash -c "./manage.py collectstatic --noinput && ./manage.py migrate && 
      gunicorn -b 0.0.0.0:8000 core.wsgi:application —log-level debug"
    ports:
      - '8000:8000'
    depends_on:
      - db
    env_file:
      - backend/web.env
    volumes:
      - ./backend/:/backend
      - static_volume:/backend/static
      - media_volume:/backend/media

  # Frontend Next.js
  frontend:
    build:
      context: ./frontend
    depends_on:
      - backend
    volumes:
      - .:/app/frontend
      - /app/node_modules
      - /app/.next
    ports:
      - '3000:3000'

  # Nginx
  nginx:
    build:
      dockerfile: ./Dockerfile
      context: ./nginx
    ports:
      - '80:80'
    volumes:
      - static_volume:/backend/static
      - media_volume:/backend/media
    depends_on:
      - backend
      - frontend
      - db

  # Redis
  redis:
    image: redis:7.0.5-alpine
    container_name: redis

  # Celery Worker 1
  celery-worker:
    restart: always
    build:
      context: ./backend
    entrypoint: /bin/sh
    command: -c "export PYTHONPATH=/backend:/backend/backend && celery -A backend.celery_app:app worker --loglevel=info"
    volumes:
      - .:/backend
    container_name: celery-worker
    depends_on:
      - db
      - redis
      - backend
    links:
      - redis
    env_file:
      - backend/web.env

  celery-beat:
    restart: always
    build:
      context: ./backend
    entrypoint: /bin/sh
    command: -c "export PYTHONPATH=/backend:/backend/backend && celery -A backend.celery_app:app beat --loglevel=info"
    volumes:
      - .:/backend
    container_name: celery-beat
    depends_on:
      - db
      - redis
      - backend
    links:
      - redis
    env_file:
      - backend/web.env


  # Flower
  flower:
    image: docker.io/mher/flower
    command: ["celery", "--broker=redis://redis:6379/0", "flower", "--port=5555"]
    ports:
      - 5555:5555



volumes:
  db:
    driver: local
  react_build:
  static_volume:
    driver: local
  media_volume:
    driver: local

and the traceback:

celery-worker               | [2023-10-13 19:02:04,994: WARNING/ForkPoolWorker-2] Message: 'Task %(name)s[%(id)s] %(description)s: %(exc)s'
celery-worker               | Arguments: {'hostname': 'celery@c4d7e87c7784', 'id': 'dc04404e-3021-46c1-b6e6-385b52125953', 'name': 'accounts.tasks.send_confirmation_email', 'exc': "TemplateDoesNotExist('email/confirm_email.txt')", 'traceback': 'Traceback (most recent call last):n  File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 451, in trace_taskn    R = retval = fun(*args, **kwargs)n                 ^^^^^^^^^^^^^^^^^^^^n  File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 734, in __protected_call__n    return self.run(*args, **kwargs)n           ^^^^^^^^^^^^^^^^^^^^^^^^^n  File "/backend/backend/accounts/tasks.py", line 19, in send_confirmation_emailn    message = render_to_string(template_name, context=data)n              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^n  File "/usr/local/lib/python3.11/site-packages/django/template/loader.py", line 61, in render_to_stringn    template = get_template(template_name, using=using)n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^n  File "/usr/local/lib/python3.11/site-packages/django/template/loader.py", line 19, in get_templaten    raise TemplateDoesNotExist(template_name, chain=chain)ndjango.template.exceptions.TemplateDoesNotExist: email/confirm_email.txtn', 'args': '[]', 'kwargs': "{'template_name': 'email/confirm_email.txt', 'current_url': 'http://localhost:8000/api/v1/account/email-confirmation-result/', 'email': '[email protected]', 'token_id': UUID('6ac7db2a-26bb-45aa-beea-187c56cd25c5'), 'user_id': 22}", 'description': 'raised unexpected', 'internal': False}