Question 258401 in Redis Question posted in
The official documentation can be found here.

Redis – Beat scheduler not working automatically but does work when manually triggered

I am learning to use celery. I want to automate API triggers using beat. The follwing code works if I manually call increment_value.delay() from the shell. However, it does not automate. Also, I think Redis still gets updated at every interval. Just the task method does not get called.

config.py

broker_url = f'redis://{REDIS_IP}:6379/0'
result_backend = f'redis://{REDIS_IP}:6379/0'
task_serializer = 'json'
result_serializer = 'json'
accept_content = ['json']
timezone = 'UTC'

CELERY_BEAT_SCHEDULE = {
    'run-every-15-seconds': {
        'task': 'scheduler.increment_value',
        'schedule': timedelta(seconds=15),
    },
}

scheduler.py

app = Celery()
app.config_from_object('config')


@app.task
def increment_value():
    response = requests.post(url=f"http://{FASTAPI_IP}:8000/redis/first")

    logging.info("Task Has Been Triggered")

    return response.json()

Dockerfile

FROM python:3.8-slim

WORKDIR /app

# Copy only the necessary files
COPY ./requirements.txt /app/requirements.txt
COPY . /app

# Install dependencies
RUN pip install --no-cache-dir -r requirements.txt

# Start Celery worker and beat in the background using a shell script
COPY ./start.sh /app/start.sh
RUN chmod +x /app/start.sh

CMD ["/app/start.sh"]

start.sh

#!/bin/sh
celery -A scheduler worker --loglevel=info &
celery -A scheduler beat --loglevel=info

Both worker and beat start when I run the application.

I should also mention that I am working on WSL2.

I would really appreciate if someone could point it out to me what I am doing wrong.

Tags:

2

Answers


  1. 0

    For a docker container to run, the running process must have a pid of 1.
    Your celery app might me running in the background.

    Also you can run only one service as the foreground.

    Alternatively You can use a process control set up. Supervisord is process control system, which will run as pid 1 as needed by the container to run and it can manage the execution on your celery worker and scheduler.

    Login or Signup to reply.
  2. 0

    You are using a different configuration variable to set beat schedule. Replace CELERY_BEAT_SCHEDULE with CELERYBEAT_SCHEDULE and see if it works. Also as @Shivam suggested, you can use supervisor instead of running worker in the background for better signal handling mechanisms.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search