skip to Main Content

I want to set-up liveness and readiness probes for Celery worker pods.
Since these worker pods doesn’t have a specific port associated to them I am finding it difficult.
Main Django app nginx server was easier to set-up.

I am very new to k8s so not much familiar to the different ways to do it.

2

Answers


  1. liveness probe for celery worker:
    This command only works when remote control is enabled.

    $ celery inspect ping -d <worker_name> --timeout=<timeout_time>
    

    When a celery worker uses a solo pool, healthcheck waits for the task to finish. In this case, you must increase the timeout waiting for a response.

    so in yaml:

          livenessProbe:
            initialDelaySeconds: 45
            periodSeconds: 60
            timeoutSeconds: <timeout_time>
            exec:
              command:
              - "/bin/bash"
              - "-c"
              - "celery inspect ping -d <worker_name> | grep -q OK"
    

    Of course you have to change the worker name and timeout to your own values

    Login or Signup to reply.
  2. Basically, configuration files in YAML for Celery liveness and readiness probes can be set up in a general way as described in Kubernetes docs. You can specifically adjust them for Celery pods based on your requirements, for example:

            livenessProbe:
              exec:
                # bash is needed to replace the environment variable
                command: [
                  "bash",
                  "-c",
                  "celery inspect ping -A apps -d celery@$HOSTNAME"
                ]
              initialDelaySeconds: 30
              periodSeconds: 60  
              timeoutSeconds: 10 
    

    There is still ongoing GitHub issue on this matter – other solutions that might be useful for your set-up are described here.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search