Question posted in
The official documentation can be found here.

Failing to authenticate to redis with Airflow Celery worker

I am trying to run Airflow Celery worker with redis backend but cant get the connection between the worker and redis to work.

Here is how the configuration looks like:

broker_url = redis://:<my-password>@localhost:6379/0

I am getting error:

ValueError: invalid literal for int() with base 10: 'my-password-string'

Anyone know any fix for this?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    So i solved it now.

    My password was generated from random characters piped through base64 and it was causing problems.

    I changed my password to bit shorter and simplier one and airflow worker now runs normally.


  2. 0

    That’s because there’re special characters in your password, you can simply do this:
    e.g. your password is my#pw#

    1. do url-encoding, then you get my%23pw%23
    2. append another % before % to allow airflow read the config successfully

    then the final password is my%%23pw%%23, so e.g. the redis broker url in airflow.cfg should be like this:

    broker_url = redis://:my%%23pw%%23@redis-host:6379/0

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search