Question posted in
The official documentation can be found here.

Redis – celery worker throws AttributeError: module 'os' has no attribute 'register_at_fork'

I am running celery worker(version 4.4) on windows machine, when I run the worker with -P eventlet option it throws Attribute error.
Error logs are as follows:-

pipenv run celery worker -A src.celery_app -l info -P eventlet --without-mingle --without-heartbeat --without-gossip -Q queue1 -n worker1
Traceback (most recent call last):
  File "d:python37librunpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "d:python37librunpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:UsersG-US01.Test.virtualenvscelery-z5n-38VtScriptscelery.exe__main__.py", line 7, in <module>
  File "c:usersg-us01.test.virtualenvscelery-z5n-38vtlibsite-packagescelery__main__.py", line 14, in main
    maybe_patch_concurrency()
  File "c:usersg-us01.test.virtualenvscelery-z5n-38vtlibsite-packagescelery__init__.py", line 152, in maybe_patch_concurrency
    patcher()
  File "c:usersg-us01.test.virtualenvscelery-z5n-38vtlibsite-packagescelery__init__.py", line 109, in _patch_eventlet
    eventlet.monkey_patch()
  File "c:usersg-us01.test.virtualenvscelery-z5n-38vtlibsite-packageseventletpatcher.py", line 334, in monkey_patch
    fix_threading_active()
  File "c:usersg-us01.test.virtualenvscelery-z5n-38vtlibsite-packageseventletpatcher.py", line 331, in fix_threading_active
    _os.register_at_fork(
AttributeError: module 'os' has no attribute 'register_at_fork'

I have installed eventlet in virtual environment, the pipfile contents are as follows:-

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]
rope = "*"
autopep8 = "*"

[packages]
eventlet = "*"
psutil = "*"
celery = "*"
pythonnet = "*"
redis = "*"
gevent = "*"

[requires]
python_version = "3.7"

Please let me know where I am going wrong.

Tags:

4

Answers


  1. 0

    One reason you are facing this is because of the fact that Celery works on a pre-fork model.
    So if the underlying OS does not support it, you will have a tough time running celery. As per my knowledge, this model does not exist for the Windows kernel.

    You can still use Cygwin if you want to make it work on windows or have Linux subsystem available for Windows

    Sources:-

    1. How to run celery on windows?
    2. https://docs.celeryproject.org/en/stable/faq.html#does-celery-support-windows
    Login or Signup to reply.
  2. 0

    os.register_at_fork is a new function available since Python 3.7, it is only available for Unix systems (Source from Python doc) and Eventlet use it to patch threading library.

    There is an issue opened in Eventlet Github:
    https://github.com/eventlet/eventlet/issues/644

    So I don’t think Celery -A app worker -l info -P eventlet is any longer a good alternative for Windows.

    Personally I ran:

    Celery -A app worker -l info

    And it worked with Windows 10, Python 3.7 and Celery 4.4.7.

    Login or Signup to reply.
  3. 0

    is the eventlet version 0.26; pip install eventlet==0.26

    Login or Signup to reply.
  4. 0

    As hinted at by @金奕峰, os.register_at_fork was introduced in eventlet v0.27.0 c.f. commit compare on github. Specifying version 0.26.0 in your virtual environment’s package list might solve the problem (for now).

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search