Question posted in
The official documentation can be found here.

Redis – SynchronousOnlyOperation error when upgrading from python 3.6 to python 3.7 using django channels

I’ve been attempting to upgrade python from 3.6 to 3.7 for our Django & Django channels application. With that change, Django throws a SynchronousOnlyOperation anytime any HTTP request is made (even if it has nothing to do with WebSockets). My guess is that somehow the python upgrade has made the Django check more strict.

I believe that Django channels are serving both the HTTP requests and WebSocket requests so it expects all code to be async compliant.

How do I get Django channels runserver to run the wsgi app synchronously, while the channel consumers asynchronously?

# project/asgi.py
application = ProtocolTypeRouter({"http": get_asgi_application(), "websocket": routing})

# project/wsgi.py
application = get_wsgi_application()

Stacktrace:
It’s clear to me that the auth middleware that is running for a normal wsgi view is not async compliant, how can I get it to run in a sync environment?

ERROR    Internal Server Error: /a/api
Traceback (most recent call last):
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/sessions/backends/base.py", line 233, in _get_session
    return self._session_cache
AttributeError: 'SessionStore' object has no attribute '_session_cache'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/___/___/___/___/apps/core/middleware.py", line 60, in __call__
    request.is_user_verified = request.user.is_verified()
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/utils/functional.py", line 240, in inner
    self._setup()
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/utils/functional.py", line 376, in _setup
    self._wrapped = self._setupfunc()
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django_otp/middleware.py", line 38, in _verify_user
    user.otp_device = None
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/utils/functional.py", line 270, in __setattr__
    self._setup()
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/utils/functional.py", line 376, in _setup
    self._wrapped = self._setupfunc()
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/auth/middleware.py", line 24, in <lambda>
    request.user = SimpleLazyObject(lambda: get_user(request))
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/auth/middleware.py", line 12, in get_user
    request._cached_user = auth.get_user(request)
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/auth/__init__.py", line 174, in get_user
    user_id = _get_user_session_key(request)
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/auth/__init__.py", line 58, in _get_user_session_key
    return get_user_model()._meta.pk.to_python(request.session[SESSION_KEY])
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/sessions/backends/base.py", line 65, in __getitem__
    return self._session[key]
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/sessions/backends/base.py", line 238, in _get_session
    self._session_cache = self.load()
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/sessions/backends/db.py", line 43, in load
    s = self._get_session_from_db()
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/contrib/sessions/backends/db.py", line 34, in _get_session_from_db
    expire_date__gt=timezone.now()
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/db/models/query.py", line 425, in get
    num = len(clone)
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/db/models/query.py", line 269, in __len__
    self._fetch_all()
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/db/models/query.py", line 1303, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1154, in execute_sql
    cursor = self.connection.cursor()
  File "/___/___/.pyenv/versions/pd37/lib/python3.7/site-packages/django/utils/asyncio.py", line 24, in inner
    raise SynchronousOnlyOperation(message)
django.core.exceptions.SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async.

Versions

Django==3.1.1
channels==3.0.2
channels-redis==3.2.0
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    Finally got to the bottom of this. For us, we were using gevent and had this line in our manage.py

    # manage.py

import gevent

gevent.patch_all()

    For whatever reason, that does not play nicely with python 3.7+, Django channels, and Django (where python 3.6, Django channels, and Django worked fine).

    We were lucky and were able to remove this from our codebase; we did not figure out how to fix the gevent patch.


  2. 0

    So a summary of what I’ve tried:

    • First, I have a channels app that had no database at all, no contrib apps except staticfiles.
    • I added database, DRF and one app with a single model and list/detail views
    • Enabled the sync http consumer:
    # dwtools.routing
from channels.routing import ProtocolTypeRouter, URLRouter
from django.core.asgi import get_asgi_application

import romanize.routing

application = ProtocolTypeRouter(
    {
        "websocket": URLRouter(romanize.routing.websocket_urlpatterns),
        "http": get_asgi_application(),
    },
)
    • settings:
    INSTALLED_APPS = [
    "channels",
    "romanize.apps.RomanizeConfig",
    "django.contrib.staticfiles",
    "rest_framework",
    "agencies.apps.AgenciesConfig",
]
WSGI_APPLICATION = "dwtools.wsgi.application"
ASGI_APPLICATION = "dwtools.routing.application"

    I can pull up DRF’s standard views, create entries etc.

    Then added a simple middleware that accesses the database (gets a count of the number of Agencies and puts it in request). Not even using MiddlewareMixin, just barebones init can call protocol.

    System check identified no issues (0 silenced).
December 07, 2020 - 14:38:56
Django version 3.1.4, using settings 'dwtools.settings'
Starting ASGI/Channels version 3.0.2 development server at http://127.0.0.1:3401/
#             ^^^^^^^^               ^^^^^^^^^^^ ( Channels runserver)
Quit the server with CONTROL-C.
HTTP GET /agencies/ 200 [0.05, 127.0.0.1:60460]

    The stack uses local nginx to route to port 3401 for /ws (websocket) and /agencies (http). Perhaps you spot something…

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search