I have a legacy Django project (django-1.1.29) and some tests in pytest (pytest-4.3.0), all running inside Docker. The database is PostgreSQL 10 and is a docker-compose service on which the application depends on. The python version is 2.7.18.
Lately the tests started to fail with a strange error:
InternalError: variable not found in subplan target list
.
The error occurs only when I count the number of objects of a certain model, e.g. Problem.objects.count()
. The instruction is turned into the following query
(0.000) SELECT COUNT(*) AS "__count" FROM "problems_problem"; args=()
by Django.
The whole log is here:
self = <integration_tests.project_name.subjects.test_views.test_theme_problem_view_set.TestThemeProblemViewSet object at 0x7f01faccbe50>
jclient = <project_name.common.test_utils.json_client.JSONClient object at 0x7f01fadb8ed0>
def test_all_is_ok(self, jclient, subject_model, content_manager):
url, data = self._main_prepare(jclient, subject_model, content_manager)
response = jclient.post_json(url, data[0])
assert response.status_code == 201
> assert Problem.objects.count() == 1
integration_tests/project_name/subjects/test_views/test_theme_problem_view_set.py:86:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python2.7/dist-packages/django/db/models/manager.py:85: in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
/usr/local/lib/python2.7/dist-packages/django/db/models/query.py:364: in count
return self.query.get_count(using=self.db)
/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py:499: in get_count
number = obj.get_aggregation(using, ['__count'])['__count']
/usr/local/lib/python2.7/dist-packages/django/db/models/sql/query.py:480: in get_aggregation
result = compiler.execute_sql(SINGLE)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <django.db.models.sql.compiler.SQLCompiler object at 0x7f01faee3a50>
result_type = 'single', chunked_fetch = False
def execute_sql(self, result_type=MULTI, chunked_fetch=False):
"""
Run the query against the database and returns the result(s). The
return value is a single data item if result_type is SINGLE, or an
iterator over the results if the result_type is MULTI.
result_type is either MULTI (use fetchmany() to retrieve all rows),
SINGLE (only retrieve a single row), or None. In this last case, the
cursor is returned if any query is executed, since it's used by
subclasses such as InsertQuery). It's possible, however, that no query
is needed, as the filters describe an empty set. In that case, None is
returned, to avoid any unnecessary database interaction.
"""
if not result_type:
result_type = NO_RESULTS
try:
sql, params = self.as_sql()
if not sql:
raise EmptyResultSet
except EmptyResultSet:
if result_type == MULTI:
return iter([])
else:
return
if chunked_fetch:
cursor = self.connection.chunked_cursor()
else:
cursor = self.connection.cursor()
try:
cursor.execute(sql, params)
except Exception as original_exception:
try:
# Might fail for server-side cursors (e.g. connection closed)
cursor.close()
except Exception:
# Ignore clean up errors and raise the original error instead.
# Python 2 doesn't chain exceptions. Remove this error
# silencing when dropping Python 2 compatibility.
pass
> raise original_exception
E InternalError: variable not found in subplan target list
/usr/local/lib/python2.7/dist-packages/django/db/models/sql/compiler.py:899: InternalError
I would really appreciate if anyone could help me fix this issue or at least give some hints where to find the reasons.
Here is what I’ve attempted to do so far but none of these helped:
- Use different versions of PostgreSQL (10, 11, 12, 13)
- Disable the server-side cursor
2
Answers
It appears postgres has shipped something broken into all their minor versions:
https://www.postgresql.org/message-id/2121219.1644607692%40sss.pgh.pa.us
For example, the postgres:12 dockerhub is now the same as the postgres:12.10 dockerhub image rather than 12.9. If you explicitly specify postgres:12.9 (or the previous minor release for any of the other versions), I believe it will start working.
I had the same issue. Temporary fix but what I did was modify the query performed by the admin.
Try this in your admin.py
Edit: This only fixes issues you may have with your django admin