"InterfaceError: connection already closed" being raised when a test is run

[ozgurisil]

Hello,

First off, congratulations on this awesome app. I am using Python 2.7, Django 1.9 and Postgresql 9.3.

Settings dict for django-q is as follows:

Q_CLUSTER = {
    'name': 'pulsecheck',
    'workers': 2,
    'recycle': 500,
    'compress': True,
    'save_limit': 250,
    'label': 'Task Management',
    'redis': {
        'host': '127.0.0.1',
        'port': 6379,
        'db': 0, }
}

This is what I have in tests.py:

Conf.SYNC = True

...

def test_stuff(self):
    ...
    async('some_stuff', var1, var2)
    ...
    assert ...

When tests are run, this is what I get:

Creating test database for alias 'default'...
16:53:03 [Q] INFO MainProcess ready for work at 30995
16:53:03 [Q] INFO MainProcess processing [stairway-iowa-solar-lactose]
16:53:03 [Q] INFO MainProcess stopped doing work
16:53:03 [Q] INFO MainProcess monitoring at 30995
16:53:03 [Q] ERROR connection already closed
16:53:03 [Q] ERROR Failed [stairway-iowa-solar-lactose] - connection already closed
16:53:03 [Q] INFO MainProcess stopped monitoring results
E
======================================================================
ERROR: test_activation_successful (core.tests.AuthTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ozgurisil/dev/pulsecheck_env/src/pulsecheck/core/tests.py", line 358, in test_activation_successful
    user = User.objects.get(username='testuser')
  File "/home/ozgurisil/dev/pulsecheck_env/local/lib/python2.7/site-packages/django/db/models/manager.py", line 122, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/ozgurisil/dev/pulsecheck_env/local/lib/python2.7/site-packages/django/db/models/query.py", line 381, in get
    num = len(clone)
  File "/home/ozgurisil/dev/pulsecheck_env/local/lib/python2.7/site-packages/django/db/models/query.py", line 240, in __len__
    self._fetch_all()
  File "/home/ozgurisil/dev/pulsecheck_env/local/lib/python2.7/site-packages/django/db/models/query.py", line 1074, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/home/ozgurisil/dev/pulsecheck_env/local/lib/python2.7/site-packages/django/db/models/query.py", line 52, in __iter__
    results = compiler.execute_sql()
  File "/home/ozgurisil/dev/pulsecheck_env/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 846, in execute_sql
    cursor = self.connection.cursor()
  File "/home/ozgurisil/dev/pulsecheck_env/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 233, in cursor
    cursor = self.make_cursor(self._cursor())
  File "/home/ozgurisil/dev/pulsecheck_env/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 206, in _cursor
    return self.create_cursor()
  File "/home/ozgurisil/dev/pulsecheck_env/local/lib/python2.7/site-packages/django/db/utils.py", line 95, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/ozgurisil/dev/pulsecheck_env/local/lib/python2.7/site-packages/django/db/backends/base/base.py", line 206, in _cursor
    return self.create_cursor()
  File "/home/ozgurisil/dev/pulsecheck_env/local/lib/python2.7/site-packages/django/db/backends/postgresql/base.py", line 210, in create_cursor
    cursor = self.connection.cursor()
InterfaceError: connection already closed

----------------------------------------------------------------------
Ran 1 test in 1.382s

PS: The error still occurs if async(..., sync=True) format is used.

[khamaileon]

+1

[elnygren]

Can confirm this on Django 1.9.7 / Python 3.5.1 / PostgreSQL.

./manage.py tests fail because of this with sync mode and without sync mode I can't get any async tasks to run at all... any guidelines available on doing automated testing ?

EDIT: some info

Q_CLUSTER = {
    'name': 'example',
    'workers': 2,
    'recycle': 500,
    'timeout': 30,
    'retry': 60,
    'compress': True,
    'save_limit': 0, 
    'queue_limit': 100,
    'cpu_affinity': 1,
    'label': 'Django Q',
    'orm': 'default'
}

CONN_MAX_AGE=0 (I've tried None here too)

[Koed00]

All the Travis CI tests are run with sqlite3. What I'll try to do this week is add postgres to it. That is probably a fairer representation of what most people with use anyway.
We added quite a lot of extra close statements in the code add some point to prevent connection sharing between the workers. The connection would then hop threads and this would cause a security error in Postgres. I guess something changed in Django 1.9 . Should not be too hard to track down once I have the test environment set up for it.

[Koed00]

Turns out this is something both django https://code.djangoproject.com/ticket/15802 and Celery struggle with too celery/django-celery#121
It might be outside my expertise, but I'll give it try.

[khamaileon]

I think I've come up with a solution. I'm now using TransactionTestCase class instead of TestCase. This keeps connection alive. Note that I'm using Django Q and not Celery.

[avalanchy]

My tests passes with TransactionTestCase (or in my case pytest.mark.django_db(transaction=True)) with celery_worker fixture. Looks like connection is not dropped anymore.

[Eagllus]

Seems this issue can be closed because the fix is known.
Also thanks for pointing this out! I also updated my tests to use TransactionTestCase.