Error after update to 4.4.1 (debug_toolbar.E001) The Django Debug Toolbar can't be used with tests

[MarcoGlauser]

After upgrading to 4.4.1 we got an error when trying to run our tests, despite DEBUG being False:

(debug_toolbar.E001) The Django Debug Toolbar can't be used with tests
	HINT: Django changes the DEBUG setting to False when running tests. By default the Django Debug Toolbar is installed because DEBUG is set to True. For most cases, you need to avoid installing the toolbar when running tests. If you feel this check is in error, you can set `DEBUG_TOOLBAR_CONFIG['IS_RUNNING_TESTS'] = False` to bypass this check.

It took us a while to figure out what went wrong. It turns out the default installation method, recommended in the docs, doesn't work with the the changes from 4.4.0 (even if DEBUG is False, which should deactivate Debug Toolbar)

After some back and forth we figured out that the example directory configuration was the only way to run our tests:

ENABLE_DEBUG_TOOLBAR = DEBUG and "test" not in sys.argv
if ENABLE_DEBUG_TOOLBAR:
    INSTALLED_APPS += [
        "debug_toolbar",
    ]
    MIDDLEWARE += [
        "debug_toolbar.middleware.DebugToolbarMiddleware",
    ]

The default installation docs are haven't changed for the 4.4.0 release and the example app is different than the recommended installation in the docs. Is this a missed docs update or an unintended breaking change?

[matthiask]

Thank you. Yes, we should have also changed the installation instructions, sorry for that. You discovered the best way to do it.

[tim-schilling]

It took us a while to figure out what went wrong. It turns out the default installation method, recommended in the docs, doesn't work with the the changes from 4.4.0 (even if DEBUG is False, which should deactivate Debug Toolbar)

To be clear, having DEBUG = False doesn't prevent the toolbar from some of the panels from setting up instrumentation. The only way to avoid any instrumentation is to not add the toolbar to INSTALLED_APPS.

[tim-schilling]

@MarcoGlauser do you remember what you originally had for the DEBUG_TOOLBAR_CONFIG setting? Were you customizing SHOW_TOOLBAR_CALLBACK?

[MarcoGlauser]

Thank you all for responding so quickly to this :)
We did not set any config settings, just the standard install as it's in the docs.

While trying to figure out what changed yesterday, I created a reproduction:
https://github.com/MarcoGlauser/debug-toolbar-repro/blob/main/mysite/mysite/settings.py

[tim-schilling]

Ah, thank you!