Exceptions during module initialization are swallowed

[jchampio]

Please complete the following information:

• OS: FreeBSD, macOS
• Psycopg version: 2.8.6 (but still present in 2.9.6; see below)
• Python version: 3.11 (incompatible with 2.8.6; see below)
• PostgreSQL version: 16beta
• pip version: 23.1.2

Describe the bug

I accidentally installed psycopg2 version 2.8.6 on Python 3.11. This is not supported, but it took me a while to figure out what was wrong because of the following error message:

server/test_oauth.py:19: in <module>
    import psycopg2
../python_venv/lib/python3.11/site-packages/psycopg2/__init__.py:51: in <module>
    from psycopg2._psycopg import (                     # noqa
E   SystemError: initialization of _psycopg raised unreported exception

In this case, initialization of _psycopg raised unreported exception is generated when the module initialization function fails but returns a pointer anyway. It looks like Python expects the module to return NULL on failure instead. If I apply the following patch to 2.8.6:

diff --git a/psycopg/psycopgmodule.c b/psycopg/psycopgmodule.c
index 36fd1f9c..939875e1 100644
--- a/psycopg/psycopgmodule.c
+++ b/psycopg/psycopgmodule.c
@@ -1103,7 +1103,7 @@ INIT_MODULE(_psycopg)(void)
     if (!module) { goto exit; }
 
     if (0 > add_module_constants(module)) { goto exit; }
-    if (0 > add_module_types(module)) { goto exit; }
+    if (0 > add_module_types(module)) { return NULL; }
     if (0 > datetime_init()) { goto exit; }
     if (0 > mxdatetime_init(module)) { goto exit; }
     if (0 > encodings_init(module)) { goto exit; }

then I get a useful message:

server/test_oauth.py:19: in <module>
    import psycopg2
../../../build/python_venv/lib/python3.11/site-packages/psycopg2/__init__.py:51: in <module>
    from psycopg2._psycopg import (                     # noqa
E   SystemError: type psycopg2.extensions.ReplicationConnection has the Py_TPFLAGS_HAVE_GC flag but has no traverse function

which in this case pointed me to the PR for 3.11 support and solved my problem.

Can the INIT_MODULE function be tweaked to always return NULL on failure? That would help with debugging corner cases; it looks like I'm not the only one to run into this [1, 2, 3, 4].

[dvarrazzo]

Can you propose a patch?

[dvarrazzo]

Thank you for your contributions, @jchampio :) I've just released psycopg2 2.9.7 to include the two fixes you made.

[jchampio]

@dvarrazzo Thank you very much!