Can't dynamically import httpx anymore

[MarieSaphira]

Since the update of httpcore to 1.17.2 yesterday, it's not possible to dynamically import httpx in a thread anymore. They changed some imports from lazy to conditional (see this pull request).

Example

import threading
from importlib import import_module

def thread_function():
    module = import_module("httpx")
    module.get("https://example.com")

if __name__ == "__main__":
    x = threading.Thread(target=thread_function)
    x.start()

throws a RuntimeError: can't register atexit after shutdown

Workaround

Explicitly install httpcore=0.17.1

Remark

Sorry if this is not the right place to start this discussion, maybe this should be discussed in httpcore instead?

[karpetrosyan]

It appears to be a problem with httpcore.

[karpetrosyan]

It works fine with trio, but not for anyio.
Also it can be fixed simply by adding x.join() at the end

[tomchristie]

I'm with @MarieSaphira on this one.

(Yes it's resolvable on the dev side, but we can have a more friendly behaviour here)

[tomchristie]

@MarieSaphira would you like to raise an issue on httpcore for this?

[MarieSaphira]

Well, @karosis88 has a point, we were just using threads inaccurately. Joining them fixed the issue for us. So not sure if httpcore needs to accommodate that.

[tomchristie]

Okay. Would anyways be resolved by #699.

[karpetrosyan]

The function threading._atexit_register, which is executed when anyio is imported, registers the function to be called after the main thread's final line has been executed. Since we are calling that method after the main thread has terminated, an error has been raised.

This code also fails with the same exception.

import threading
from importlib import import_module
from time import sleep

def thread_function():
    sleep(0.5)
    threading._register_atexit(lambda: ...)

if __name__ == "__main__":
    x = threading.Thread(target=thread_function)
    x.start()

or this one

import threading

def thread_function():
    import anyio

if __name__ == "__main__":
    x = threading.Thread(target=thread_function)
    x.start()

I believe that since this isn't an issue at all, fixing it in httpcore makes no sense.

[karpetrosyan]

You could use daemon thread or "thread.join()" to fix the issue.