test_async_timeout makes multiprocessing tests with threads slow

[serhiy-storchaka]

I noticed an interesting effect. When run all tests in test.test_multiprocessing_spawn.test_threads, progress lingers (on 30 almost seconds) on test_terminate. But when run only test_terminate, it completes quickly. Further research revealed that it is only slow after test_async_timeout. If run test_async_timeout only, it completes quickly, but then Python freezes on 30 seconds (in tearDownClass).

It is because test_async_timeout runs a long task in a thread, and then either test_terminate or tearDownClass need to wait until it is finished. This slow task can also affect all other tests, because it monopolizes one worker.

The solution is to use a new pool for test_async_timeout and terminate it immediately after testing. It does not help for threads, but in that case we can unblock the slow task using Event.

There is other issue with test_terminate: it terminates the pool that is global for the class, so no other tests that uses the pool can be run after it. It only worked because test_terminate was at the end of sorted methods list. The solution is to use new pool in test_terminate.

It saves 1.5 minutes and makes tests more robust.

Linked PRs

• gh-111284: Make multiprocessing tests with threads faster and more reliable #111285
• [3.12] gh-111284: Make multiprocessing tests with threads faster and more reliable (GH-111285) #111510
• [3.11] gh-111284: Make multiprocessing tests with threads faster and more reliable (GH-111285) #111511