test_secede_rejoin_busy flaky

[jrbourbeau]

We observed distributed/tests/test_threadpoolexecutor.py::test_secede_rejoin_busy fail over in #5260 (see this CI build)

 ___________________________ test_secede_rejoin_busy ____________________________

    def test_secede_rejoin_busy():
        with ThreadPoolExecutor(2) as e:
            # Prime threads
            f1 = e.submit(sleep, 0.1)
            f2 = e.submit(sleep, 0.1)
            f1.result()
            f2.result()
    
            def f():
                assert threading.current_thread() in e._threads
                secede()
                sleep(0.2)
                assert threading.current_thread() not in e._threads
                rejoin()
                assert len(e._threads) == 2
                assert threading.current_thread() in e._threads
                return threading.current_thread()
    
            future = e.submit(f)
            for _ in range(6):
                e.submit(sleep, 0.4)
            start = time()
            special_thread = future.result()
            stop = time()
    
            assert 0.2 < stop - start < 0.6
    
            assert len(e._threads) == 2
            assert special_thread in e._threads
    
            def f():
                sleep(0.01)
                return threading.current_thread()
    
            futures = [e.submit(f) for _ in range(10)]
>           assert special_thread in {future.result() for future in futures}
E           assert <Thread(DaskThreadPoolExecutor-14749-575, started daemon 123146059800576)> in {<Thread(DaskThreadPoolExecutor-14749-576, started daemon 123146076590080)>}

[fjetter]

Didn't happen for at least 30days. Closing

[fjetter]

Encountered again on #6197