Log worker_client event

[jrbourbeau]

Using worker_client is supported but using them well can require some care. This PR makes sure we log an event when creating a client via worker_client, which can be used later when debugging, etc.

[jrbourbeau]

Not sure how useful this extra info actually is

[github-actions]

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

    25 files  ± 0      25 suites  ±0   10h 22m 24s ⏱️ + 7m 15s
 4 104 tests + 2   3 985 ✅ + 1    113 💤 ±0   6 ❌ +1 
47 426 runs  +24  45 285 ✅ +17  2 130 💤 +1  11 ❌ +6 

For more details on these failures, see this check.

Results for commit 26a8afe. ± Comparison against base commit ad5f98c.

♻️ This comment has been updated with latest results.

[phofl]

The change itself looks good to me, but it's missing the case that is more critical to me:

@delayed
def bla(i):
    arr = da.random.random((500, 500, 100), chunks=(10, 10, 10))
    result = dask.compute(arr.sum())
    return result

if __name__ == "__main__":
    cluster = LocalCluster()
    client = cluster.get_client()
    f = bla(1)
    dask.compute(f)

the accidental worker client creation. I think this happens somewhere in _get_client on the worker

[jrbourbeau]

but it's missing the case that is more critical to me

Ah, yeah, we definitely want to capture that case too. I've updated things to capture both the implicit client creation case via .compute() and the explicit worker_client case. It's not easy to distinguish between the two cases though, so we're logging the same event in both cases.

[phofl]

Could we log an event where you have it now and then a different event in the worker_client context manager?

Subtracting both counters would then tell us if it's intentional or accidental?

[jrbourbeau]

Sure, we now have a worker-get-client event which is logged in both cases and a worker-client event that's only logged in the worker_client(...) case

[phofl]

thx