TSO Server Close() gets stuck sometimes

[binshi-bing]

Bug Report

What did you do?

During EKS upgrading in the prod env, after the operator restarted all API servers first then restarted all TSO servers, the TiFlash can't get TSO. Eventually, the operator deleted all API PODs to restore the TSO service.

What did you expect to see?

TiFlash should be served by TSO service normally after short time for service discovery.

What did you see instead?

We suspect that TSO Server Close() got stuck with the same root cause of issue #6304 in which it got stuck at s.serverLoopWg.Wait() in tso.Server.Close() every 3 ~ 5 times when we rerun Rerun TestMixedTSODeployment repeatedly.
Issue Number: #6304

What version of PD are you using (pd-server -V)?

tikv/pd master as of 8b16b71

[binshi-bing]

possibly fixed by #6529