Bug Report: txthrottler watches health streams for every keyspace/shard

[timvaillancourt]

Overview of the Issue

The Transaction Throttler uses healthcheck streams to gather the replication lag state of other tablets in a shard.

Currently, vttablet with --enable-tx-throttler set is opening healthcheck streams to every keyspace/shard in the cell when only the tablets of the local keyspace/shard are necessary. This is inefficient and adds a lot of noise to the logs

This can be seen in the logs when health check updates are logged for unrelated keyspace/shards:

HealthCheckUpdate(Serving State): tablet: TABLET-ALIAS-HERE (REDACTED) serving true => false for WRONG-KEYSPACE-HERE/-80 (REPLICA) reason: vttablet: rpc error: code = Unavailable desc = closing transport due to: connection error: desc = "error reading from server: EOF", received prior goaway: code: NO_ERROR

(TL;DR: an unnecessary healthcheck update for the wrong keyspace/shard)

Reproduction Steps

1. Deploy v19+ with --enable-tx-throttler
2. Notice it is streaming healthchecks from unrelated shards

Binary Version

This issue is present on `main`/v20

Operating System and Environment details

N/A

Log Fragments

No response