Prevent clients from sticking to old pools after config update #113
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
drdrsh
commented
Aug 9, 2022
| /// Postgres user for this client (This comes from the user in the connection string) | ||
| target_user_name: String, | ||
|
|
||
| /// Used to notify clients about an impending shutdown |
levkk
reviewed
Aug 9, 2022
src/client.rs
Outdated
|
|
||
| // The query router determines where the query is going to go, | ||
| // e.g. primary, replica, which shard. | ||
| let mut query_router = QueryRouter::new(pool.clone()); |
There was a problem hiding this comment.
The query router should be outside the client loop because it keeps the state of the client connection, re: current shard, current role, etc. For example, when a client issues SET SHARD TO, it expects that setting to persist until it's changed again by the client -- it must survive multiple transactions.
The QueryRouter just needs to fetch the pool settings dynamically like it used to before (you can pass it the target_pool_name and target_user_name.
|
It took me sometime to add a test to verify the pool refresh after reload. So, I open a connection, leave in transaction, reload configs where I swapped database names for shards and then run |
jmeagher
pushed a commit
to jmeagher/pgcat
that referenced
this pull request
Feb 17, 2023
* Prevent clients from sticking to old pools after config update (postgresml#113) * Re-acquire pool at the beginning of Protocol loop * Fix query router + add tests for recycling behavior * create a prometheus exporter on a standard http port (postgresml#107) * create a hyper server and add option to enable it in config * move prometheus stuff to its own file; update format * create metric type and help lookup table * finish the metric help type map * switch to a boolean and a standard port * dont emit unimplemented metrics * fail if curl returns a non 200 * resolve conflicts * move log out of config.show and into main * terminating new line * upgrade curl * include unimplemented stats * Validates pgcat is closed after shutdown python tests (postgresml#116) * Validates pgcat is closed after shutdown python tests * Fix pgrep logic * Moves sigterm step to after cleanup to decouple * Replace subprocess with os.system for running pgcat * fix docker compose port allocation for local dev (postgresml#117) change docker compose port to right prometheus port * Update CONTRIBUTING.md * Health check delay (postgresml#118) * initial commit of server check delay implementation * fmt * spelling * Update name to last_healthcheck and some comments * Moved server tested stat to after require_healthcheck check * Make health check delay configurable * Rename to last_activity * Fix typo * Add debug log for healthcheck * Add address to debug log * Speed up CI a bit (postgresml#119) * Sleep for 1s * use premade image * quicker * revert shutdown timeout * Fix debug log (postgresml#120) * Make prometheus port configurable (postgresml#121) * Make prometheus port configurable * Update circleci config * Statement timeout + replica imbalance fix (postgresml#122) * Statement timeout * send error message too * Correct error messages * Fix replica inbalance * disable stmt timeout by default * Redundant mark_bad * revert healthcheck delay * tests * set it to 0 * reload config again * pgcat toml Co-authored-by: Nicholas Dujay <3258756+dat2@users.noreply.github.com> Co-authored-by: zainkabani <77307340+zainkabani@users.noreply.github.com> Co-authored-by: Lev Kokotov <levkk@users.noreply.github.com> Co-authored-by: Pradeep Chhetri <30620077+chhetripradeep@users.noreply.github.com>
jmeagher
pushed a commit
to jmeagher/pgcat
that referenced
this pull request
Feb 17, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When configs are
RELOADed, we replace old pools with new pools. Clients that hold on to their old pool references will keep using the old pools until they disconnect. This is undesirable.In this PR, we give each client the name of the target pool and target username and have it re-acquire a pool reference at the beginning of the protocol loop (at the beginning of a new session or after finishing transaction). This will not change behavior for clients that are in the middle of a transaction or clients in session mode.