Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connection Pool #14

Open
sbellware opened this issue Mar 25, 2024 · 0 comments
Open

Connection Pool #14

sbellware opened this issue Mar 25, 2024 · 0 comments

Comments

@sbellware
Copy link
Contributor

sbellware commented Mar 25, 2024

  • Raw PG connections are pooled
  • Session checks out a connection

Consumer

  • Prior to a consumer fetching a batch, a connection is checked out from the pool
  • Prior to every DB interaction, a new connection is checked out
  • After processing the batch [sic] check it back in

Session

  • Session has a reference to the connection pool
  • Move most of implementation to Connection class

Connection

  • Egress adapter of PG's raw connection
  • Session's implementation that pertains to connection moves to MessageStore::Connection

Connection Pool

  • When a connection is checked out, it's tested
  • If the connection is inactive, discard it, and as the pool for a NEW connection
  • Connection pool's list of connections is process-global (ie: Ruby process-wide singleton; an instance var in class context)
  • Connection pool does checkout based on Thread.current
  • If there is a connection is in thread local storage, use it (still test it, though)
  • If there is no connection in thread local storage, get it from the pool's list
  • Use shift and unshift on the list. Threat it as a queue.

Reaper

  • Connection pool may need a reaper for threads

Connection Pool Interface

  • get
  • get_new [sic]
  • checkout

Connection Pool Checkout

  • get connection from list
  • remove it from list
  • test it
  • if inactive, get a new one (and check it in)
  • return the connection
  • If connection is associated with a thread, it doesn't have to be tested (it's assumed to be active because threads are short-lived)
  • A connection associated with a thread is not ALSO in the pool's connection list
    So, connection doesn't have any "reconnect" ability. The pool does the "reconnect" by disposing of an inactive connection, and then constructing a new, active connection

Connection Pool Checkin

  • remove it from thread local storage to disassociate it from the thread

  • add it to the list

  • Every time consumer fetches a batch, the connection is tested, and possibly a new raw PG connection is created (if the previous one was inactive)

  • Need abstraction for raw PG connection that has the "re-connect" implementation

Reference

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant