Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DB Connection Pooling and Proxying docs #207

Merged
merged 2 commits into from
Nov 27, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions docs/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,3 +59,35 @@ For each of the item category and name, calculate the SHA-256 HMAC value of the
For encrypting the item value, first the value key is derived. Using the item HMAC key, calculate `HMAC-SHA-256(u_int32(len(category)) || category || u_int32(len(name)) || name)`, producing 32 bytes of output. Generate a random nonce value, and using the HMAC output value as a ChaCha20Poly1305 key, encrypt the value, prepending the random nonce.

Finally, the item tags are encrypted. All tag names are encrypted as searchable values in the same manner and the item category and name, using the tag name key and tag HMAC key. For encrypted tags, the value is encrypted in the same manner, using the tag value key and tag HMAC key.

## Database Connection Pooling for Improved Performance and Scalability

Managing database connections efficiently is crucial for optimizing performance, especially in projects like Aries Askar that rely on Postgres. Connection pooling tools like [pgBouncer](https://www.pgbouncer.org/) or AWS' [RDS Proxy](https://aws.amazon.com/rds/proxy/) can significantly enhance the performance of your Postgres database.

### Understanding Connection Pooling

Connection pooling involves overseeing a pool of established database connections that applications can reuse. Instead of initiating a new database connection every time an application interacts with the database, a connection pool maintains a set of pre-established connections, enabling faster access and reducing the overhead of creating new connections.

Though Aries Askar, via the [SQLx](https://github.com/launchbadge/sqlx) library, internally manages connection pooling, running multiple replicas of the same application results in each replica maintaining its individual connection pool. This can lead to a substantial number of connections to the database, potentially causing performance issues. Tools like pgBouncer or AWS RDS Proxy can effectively manage connections, reducing the database connections and enhancing performance.

### pgBouncer

[pgBouncer](https://www.pgbouncer.org/) is a lightweight connection pooler for PostgreSQL databases. It acts as an intermediary between the application and the database, proficiently managing connections and alleviating the workload on the database server. By pooling connections and managing them efficiently, pgBouncer improves performance by mitigating connection overhead and resource consumption.

To implement pgBouncer with Aries Askar, follow these steps:
1. Install and configure pgBouncer as per the official documentation and your specific requirements.
2. Update the application's database configuration to point to the pgBouncer endpoint.
3. Monitor and fine-tune pgBouncer settings to optimize performance based on your workload demands.

### AWS RDS Proxy

For users of Amazon Web Services (AWS), [RDS Proxy](https://aws.amazon.com/rds/proxy/) provides a managed database proxy service that enhances scalability, availability, and security of database connections. RDS Proxy efficiently manages database connections, transferring the burden of connection management from the database instance to the proxy service.

To leverage AWS RDS Proxy in the Aries Askar project:
1. Set up an RDS Proxy for your Postgres database using the AWS Management Console or AWS CLI. ([Using Amazon RDS Proxy](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy.html))
2. Update the application's database connection configuration to point to the RDS Proxy endpoint.
3. Configure and fine-tune RDS Proxy settings based on your application's requirements and performance metrics.

### Case Study: AWS RDS Proxy Performance Improvement

Refer to [hyperledger/aries-askar#195](https://github.com/hyperledger/aries-askar/issues/195) for in-depth insights and experiences on how AWS RDS Proxy contributed to enhancing database performance within the Aries Askar project. This issue contains valuable information, use cases, and considerations when implementing RDS Proxy.