Provide memory usage limits on session creation from startup params

[scsmithr]

Summary

When making a request to Cloud for connection authentication, Cloud will respond back with that database ID. Eventually Cloud should respond back with an optional set of resource limits as well. For example, a session connecting to a database on a shared node should be limited to .5G of memory. This will then be used during session creation to create a RuntimeEnv with this limit.

We'll want to specify disk/cpu usage in the future, but I'm not yet sure how we accomplish that at the session level yet.

See what we're currently doing for the database id:

https://github.com/glaredb/glaredb/blob/8d758de55c8112623e13c24de9341b29ea63e5ce/crates/pgsrv/src/handler.rs#L105-L116

Specifications

• Extend response body that we get back from cloud to include mem limit.
• Put that limit (if it exists) in the startup params that get forwarded to the underlying node.
• Parse limit on GlareDB side (if exists), and include it during session creation.

Rationale

Limit session resource usage on shared nodes.

Impact

Cloud needs the expand the endpoint to optionally include the limits when it sends back a response to pgsrv.

cc @greyscaled @f0ssel

Tasks

Related issues:
apache/datafusion#587
apache/datafusion#3941

[greyscaled]

Could we start with a hard-coded limit?

[scsmithr]

Yes, I was thinking we could do:

• No limit if connecting to "dedicated" node.
• 500M memory limit if connecting to "shared" node.

[greyscaled]

@scsmithr I like what's described here:

• alpha = hard-coded limit of 500M for shared, no such limit for dedicated

I think we can implement that in cloud and actually include that information to accomplish https://github.com/GlareDB/cloud/issues/578

I imagine seeing this possibly show up in the header, and for sure in the settings (now that we have them 😎

[greyscaled]

How do we actually limit resource utilization?

[greyscaled]

Garrett: Can we infer it from disk usage?
Sean: How do you limit tokio per session?

[greyscaled]

Mem and disk more important than cpu

[greyscaled]

I think we can start by stress testing and getting an reasonable estimate for number of people we can roll alpha out to without causing problems.

Again what matters most is memory and disk.

[greyscaled]

Sean: What if we attached a disk to each node

[greyscaled]

Garrett: Does it need to be shared?
Sean: It doesn't matter we don't care to save the data, it's to prevent spillage.

[greyscaled]

2023-04-17:

Adding the param is easy, but we need to be able to actually limit the memory too.

DF?

• We don't know how well it works (some issues in GH)
• Sometimes it doesn't actually limit, or record the used memory
• But we can still give it a go/figure it out?

[scsmithr]

Datafusion has a runtime config that allows for setting a memory pool:

https://github.com/GlareDB/arrow-datafusion/blob/e5452b52d571f350fef26cece6faecbb60ab8131/datafusion/execution/src/runtime_env.rs#L107-L110

We'll likely want to use the GreedyMemoryPool since the pool will be per-session:

https://github.com/GlareDB/arrow-datafusion/blob/e5452b52d571f350fef26cece6faecbb60ab8131/datafusion/execution/src/memory_pool/pool.rs#L48-L53

And we would construct the RuntimeConfig here, and build a RuntimeEnv from that:

https://github.com/glaredb/glaredb/blob/fb59215e3f94a9fc39f605c379ff840af1700fc7/crates/sqlexec/src/context.rs#L81

[f0ssel]

Really appreciate the guidance here 👍