Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does Cube.js support intelligent querying of a data source with multiple read replicas? #8531

Open
BeautyFades opened this issue Jul 30, 2024 · 1 comment
Assignees
Labels
question The issue is a question. Please use Stack Overflow for questions.

Comments

@BeautyFades
Copy link

Hello there! I'm not a network engineer nor do I have plenty of experience developing production-level high performance read-replicas, so bear with me...

Problem
I have an use-case for Cube.js in which it is used on top of an OLTP datastore (Postgres in AWS RDS) instead of the usual/expected OLAP warehouse.

This Postgres instance, due to performance and scalability constraints is broken into N read replicas, each having their own connection details. My question is: does Cube.js support out of the box source load balancing, or any sort of multiple connection logic that allows the same cube/query to be routed to potentially different data sources? Or can I set up a proxy or load balancer (RDS Proxy for instance) and have Cube connect to the proxy, and let it do the load balancing for me?

In the docs I never saw anything of this kind mentioned, nor did I see support for proxy connections. Has anyone ever faced this use-case before and could share details?

Thanks!

@BeautyFades BeautyFades added the question The issue is a question. Please use Stack Overflow for questions. label Jul 30, 2024
@igorlukanin igorlukanin self-assigned this Sep 2, 2024
@igorlukanin
Copy link
Member

Hi @BeautyFades 👋

does Cube.js support out of the box source load balancing [of data sources]

No, nothing of this kind of built into Cube.

or any sort of multiple connection logic that allows the same cube/query to be routed to potentially different data sources?

You can implement multitenancy and setup driver_factory in a way that different tenants have different credentials for the same data source. Then, you can manually randomize/round-robin through tenants, effectively spreading the load over Postgres connections.

Or can I set up a proxy or load balancer (RDS Proxy for instance) and have Cube connect to the proxy, and let it do the load balancing for me?

If Cube's Postgres driver can connect to it, then sure. I believe you can just test to see if this setup works for you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question The issue is a question. Please use Stack Overflow for questions.
Projects
None yet
Development

No branches or pull requests

2 participants