Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable external metastores #483

Merged
merged 8 commits into from
Jan 3, 2024
Merged

Enable external metastores #483

merged 8 commits into from
Jan 3, 2024

Conversation

gruuya
Copy link
Contributor

@gruuya gruuya commented Dec 29, 2023

External metastore support via a gRPC interface, Closes #485.

Also rename floc crate to clade.
@gruuya gruuya marked this pull request as ready for review December 29, 2023 15:14
@gruuya gruuya requested a review from mildbyte December 29, 2023 15:14
clade/proto/schema.proto Outdated Show resolved Hide resolved

service SchemaStoreService {
// List the available schemas
rpc List(catalog.CatalogReference) returns (ListSchemaResponse);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a best practice here is naming these types as per the method (https://protobuf.dev/programming-guides/style/#message-field-names), e.g. GetCatalog(GetCatalogRequest) returns (GetCatalogResponse)

clade/proto/catalog.proto Outdated Show resolved Hide resolved
src/catalog/external.rs Outdated Show resolved Hide resolved
src/catalog/external.rs Outdated Show resolved Hide resolved
src/catalog/mod.rs Show resolved Hide resolved
pub const DEFAULT_SCHEMA: &str = "public";
pub const STAGING_SCHEMA: &str = "staging";

#[derive(Debug, thiserror::Error)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actual error handling library, nice

let (context, clade) = start_clade_server().await;
tokio::task::spawn(clade);

tokio::time::sleep(tokio::time::Duration::from_secs(1)).await;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mmmmm is there no better way to wait until it's up and running? this can randomly fail if something is having a slow day

clade/Cargo.toml Outdated
Comment on lines 6 to 7
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

@@ -0,0 +1,13 @@
[package]
name = "clade"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might as well keep calling it clade, though we might need to get more people involved in naming it if we ever want to spin it out into a separate open source project. (alternative name: simply seafowl-metastore)

@mildbyte
Copy link
Contributor

Added the metadata required to expose a reflection service (useful for GUI gRPC clients): #484

@gruuya gruuya merged commit ddf9c89 into main Jan 3, 2024
1 check passed
@gruuya gruuya deleted the external-catalog branch January 3, 2024 10:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable external metastore implementations
2 participants