Add support for ClickHouse FORMAT on INSERT
#1628
Conversation
Adds supports for the `SETTINGS` and `FORMAT` keywords used for ClickHouse when inserting data with other syntax than SQL. This can happen e.g. when using the ClickHouse CLI tool to pipe input from files or similar.
| write!(f, "{format_clause}")?; | ||
| } | ||
|
|
||
| if self.source.is_none() && self.columns.is_empty() && self.format_clause.is_none() { |
There was a problem hiding this comment.
| if self.source.is_none() && self.columns.is_empty() && self.format_clause.is_none() { | |
| if self.source.is_none() && self.columns.is_empty() { |
thinking the format isn't necessarily tied to the columns?
| pub enum FormatClause { | ||
| Identifier(Ident), | ||
| Identifier { | ||
| ident: Ident, | ||
| expr: Option<Vec<Expr>>, | ||
| }, | ||
| Null, | ||
| } |
There was a problem hiding this comment.
it doesn't look correct to reuse this struct? The existing FormatClause is currently used on a query which has different variants/syntax than on insert?
maybe we can do something like
struct OutputFormatClause {
name: Ident
values: Vec<Expr>
}can we also add doc comments describing the struct/properties with a link to the doc?
https://clickhouse.com/docs/en/sql-reference/formats
| @@ -495,6 +495,20 @@ pub struct Insert { | |||
| pub priority: Option<MysqlInsertPriority>, | |||
| /// Only for mysql | |||
| pub insert_alias: Option<InsertAliases>, | |||
| /// Settings used in together with a specified `FORMAT`. | |||
There was a problem hiding this comment.
hmm double checking if this line is correct, is it required that a format clause is present if setting is present? Maybe we can avoid describing semantics in any case, we just mentiong that this is clickhouse settings with a link to the docs
| r#"INSERT INTO tbl FORMAT JSONEachRow {"id": 1, "value": "foo"}, {"id": 2, "value": "bar"}"#, | ||
| r#"INSERT INTO tbl FORMAT JSONEachRow ["first", "second", "third"]"#, | ||
| r#"INSERT INTO tbl FORMAT JSONEachRow [{"first": 1}]"#, | ||
| r#"INSERT INTO tbl FORMAT jsoneachrow {"id": 1}"#, | ||
| r#"INSERT INTO tbl (foo) FORMAT JSONAsObject {"foo": {"bar": {"x": "y"}, "baz": 1}}"#, |
There was a problem hiding this comment.
can we dedupe some of the test cases that seem to check case sensitivity? unless the impl is doing special case sensitive handling (which doesnt seem to be the case) I imagine no need to explicitly test for it
| @@ -931,7 +931,7 @@ pub const RESERVED_FOR_TABLE_ALIAS: &[Keyword] = &[ | |||
| Keyword::PREWHERE, | |||
| // for ClickHouse SELECT * FROM t SETTINGS ... | |||
| Keyword::SETTINGS, | |||
| // for ClickHouse SELECT * FROM t FORMAT... | |||
| // for ClickHouse SELECT * FROM t FORMAT... or INSERT INTO t FORMAT... | |||
There was a problem hiding this comment.
| // for ClickHouse SELECT * FROM t FORMAT... or INSERT INTO t FORMAT... | |
| // for ClickHouse SELECT * FROM t FORMAT... |
we could also remove the full description, the keyword isn't really tied to any specific syntax/dialect
| // Parses format clause used for [ClickHouse]. Formats are different when using `SELECT` and | ||
| // `INSERT` and also when using the CLI for pipes. It may or may not take an additional | ||
| // expression after the format so we try to parse the expression but allow failure. | ||
| // | ||
| // Since we know we never take an additional expression in `SELECT` context we never only try | ||
| // to parse if `can_have_expression` is true. | ||
| // | ||
| // <https://clickhouse.com/docs/en/interfaces/formats> |
There was a problem hiding this comment.
ah so per earlier comment I'm imagining if we use different representations for the format clauses the impl isn't affected?
Adds supports for the
SETTINGSandFORMATkeywords used for ClickHouse when inserting data with other syntax than SQL. This can happen e.g. when using the ClickHouse CLI tool to pipe input from files or similar.INSERT INTOStatement