Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SHOW CREATE SCHEMA on Glue fails when the database contains parameters #24744

Open
ebyhr opened this issue Jan 20, 2025 · 6 comments · May be fixed by #25116
Open

SHOW CREATE SCHEMA on Glue fails when the database contains parameters #24744

ebyhr opened this issue Jan 20, 2025 · 6 comments · May be fixed by #25116
Labels
good first issue Good for newcomers iceberg Iceberg connector

Comments

@ebyhr
Copy link
Member

ebyhr commented Jan 20, 2025 •
edited
Loading 

aws glue create-database --database-input "{\"Name\":\"tempdb\", \"Parameters\":{\"string\": \"string\"}}"
trino> SHOW CREATE SCHEMA iceberg.tempdb;
Query 20250120_064618_00002_9svff failed: No PropertyMetadata for property: string
io.trino.spi.TrinoException: No PropertyMetadata for property: string
	at io.trino.metadata.PropertyUtil.lambda$toSqlProperties$0(PropertyUtil.java:197)
	at com.google.common.collect.SingletonImmutableBiMap.forEach(SingletonImmutableBiMap.java:71)
	at io.trino.metadata.PropertyUtil.toSqlProperties(PropertyUtil.java:190)
	at io.trino.sql.rewrite.ShowQueriesRewrite$Visitor.showCreateSchema(ShowQueriesRewrite.java:674)
	at io.trino.sql.rewrite.ShowQueriesRewrite$Visitor.visitShowCreate(ShowQueriesRewrite.java:520)
	at io.trino.sql.rewrite.ShowQueriesRewrite$Visitor.visitShowCreate(ShowQueriesRewrite.java:219)
	at io.trino.sql.tree.ShowCreate.accept(ShowCreate.java:60)
	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
	at io.trino.sql.rewrite.ShowQueriesRewrite.rewrite(ShowQueriesRewrite.java:216)
	at io.trino.sql.rewrite.StatementRewrite.rewrite(StatementRewrite.java:54)
	at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:93)
	at io.trino.sql.analyzer.Analyzer.analyze(Analyzer.java:87)
	at io.trino.execution.SqlQueryExecution.analyze(SqlQueryExecution.java:289)
	at io.trino.execution.SqlQueryExecution.<init>(SqlQueryExecution.java:222)
	at io.trino.execution.SqlQueryExecution$SqlQueryExecutionFactory.createQueryExecution(SqlQueryExecution.java:892)
	at io.trino.dispatcher.LocalDispatchQueryFactory.lambda$createDispatchQuery$0(LocalDispatchQueryFactory.java:153)
	at io.trino.$gen.Trino_testversion____20250120_064244_71.call(Unknown Source)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:75)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1575)

The cause is the following. We should check all catalogs' implementation (HMS, REST, JDBC, Snowflake, Nessie) in addition to Glue.

trino/plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/catalog/glue/TrinoGlueCatalog.java

Lines 311 to 313 in ea17c2e

if (database.getParameters() != null) {
metadata.putAll(database.getParameters());
}

@ebyhr ebyhr added the iceberg Iceberg connector label Jan 20, 2025
@ebyhr ebyhr added the good first issue Good for newcomers label Jan 31, 2025
@daliparthi
Copy link

Hello,

I don't think there is any issue with aws glue command. But I see you combined "SHOW" and "CREATE" in your sql command in trino.
You cannot combine SHOW and CREATE like below.

trino> SHOW CREATE SCHEMA iceberg.tempdb;

Thanks & Regards
Koteswara Rao Daliparthi

@ebyhr
Copy link
Member Author

ebyhr commented Feb 12, 2025 •
edited
Loading

@daliparthi It's a valid syntax in Trino. See https://trino.io/docs/current/sql/show-create-schema.html

@daliparthi
Copy link

oops sorry about that, i see what you are talking about

@pDhyey
Copy link

pDhyey commented Feb 15, 2025 •
edited
Loading

@ebyhr
I have analyzed and found the RCA for the same, looks like schema properties for iceberg just supports location,
so the parameters we pass while creating glue schema are not added in when we call schemaPropertyManager.getAllProperties(catalogHandle) in showCreateSchema.

So the solution to add a parameter property is the way to go or do we need discussion with wider audience ?

@ebyhr
Copy link
Member Author

ebyhr commented Feb 15, 2025

The solution is returning only location property in loadNamespaceMetadata method.

@pDhyey
Copy link

pDhyey commented Feb 23, 2025

#25116 has been raised to account for this

@ebyhr ebyhr linked a pull request Feb 25, 2025 that will close this issue
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first issue Good for newcomers iceberg Iceberg connector
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update metadata to use only location from parametes for glue pDhyey/trino
3 participants
@ebyhr @daliparthi @pDhyey