Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure cleanupQuery gets called for procedures #19692

Conversation

findepi
Copy link
Member

@findepi findepi commented Nov 9, 2023

Ensure cleanupQuery is called for cases when ConnectorMetadata is constructed outside of typical query processing
(Connector.getMetadata).

@findepi findepi requested review from ebyhr and findinpath November 9, 2023 15:17
@cla-bot cla-bot bot added the cla-signed label Nov 9, 2023
@github-actions github-actions bot added tests:hive delta-lake Delta Lake connector hive Hive connector labels Nov 9, 2023
Copy link
Contributor

@findinpath findinpath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have an increasing number of tweaks in the procedures:

  • try (ThreadContextClassLoader ignored = new ThreadContextClassLoader(getClass().getClassLoader())) { ....}
  • try (UncheckedCloseable ignore = () -> metadata.cleanupQuery(session)) { .... }

I'm not sure how the engineers maintaining eventually the procedure code/ writing new procedures are supposed to know about them.
Ideally they would be transparent for the the procedure developer, but i don't see a better solution currently.

@findepi findepi force-pushed the findepi/ensure-cleanupquery-gets-called-for-procedures-5e9cf1 branch from 3b6397e to 1c8646d Compare November 10, 2023 09:09
@findepi findepi force-pushed the findepi/ensure-cleanupquery-gets-called-for-procedures-5e9cf1 branch 2 times, most recently from c0f586e to 8f1f4b1 Compare November 10, 2023 10:41
Ensure `cleanupQuery` is called for cases when `ConnectorMetadata` is
constructed outside of typical query processing
(`Connector.getMetadata`).
@findepi findepi force-pushed the findepi/ensure-cleanupquery-gets-called-for-procedures-5e9cf1 branch from 8f1f4b1 to e2c3ed3 Compare November 10, 2023 10:57
@github-actions github-actions bot added the iceberg Iceberg connector label Nov 10, 2023
@findepi findepi merged commit 4f8b437 into trinodb:master Nov 10, 2023
@findepi findepi deleted the findepi/ensure-cleanupquery-gets-called-for-procedures-5e9cf1 branch November 10, 2023 21:06
@github-actions github-actions bot added this to the 434 milestone Nov 10, 2023
@mosabua
Copy link
Member

mosabua commented Nov 17, 2023

No release notes entry needed @findepi @findinpath @ebyhr ?

@findinpath
Copy link
Contributor

No release notes entry needed @findepi @findinpath @ebyhr ?

Hive
Perform cleanup of resources after executing procedures

@findepi
Copy link
Member Author

findepi commented Nov 20, 2023

Hive
Perform cleanup of resources after executing procedures

@findinpath please try to think in end-user's terms.
Try to say why a user should care, and when. If we cannot make user excited about given change, we should omit from Release Notes.

@mosabua
Copy link
Member

mosabua commented Nov 20, 2023

@findepi @findinpath .. so maybe we say something like "Prevent memory leak" or "Increase stability" .... or it really is not user facing enough and we decide to leave out .. one way or another.. please let me know

@findinpath
Copy link
Contributor

If we cannot make user excited about given change, we should omit from Release Notes.

I'd say that at the moment it is not user facing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed delta-lake Delta Lake connector hive Hive connector iceberg Iceberg connector
Development

Successfully merging this pull request may close these issues.

4 participants