Destination Iceberg V2: support aws instance profile auth #50876
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
octavia-squidington-iii
added
area/connectors
edgao
changed the title
edgao
force-pushed
the
edgao/iceberg_better_creds_handling
branch
from
37c4d02 to
0b42c56
Compare
|
For the eventual reviewer, I was able to test this and successfully authenticate via an IAM instance role without providing the IAM key and secret using the |
octavia-squidington-iii
added
the
area/documentation
frifriSF59
reviewed
Jan 9, 2025
airbyte-cdk/bulk/core/load/src/main/kotlin/io/airbyte/cdk/load/check/CheckOperation.kt
Show resolved
Hide resolved
frifriSF59
reviewed
Jan 9, 2025
|
|
||
| // If we don't have explicit S3 creds, fall back to the default creds provider chain. | ||
| // For example, this should allow us to use AWS instance profiles. | ||
| val awsAccessKeyId = config.awsAccessKeyConfiguration.accessKeyId |
There was a problem hiding this comment.
I think this is fine but now that I see this behavior, I think you are right, we should use that pattern for the Assume role path. I don't think we should make the change in this PR though
frifriSF59
approved these changes
Jan 9, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
this seems to work:
also, updates the CDK to log the throwable during a
checkfailure. afaict platform doesn't expose the trace message, and I don't know if we actually want to dump the full stacktrace into the check status message.@frifriSF59 lmk if you already have something like this as part of https://github.com/airbytehq/airbyte-internal-issues/issues/11082 though
(take from beta channel https://airbytehq-team.slack.com/archives/C087GAG7X25/p1735921337428349 )