Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add imagepullsecrets sample into sample test #2113

Merged
merged 1 commit into from
Sep 14, 2019

Conversation

numerology
Copy link

@numerology numerology commented Sep 13, 2019

Part of #1813


This change is Reviewable

@Ark-kun
Copy link
Contributor

Ark-kun commented Sep 13, 2019

/lgtm
/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Ark-kun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 216c80f into kubeflow:master Sep 14, 2019
@numerology numerology deleted the add-ips-sample-test branch September 16, 2019 16:25
@gaoning777
Copy link
Contributor

The secret 'secretA' does not exist in the test infra, does it?

@numerology
Copy link
Author

The secret 'secretA' does not exist in the test infra, does it?

No it does not. However it doesn't prevent us from running this sample. I guess k8s uses image pull secrets lazily. I put the function call here just to verify whether we can call this function successfully.

Ideally it will be great if we can really set up a protected registry and an associated secret in order to pull image from there, but that will be much harder and cumbersome.

@gaoning777
Copy link
Contributor

If the runtime feature is not verified, maybe we can simply test the compilation without running the pipeline. The extra step of running the pipeline only adds the value of verifying the pipeline will work with a non-existing image secret.

magdalenakuhn17 pushed a commit to magdalenakuhn17/pipelines that referenced this pull request Oct 22, 2023
)

Until now KServe was using the global HTTP endpoint for S3 by default
unless the user explicitly provides it with the corresponding
annotation. If one wants to use IAM roles for service accounts instead
of Secrets with HMAC credentials, storage-initializer will fail in case:

* the bucket is configured with SSE
* if the bucket allows HTTPS requests only
* of an airgapped environment

Specifically, if the bucket is configured with SSE it fails with:

  botocore.exceptions.ClientError: An error occurred (InvalidArgument) when calling the GetObject operation: Requests specifying Server Side Encryption with AWS KMS managed keys must be made over a secure connection.

If the bucket policy allows HTTPS requests only it fails with:

  botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied

In case of an airgapped environment, S3 is only available via the
regional VPC endpoint so requests to the global endpoint will hang.

Handle the above cases by letting boto3 decide the S3 endpoint based
on the environment.

In case of IRSA, AWS will set AWS_REGION and AWS_DEFAULT_REGION and as
such boto3 will use the regional endpoint. boto3 will use HTTPS by
default which is inline with the default value of s3-usehttps
annotation. Finally, if the bucket is in different region boto3 will be
automatically redirected there.

This change should not break existing deployments that don't use IRSA
but don't set s3-endpoint either; boto3 will default using the global
HTTPS endpoint, https://s3.amazonaws.com.

Handle also the case where endpoint_url is unset or empty, where boto3
fails with:

  ValueError: Invalid endpoint:

With this commit, one can use service accounts with the
`eks.amazonaws.com/role-arn` and `eks.amazonaws.com/sts-regional-endpoints`
annotations directly without the need of an attached secret.

Refs kubeflow#2003
Refs kubeflow#2113

Signed-off-by: Dimitris Aragiorgis <dimara@arrikto.com>

Signed-off-by: Dimitris Aragiorgis <dimara@arrikto.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants