Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve visualization server docker image #2003

Conversation

ajchili
Copy link
Member

@ajchili ajchili commented Aug 30, 2019

Implemented better caching, switched base image, and reduced image size.


This change is Reviewable

@ajchili
Copy link
Member Author

ajchili commented Aug 30, 2019

/assign @IronPan @neuromage

@IronPan
Copy link
Member

IronPan commented Aug 30, 2019

/lgtm
/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: IronPan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 2eac092 into kubeflow:master Aug 30, 2019
magdalenakuhn17 pushed a commit to magdalenakuhn17/pipelines that referenced this pull request Oct 22, 2023
)

Until now KServe was using the global HTTP endpoint for S3 by default
unless the user explicitly provides it with the corresponding
annotation. If one wants to use IAM roles for service accounts instead
of Secrets with HMAC credentials, storage-initializer will fail in case:

* the bucket is configured with SSE
* if the bucket allows HTTPS requests only
* of an airgapped environment

Specifically, if the bucket is configured with SSE it fails with:

  botocore.exceptions.ClientError: An error occurred (InvalidArgument) when calling the GetObject operation: Requests specifying Server Side Encryption with AWS KMS managed keys must be made over a secure connection.

If the bucket policy allows HTTPS requests only it fails with:

  botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjects operation: Access Denied

In case of an airgapped environment, S3 is only available via the
regional VPC endpoint so requests to the global endpoint will hang.

Handle the above cases by letting boto3 decide the S3 endpoint based
on the environment.

In case of IRSA, AWS will set AWS_REGION and AWS_DEFAULT_REGION and as
such boto3 will use the regional endpoint. boto3 will use HTTPS by
default which is inline with the default value of s3-usehttps
annotation. Finally, if the bucket is in different region boto3 will be
automatically redirected there.

This change should not break existing deployments that don't use IRSA
but don't set s3-endpoint either; boto3 will default using the global
HTTPS endpoint, https://s3.amazonaws.com.

Handle also the case where endpoint_url is unset or empty, where boto3
fails with:

  ValueError: Invalid endpoint:

With this commit, one can use service accounts with the
`eks.amazonaws.com/role-arn` and `eks.amazonaws.com/sts-regional-endpoints`
annotations directly without the need of an attached secret.

Refs kubeflow#2003
Refs kubeflow#2113

Signed-off-by: Dimitris Aragiorgis <dimara@arrikto.com>

Signed-off-by: Dimitris Aragiorgis <dimara@arrikto.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants