Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RHOAIENG-13638] - Do not allow isvc creation in protected isvc #311

Merged
merged 3 commits into from
Nov 28, 2024

Conversation

spolti
Copy link
Member

@spolti spolti commented Nov 25, 2024

chore: Fixes [RHOAIENG-13638] - Kserve model is not Ready after a kserve model is created and deleted from istio-system namespace

Description

How Has This Been Tested?

Unit tests and test on cluster.

To test it use the following DSC:

spec:
  components:
    codeflare:
      managementState: Removed
    kserve:
      devFlags:
        manifests:
          - contextDir: config
            sourcePath: ''
            uri: 'https://github.com/spolti/odh-model-controller/tarball/RHOAIENG-13638-t'
      managementState: Managed
      serving:
        ingressGateway:
          certificate:
            secretName: knative-serving-cert
            type: SelfSigned
        managementState: Managed
        name: knative-serving
    modelregistry:
      registriesNamespace: odh-model-registries
    trustyai:
      managementState: Removed
    ray:
      managementState: Removed
    kueue:
      managementState: Removed
    workbenches:
      managementState: Removed
    dashboard:
      managementState: Removed
    modelmeshserving:
      managementState: Removed
    datasciencepipelines:
      managementState: Removed
    trainingoperator: {}

Try to create a new ISVC using one of the protected namespaces:

  • knative-serving
  • Application Namespace
  • Mesh Namespace

ISVC:

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: example-onnx-mnist
  # annotations:
  #   serving.kserve.io/deploymentMode: ModelMesh
spec:
  predictor:
    model:
      modelFormat:
        name: onnx
      runtime: ovms-1.x
      storage:
        key: localMinIO
        path: onnx/mnist.onnx
        parameters: # Parameters to override the default values inside the common secret.
          bucket: modelmesh-example-models

You should receive a issue like this preventing the isvc to be created:

$ oc create -f mnist-isvc.yaml -n opendatahub
Error from server (Forbidden): error when creating "mnist-isvc.yaml": admission webhook "validating.isvc.odh-model-controller.opendatahub.io" denied the request: namespace opendatahub is protected, The InferenceService example-onnx-mnist will not be created

Merge criteria:

  • The commits are squashed in a cohesive manner and have meaningful messages.
  • Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
  • The developer has manually tested the changes and verified that the changes work

@openshift-ci openshift-ci bot requested review from hdefazio and mwaykole November 25, 2024 17:53
@spolti spolti force-pushed the RHOAIENG-13638 branch 2 times, most recently from 66489d3 to d75d5de Compare November 25, 2024 19:30
@spolti
Copy link
Member Author

spolti commented Nov 25, 2024

the failure is interesting, the 0.12.0 was already there:
https://github.com/opendatahub-io/odh-model-controller/blob/main/config/base/params.env#L1

Guess I would need to move the tags to fast, yes?

cc @Jooho @israel-hdez

#313

Copy link
Contributor

@israel-hdez israel-hdez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, change the target branch to incubating.
The main branch is now a stable/downstream branch.

@Jooho Jooho changed the base branch from main to incubating November 26, 2024 17:33
@spolti spolti changed the base branch from incubating to main November 26, 2024 19:01
@spolti
Copy link
Member Author

spolti commented Nov 26, 2024

IIMHO, we should wait until the end of the sprint or the beginning of the next.

@spolti spolti changed the base branch from main to incubating November 26, 2024 19:37
chore: Fixes [RHOAIENG-13638] - Kserve model is not Ready after a kserve model is created and deleted from istio-system namespace

Signed-off-by: Spolti <fspolti@redhat.com>
@mwaykole
Copy link
Member

hi @spolti i tried to create model in redhat-ods-applications-auth-provider and i was able to create the model

servingruntime.serving.kserve.io/ovms-runtime created
inferenceservice.serving.kserve.io/test-mnist-serverless created

for redhat-ods-applications seems to be working fine

oc apply -f isvc_ovms_oci_serverless.yaml -n redhat-ods-applications
servingruntime.serving.kserve.io/ovms-runtime unchanged
Error from server (Forbidden): error when creating "isvc_ovms_oci_serverless.yaml": admission webhook "validating.isvc.odh-model-controller.opendatahub.io" denied the request: Namespace redhat-ods-applications is protected, the InferenceService test-mnist-serverless will not be created

@spolti
Copy link
Member Author

spolti commented Nov 28, 2024

Hi @mwaykole, thanks for testing it, I spoke with @Jooho about it and we decided to protect only the 3 mentioned namespaces for now, but I am happy to add more if needed.
cc @israel-hdez

Copy link
Contributor

@israel-hdez israel-hdez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite good. Minor comments.

main.go Outdated Show resolved Hide resolved
controllers/utils/utils.go Show resolved Hide resolved
controllers/webhook/isvc_validator.go Outdated Show resolved Hide resolved
Signed-off-by: Spolti <fspolti@redhat.com>
@spolti
Copy link
Member Author

spolti commented Nov 28, 2024

@israel-hdez updated.

@mwaykole I've also updated the test branch with the new changes.

Co-authored-by: Edgar Hernández <ehernand@redhat.com>
Signed-off-by: Spolti <fspolti@redhat.com>
Copy link
Contributor

@israel-hdez israel-hdez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Copy link
Contributor

openshift-ci bot commented Nov 28, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: israel-hdez, spolti

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot openshift-merge-bot bot merged commit 19480d7 into opendatahub-io:incubating Nov 28, 2024
3 checks passed
@spolti spolti deleted the RHOAIENG-13638 branch November 28, 2024 21:17
spolti added a commit to spolti/odh-model-controller that referenced this pull request Nov 28, 2024
…datahub-io#311)

* [RHOAIENG-13638] - Do not allow isvc creation in protected namespace

chore: Fixes [RHOAIENG-13638] - Kserve model is not Ready after a kserve model is created and deleted from istio-system namespace

Signed-off-by: Spolti <fspolti@redhat.com>

* review suggestions

Signed-off-by: Spolti <fspolti@redhat.com>

* Update controllers/webhook/isvc_validator.go

Co-authored-by: Edgar Hernández <ehernand@redhat.com>
Signed-off-by: Spolti <fspolti@redhat.com>

---------

Signed-off-by: Spolti <fspolti@redhat.com>
Co-authored-by: Edgar Hernández <ehernand@redhat.com>
openshift-merge-bot bot pushed a commit that referenced this pull request Jan 16, 2025
* update global ca bundle logic and storage-config logic to follow up odh operator pr(1339) (#308)

Signed-off-by: jooho lee <jlee@redhat.com>

* disable dashboard and fix servingruntime display name

Signed-off-by: jooho lee <jlee@redhat.com>

* Use the main branch to build stable image tags, incubating for latest image tags (#316)

Signed-off-by: Hannah DeFazio <h2defazio@gmail.com>

* [RHOAIENG-13638] - Do not allow isvc creation in protected isvc (#311)

* [RHOAIENG-13638] - Do not allow isvc creation in protected namespace

chore: Fixes [RHOAIENG-13638] - Kserve model is not Ready after a kserve model is created and deleted from istio-system namespace

Signed-off-by: Spolti <fspolti@redhat.com>

* review suggestions

Signed-off-by: Spolti <fspolti@redhat.com>

* Update controllers/webhook/isvc_validator.go

Co-authored-by: Edgar Hernández <ehernand@redhat.com>
Signed-off-by: Spolti <fspolti@redhat.com>

---------

Signed-off-by: Spolti <fspolti@redhat.com>
Co-authored-by: Edgar Hernández <ehernand@redhat.com>

* update gitaction based on branch strategy change (#322)

Signed-off-by: jooho lee <jlee@redhat.com>

* feat: added performance metric grpahs config for nvidia nim (#320)

* feat: added performance metric grpahs config for nvidia nim

Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

* chore: modifyed the runtime id annotation

Co-authored-by: Edgar Hernández <ehernand@redhat.com>
Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

---------

Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>
Co-authored-by: Edgar Hernández <ehernand@redhat.com>

* Add NIM flag logic (#312)

Signed-off-by: mtrujillo <trujillo169@hotmail.com>

* Grab the old release tag based on creation date

Signed-off-by: Hannah DeFazio <h2defazio@gmail.com>

* Updated the checkout code command

Signed-off-by: Mariah Holder <marholde@marholde-thinkpadp16vgen1.rht.csb>

* Updated the checkout code command (#329)

Signed-off-by: Mariah Holder <marholde@marholde-thinkpadp16vgen1.rht.csb>
Co-authored-by: Mariah Holder <marholde@marholde-thinkpadp16vgen1.rht.csb>

* Add reconciliation for Kserve Raw (#274)

Signed-off-by: Vedant Mahabaleshwarkar <vmahabal@redhat.com>

* chore: added pagination support for nim catalog response (#332)

Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

* feat(mr): enable model registry inference reconcile (#326)

Signed-off-by: Alessio Pragliola <seth.pro@gmail.com>

* add upstream release metadata (#333)

Signed-off-by: heyselbi <selbi@redhat.com>

* Migration to kubebuilder v4 (#324)

* Migration to kubebuilder v4

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Restore MR E2Es

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Restore top-level files

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Cleaning

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Fixing Makefile and Containerfile

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Linter fixes

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Initial rework of manifests

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Fix manifests

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Fix lint issues

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Deactivate E2Es

Because setup is not automated, yet.

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Feedback: Filippe

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Feedback: Filippe

Test differences after `go mod tidy`

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Apply suggestions from code review: Filippe

Co-authored-by: Filippe Spolti <filippespolti@gmail.com>
Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Feedback: Filippe

* Pin go-toolset base image in Containerfile.
* Add `gosec` linter

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Update config/prometheus/monitor.yaml

Co-authored-by: Filippe Spolti <filippespolti@gmail.com>
Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Feedback: Filippe

* Small change to comments in Makefile, to make the text clearer.
* Remove (again) `gosec` linter

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Fix panic on controller startup

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

---------

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>
Co-authored-by: Filippe Spolti <filippespolti@gmail.com>

* chore: use naming convention for resources created by nim (#340)

* chore: use naming convention for resources created by nim

Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

* test: added assertions for dyamic nim resources name

Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

---------

Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

* chore: set nim runtime api call page size to 1000 (#344)

Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

* Nim enablement change default to managed and add clean up job (#342)

* initial commit for clean up of nim and managed set as default

Signed-off-by: mtrujillo <trujillo169@hotmail.com>

* remove space

Signed-off-by: mtrujillo <trujillo169@hotmail.com>

* fix code length for linting

Signed-off-by: mtrujillo <trujillo169@hotmail.com>

* fixed comments / adjusted import

Signed-off-by: mtrujillo <trujillo169@hotmail.com>

---------

Signed-off-by: mtrujillo <trujillo169@hotmail.com>

* chore: added new graph object for nim runtimes (#334)

* chore: added new graph object for nim runtimes

Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

* chore: added REQUEST_OUTCOMES nim graph

Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

* chore: added fixed typo in nim query object

Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

* chore: fixed typo in nim query object

Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

* chore: added initial query for nim gpu cache usage

Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

* chore: rewrite queries for nim new graphs

Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

---------

Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>

* Update ovms to current build (#343)

Signed-off-by: Steve Grubb <ausearch.1@gmail.com>
Co-authored-by: Steve Grubb <ausearch.1@gmail.com>

* Automatically inject expected ODH annotations to InferenceGraph and InferenceServices (#339)

* Implementation of ODH defaulters for InferenceGraph and InferenceService

On creation of InferenceGraph or InferenceService resources, the following default annotations will be added:
* `serving.knative.openshift.io/enablePassthrough: true`
* `sidecar.istio.io/inject: true`
* `sidecar.istio.io/rewriteAppHTTPProbers: true`

The annotations are added only for Serverless mode, and only if they are missing.

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Feedback: Filippe

Extract "ENABLE_WEBHOOKS" string to constant

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

---------

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Authorization for InferenceGraph (Serverless) (#345)

* Authorization for InferenceGraph (Serverless)

This adds a new controller for KServe InferenceGraph resources. This new controller will have the responsibility of creating Authorino AuthConfig resources (similarly to InferenceServices case), when authorization is available in ODH platform.

InferenceGraphs can now be annotated with `security.opendatahub.io/enable-auth: "true"` to secure InferenceGraphs and only serve requests that are authorized.

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Feedback: Filippe - Event when auth is not available

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

---------

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* [RHOAIENG-10293] add metrics resources for rawdeployment (#347)

* [RHOAIENG-10293] add metrics resources for rawdeployment

Signed-off-by: Vedant Mahabaleshwarkar <vmahabal@redhat.com>

* [RHOAIENG-10293] address feedback

Signed-off-by: Vedant Mahabaleshwarkar <vmahabal@redhat.com>

---------

Signed-off-by: Vedant Mahabaleshwarkar <vmahabal@redhat.com>

* [RHOAIENG-16851] rawdeployment route bug fixes (#341)

Signed-off-by: Vedant Mahabaleshwarkar <vmahabal@redhat.com>

* fix null pointer error (RHOAIENG-18228) (#349)

Signed-off-by: jooho lee <jlee@redhat.com>

* remove old file

Signed-off-by: jooho lee <jlee@redhat.com>

update go.mod

Signed-off-by: jooho lee <jlee@redhat.com>

---------

Signed-off-by: jooho lee <jlee@redhat.com>
Signed-off-by: Hannah DeFazio <h2defazio@gmail.com>
Signed-off-by: Spolti <fspolti@redhat.com>
Signed-off-by: Tomer Figenblat <tfigenbl@redhat.com>
Signed-off-by: mtrujillo <trujillo169@hotmail.com>
Signed-off-by: Mariah Holder <marholde@marholde-thinkpadp16vgen1.rht.csb>
Signed-off-by: Vedant Mahabaleshwarkar <vmahabal@redhat.com>
Signed-off-by: Alessio Pragliola <seth.pro@gmail.com>
Signed-off-by: heyselbi <selbi@redhat.com>
Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>
Signed-off-by: Steve Grubb <ausearch.1@gmail.com>
Co-authored-by: Hannah DeFazio <h2defazio@gmail.com>
Co-authored-by: Filippe Spolti <filippespolti@gmail.com>
Co-authored-by: Edgar Hernández <ehernand@redhat.com>
Co-authored-by: Tomer Figenblat <tomer.figenblat@gmail.com>
Co-authored-by: Marcus Trujillo <42344046+trujillm@users.noreply.github.com>
Co-authored-by: Mariah Holder <marholde@marholde-thinkpadp16vgen1.rht.csb>
Co-authored-by: Mariah Holder <94134625+mholder6@users.noreply.github.com>
Co-authored-by: Vedant Mahabaleshwarkar <vmahabal@redhat.com>
Co-authored-by: Tomer Figenblat <tfigenbl@redhat.com>
Co-authored-by: Alessio Pragliola <83355398+Al-Pragliola@users.noreply.github.com>
Co-authored-by: Selbi Nuryyeva <selbi@redhat.com>
Co-authored-by: Steven Grubb <sgrubb@redhat.com>
Co-authored-by: Steve Grubb <ausearch.1@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants