Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: modelmesh runtime is not accessible from exposed route #128

Open
vaibhavjainwiz opened this issue Dec 7, 2023 · 4 comments
Open

bug: modelmesh runtime is not accessible from exposed route #128

vaibhavjainwiz opened this issue Dec 7, 2023 · 4 comments
Assignees
@vaibhavjainwiz

Comments

@vaibhavjainwiz
Copy link
Member

Once we resolved the #127 issue by creating a NetworkPolicy. Deployed ModelMesh InferenceService exposed the route to make prediction call.
But I am facing below issue while making prediction call.

request:

curl --location 'https://example-onnx-mnist-kserve-demo.apps.rosa.vajain.9yeo.p3.openshiftapps.com/v2/models/example-onnx-mnist/infer' \
--header 'Content-Type: application/json' \
--data '{
  "model_name": "example-onnx-mnist",
  "inputs": [
    {
      "name": "Input3",
      "shape": [1, 1, 28, 28],
      "datatype": "FP32",
      "data": [
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.01176471,
        0.07058824, 0.07058824, 0.07058824, 0.49411765, 0.53333336, 0.6862745,
        0.10196079, 0.6509804, 1.0, 0.96862745, 0.49803922, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.11764706, 0.14117648,
        0.36862746, 0.6039216, 0.6666667, 0.99215686, 0.99215686, 0.99215686,
        0.99215686, 0.99215686, 0.88235295, 0.6745098, 0.99215686, 0.9490196,
        0.7647059, 0.2509804, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.19215687, 0.93333334, 0.99215686, 0.99215686, 0.99215686,
        0.99215686, 0.99215686, 0.99215686, 0.99215686, 0.99215686, 0.9843137,
        0.3647059, 0.32156864, 0.32156864, 0.21960784, 0.15294118, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.07058824,
        0.85882354, 0.99215686, 0.99215686, 0.99215686, 0.99215686, 0.99215686,
        0.7764706, 0.7137255, 0.96862745, 0.94509804, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.3137255, 0.6117647, 0.41960785, 0.99215686, 0.99215686, 0.8039216,
        0.04313726, 0.0, 0.16862746, 0.6039216, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.05490196, 0.00392157, 0.6039216, 0.99215686, 0.3529412, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.54509807, 0.99215686,
        0.74509805, 0.00784314, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.04313726, 0.74509805, 0.99215686, 0.27450982, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.13725491, 0.94509804, 0.88235295,
        0.627451, 0.42352942, 0.00392157, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.31764707, 0.9411765, 0.99215686, 0.99215686, 0.46666667,
        0.09803922, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.1764706,
        0.7294118, 0.99215686, 0.99215686, 0.5882353, 0.10588235, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0627451, 0.3647059, 0.9882353,
        0.99215686, 0.73333335, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.9764706, 0.99215686, 0.9764706, 0.2509804, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.18039216, 0.50980395, 0.7176471, 0.99215686,
        0.99215686, 0.8117647, 0.00784314, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.15294118,
        0.5803922, 0.8980392, 0.99215686, 0.99215686, 0.99215686, 0.98039216,
        0.7137255, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.09411765, 0.44705883, 0.8666667,
        0.99215686, 0.99215686, 0.99215686, 0.99215686, 0.7882353, 0.30588236,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.09019608, 0.25882354, 0.8352941, 0.99215686,
        0.99215686, 0.99215686, 0.99215686, 0.7764706, 0.31764707, 0.00784314,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.07058824, 0.67058825, 0.85882354, 0.99215686, 0.99215686,
        0.99215686, 0.99215686, 0.7647059, 0.3137255, 0.03529412, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.21568628, 0.6745098, 0.8862745, 0.99215686, 0.99215686, 0.99215686,
        0.99215686, 0.95686275, 0.52156866, 0.04313726, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.53333336, 0.99215686, 0.99215686, 0.99215686, 0.83137256, 0.5294118,
        0.5176471, 0.0627451, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
        0.0, 0.0, 0.0, 0.0, 0.0, 0.0
      ]
    }
  ]
}'

response :
503Service Unavailable

Screenshot 2023-12-07 at 6 34 17 PM
@vaibhavjainwiz vaibhavjainwiz self-assigned this Dec 7, 2023
@skonto
Copy link

skonto commented Dec 7, 2023

You might be hitting: https://maistra-2-0.maistra.io/docs/troubleshooting/troubleshooting_network_policy.

@vaibhavjainwiz
Copy link
Member Author

Below NetworkPolicy should be created to resolve this issue:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-mm-route
  namespace: kserve-demo
spec:
  podSelector:
    matchLabels:
      modelmesh-service: modelmesh-serving
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          network.openshift.io/policy-group: ingress

Please follow below thread for more details :
https://redhat-internal.slack.com/archives/C065ARTVA80/p1702293019814919?thread_ts=1701693652.733169&cid=C065ARTVA80

@vaibhavjainwiz vaibhavjainwiz mentioned this issue Dec 12, 2023
Closed
3 tasks
@vaibhavjainwiz vaibhavjainwiz linked a pull request Dec 12, 2023 that will close this issue
Allow ModelMesh and KServe to run in same namespace. #132
Closed
3 tasks
@heyselbi heyselbi moved this from New/Backlog to Under Review in ODH Model Serving Planning Dec 12, 2023
@spolti
Copy link
Member

spolti commented Mar 11, 2024

Is this issue still needed?
If so, can we maybe move it to jira?

@israel-hdez
Copy link
Contributor

Is this issue still needed? If so, can we maybe move it to jira?

@vaibhavjainwiz ^ ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vaibhavjainwiz vaibhavjainwiz

Labels
None yet
Projects
ODH Model Serving Planning
Status: Under Review
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Allow ModelMesh and KServe to run in same namespace. vaibhavjainwiz/odh-model-controller
4 participants
@spolti @skonto @vaibhavjainwiz @israel-hdez