diff --git a/modules/accessing-inference-endpoint-for-model-deployed-on-single-model-serving-platform.adoc b/modules/accessing-inference-endpoint-for-model-deployed-on-single-model-serving-platform.adoc index 777907e3..622560e7 100644 --- a/modules/accessing-inference-endpoint-for-model-deployed-on-single-model-serving-platform.adoc +++ b/modules/accessing-inference-endpoint-for-model-deployed-on-single-model-serving-platform.adoc @@ -6,6 +6,13 @@ [role='_abstract'] To make inference requests to your deployed model, you must know how to access the inference endpoint that is available. +ifdef::upstream[] +For a list of paths to use with the supported runtimes and example commands, see link:{odhdocshome}/serving-models/#inference-endpoints_serving-large-models[Inference endpoints]. +endif::[] +ifndef::upstream[] +For a list of paths to use with the supported runtimes and example commands, see link:{rhoaidocshome}{default-format-url}/serving_models/serving-large-models_serving-large-models#inference-endpoints_serving-large-models[Inference endpoints]. +endif::[] + .Prerequisites * You have logged in to {productname-long}. ifndef::upstream[] @@ -21,38 +28,9 @@ endif::[] . From the {productname-short} dashboard, click *Model Serving*. + The inference endpoint for the model is shown in the *Inference endpoint* field. -. Depending on what action you want to perform with the model (and if the model supports that action), copy the inference endpoint shown and then add a path to the end of the URL. - -ifdef::upstream[] -[NOTE] -==== -For a list of paths to use with the supported runtimes, see link:{odhdocshome}/serving-models/#inference-endpoints_serving-large-models[Inference endpoints]. -==== -endif::[] - -ifndef::upstream[] -[NOTE] -==== -For a list of paths to use with the supported runtimes, see link:{rhoaidocshome}{default-format-url}/serving_models/serving-large-models_serving-large-models#inference-endpoints[Inference endpoints]. -==== -endif::[] - +. Depending on what action you want to perform with the model (and if the model supports that action), copy the inference endpoint and then add a path to the end of the URL. . Use the endpoint to make API requests to your deployed model. -ifdef::upstream[] -[NOTE] -==== -For a list of sample commands, see link:{odhdocshome}/serving-models/#inference-endpoints_serving-large-models[Inference endpoints]. -==== -endif::[] - -ifndef::upstream[] -[NOTE] -==== -For a list of sample commands, see link:{rhoaidocshome}{default-format-url}/serving_models/serving-large-models_serving-large-models#inference-endpoints[Inference endpoints]. -==== -endif::[] - [role='_additional-resources'] .Additional resources * link:https://github.com/IBM/text-generation-inference[Text Generation Inference Server (TGIS)^] diff --git a/modules/ref-inference-endpoints.adoc b/modules/ref-inference-endpoints.adoc index 0c21883f..ad8c2d61 100644 --- a/modules/ref-inference-endpoints.adoc +++ b/modules/ref-inference-endpoints.adoc @@ -212,8 +212,8 @@ endif::[] * link:https://docs.openvino.ai/2023.3/ovms_docs_rest_api_kfs.html[OpenVINO KServe-compatible REST API documentation^] * link:https://platform.openai.com/docs/api-reference/introduction[OpenAI API documentation^] ifdef::upstream[] -* link:{odhdocshome}/serving-models/#ref-supported-runtimes_serving-large-models[Supported runtimes] +* link:{odhdocshome}/serving-models/#supported-runtimes_serving-large-models[Supported model-serving runtimes] endif::[] ifndef::upstream[] -* link:{rhoaidocshome}{default-format-url}/serving_models/serving-large-models_serving-large-models#ref-supported-runtimes[Supported runtimes] -endif::[] \ No newline at end of file +* link:{rhoaidocshome}{default-format-url}/serving_models/serving-large-models_serving-large-models#supported-model-serving-runtimes_serving-large-models[Supported model-serving runtimes]. +endif::[] diff --git a/modules/ref-supported-runtimes.adoc b/modules/ref-supported-runtimes.adoc index 152bc727..7663e866 100644 --- a/modules/ref-supported-runtimes.adoc +++ b/modules/ref-supported-runtimes.adoc @@ -60,7 +60,7 @@ endif::[] | OpenVINO Model Server | REST | None | Yes | Yes | Raw and serverless -| Text Generation Inference Server (TGIS) Standalone ServingRuntime for KServe (3) | gRPC | None | No | Yes | Raw and serverless +| Text Generation Inference Server (TGIS) Standalone ServingRuntime for KServe | gRPC | None | No | Yes | Raw and serverless | vLLM ServingRuntime for KServe | REST | None | No | Yes | Raw and serverless @@ -73,6 +73,6 @@ ifdef::upstream[] endif::[] ifndef::upstream[] -* link:{rhoaidocshome}{default-format-url}/serving_models/serving-large-models_serving-large-models#inference-endpoints[Inference endpoints] +* link:{rhoaidocshome}{default-format-url}/serving_models/serving-large-models_serving-large-models#inference-endpoints_serving-large-models[Inference endpoints] endif::[]