Fix descriptions of model-serving runtimes #311
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Tighten up intro paragraph with descriptions of model-serving runtimes and clarify inaccurate wording.
| @@ -3,11 +3,11 @@ | |||
| [id="adding-a-custom-model-serving-runtime-for-the-single-model-serving-platform_{context}"] | |||
| = Adding a custom model-serving runtime for the single-model serving platform | |||
| ifdef::upstream[] | |||
| A model-serving runtime adds support for a specified set of model frameworks (that is, formats). You have the option of using the link:{odhdocshome}/serving-models/#about-the-single-model-serving-platform_serving-large-models[pre-installed runtimes] included with {productname-short} or adding your own, custom runtimes. This is useful in instances where the pre-installed runtimes don't meet your needs. For example, you might find that the TGIS runtime does not support a particular model format that is supported by link:https://huggingface.co/docs/text-generation-inference/supported_models[Hugging Face Text Generation Inference (TGI)^]. In this case, you can create a custom runtime to add support for the model. | |||
| A model-serving runtime adds support for a specified set of model frameworks and the model formats supported by those frameworks. You can use the link:{odhdocshome}/serving-models/#about-the-single-model-serving-platform_serving-large-models[pre-installed runtimes] included with {productname-short}. You can also add your own custom runtimes if the default runtimes do not meet your needs. For example, if the TGIS runtime does not support a model format that is supported by link:https://huggingface.co/docs/text-generation-inference/supported_models[Hugging Face Text Generation Inference (TGI)^], you can create a custom runtime to add support for the model. | |||
There was a problem hiding this comment.
Suggest the following minor tweak to improve readability:
"You can use the link:{odhdocshome}/serving-models/#about-the-single-model-serving-platform_serving-large-models[pre-installed runtimes] that are included with {productname-short}"
| endif::[] | ||
|
|
||
| ifdef::self-managed,cloud-service[] | ||
| A model-serving runtime adds support for a specified set of model frameworks (that is, formats). You have the option of using the link:{rhoaidocshome}{default-format-url}/serving_models/serving-large-models_serving-large-models#about-the-single-model-serving-platform_serving-large-models[pre-installed runtimes] included with {productname-short} or adding your own, custom runtimes. This is useful in instances where the pre-installed runtimes don't meet your needs. For example, you might find that the TGIS runtime does not support a particular model format that is supported by link:https://huggingface.co/docs/text-generation-inference/supported_models[Hugging Face Text Generation Inference (TGI)^]. In this case, you can create a custom runtime to add support for the model. | ||
| A model-serving runtime adds support for a specified set of model frameworks and the model formats supported by those frameworks. You can use the link:{rhoaidocshome}{default-format-url}/serving_models/serving-large-models_serving-large-models#about-the-single-model-serving-platform_serving-large-models[pre-installed runtimes] included with {productname-short}. You can also add your own custom runtimes if the default runtimes do not meet your needs. For example, if the TGIS runtime does not support a model format that is supported by link:https://huggingface.co/docs/text-generation-inference/supported_models[Hugging Face Text Generation Inference (TGI)^], you can create a custom runtime to add support for the model. |
There was a problem hiding this comment.
jbyrne-redhat
approved these changes
Jun 10, 2024
There was a problem hiding this comment.
Updates look really good, @eturner24. The previous inaccuracy is fixed, and the text is tighter overall. You even found and fixed a stray contraction :)
One minor tweak to suggest, but PR approved nonetheless.
|
Looks good to me, @eturner24. Ready to merge. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Tighten up intro paragraph with descriptions of model-serving runtimes and clarify inaccurate wording.
How Has This Been Tested?
Ran a local build in Gatbsy and confirmed that the changed text appeared in the relevant sections
Merge criteria: