Single Model Deployment (KServe)

[danielezonca]

Currently model serving capability in ODH is based on ModelMesh that is designed for high-scale, high-density and frequently-changing model use cases.
There are other scenarios where users might prefer single model deployment and benefit of other features like scale-to-zero, revision management, tracing etc.
KServe Model Serving runtime is able to solve similar scenarios.
Single model deployment is also the better candidate to serve LLM runtimes

This component uses Knative and Istio as dependency.

This is a tracker for bringing KServe into ODH as Trier 1 component.

Requirements

• Batching
• Revision Management
• Scale-to-zero

Individual Efforts

UX: (add link)
Lead: TBD
UI: (add link)
Lead: TBD
Dev: (add link)
Lead: TBD
QE: (add link)
Lead: TBD
Doc: (add link)
Lead: TBD