[STORY] Split caikit/TGIS ServingRuntime into three distinct images

[Xaenalt]

From req doc:

P0: OOTB support Caikit/TGIS

• 3 separate options - need to doc differences: caikit+tgis (current), caikit, tgis
• Can only be enabled if KServe is enabled

Caikit+TGIS:

• Cakit+TGIS combined image/SR caikit-tgis-serving#164

TGIS standalone:

• TGIS standalone SR and ISVC caikit-tgis-serving#134

Caikit:

• Caikit Standalone image/SR caikit-tgis-serving#163

[dtrifiro]

Progress:

• Caikit standalone image
• sample manifests for kserve InferenceService and ServingRuntime
• Proper TGIS/caikit containers networking

[dtrifiro]

Related to caikit-standalone image (without tgis backend):

caikit/caikit-nlp#210

[dtrifiro]

PR at opendatahub-io/caikit-tgis-serving#107

[heyselbi]

caikit/caikit-nlp#216

[bdattoma]

@Xaenalt could you share how to use the 3 options? even if doc is not completed, maybe you could share some instructions internally. Thanks!

[dtrifiro]

@bdattoma the idea is to have the options reported here opendatahub-io/caikit-tgis-serving#121. Feel free to reach out if you need more information. Hopefully I will have a PR ready in the next few hours

[bdattoma]

@bdattoma the idea is to have the options reported here opendatahub-io/caikit-tgis-serving#121. Feel free to reach out if you need more information. Hopefully I will have a PR ready in the next few hours

okay thank you. I'll wait for your PR :)

[heyselbi]

Caikit standalone image available at caikit/caikit-nlp#244

[dtrifiro]

@bdattoma it seems that the above issue I linked above does need a few requirements such as the above PR before being actionable, so you'll have to wait some more time 😅

[heyselbi]

Follow up issue on OOTB inclusion of all images in RHODS: #17

[bdattoma]

@Xaenalt for tracking purposes, could you edit this Issue saying explicitly that the only scenario covered was the Caikit+TGIS? by reading at it it looks like this issue covered also standalone Caikit and standalone TGIS

[Xaenalt]

This got deferred to 2.6. TGIS standalone is ready for use, but won't ship in 2.5 due to management priorities, caikit standalone dockerfile has merged upstream, we just need to build downstream and that's done

[Xaenalt]

Additionally, prompt tuning workflow we're pretty close on having a working test for too

[heyselbi]

@bdattoma I will convert this issue to story and have separate issues for each image.

[bdattoma]

This got deferred to 2.6. TGIS standalone is ready for use, but won't ship in 2.5 due to management priorities, caikit standalone dockerfile has merged upstream, we just need to build downstream and that's done

thanks for the update @Xaenalt! We will plan QE effort accordingly.

@bdattoma I will convert this issue to story and have separate issues for each image.

@heyselbi lgmt, thanks!!

[heyselbi]

moved to JIRA