Feature design: Support detections on Chat API

[gkumbhat]

Use-case

End-to-end guardrails experience exposed on openAI chat API. This means, a user can use similar API as openAI chat and provide a list of detectors they want to run. Behind the scene, orchestrator will automatically run the detector at appropriate time in the lifecycle of the request (based on type of detector).

Description

To enable running detections on chat completion API, we need to design how the request and response look like for this API.

Some constrains and requirements we have talked about for designing this includes:

1. API should support full openai API, thus providing chat completion:
   1. The parts of request that are in openai API, for example, parameters and messages should remain as is
   2. The parts of response from openai API, for example choices, usage etc, should remain as is.
2. We want to provider ability to specify different kinds of detectors together, example, HAP, PII.
3. Users should be able to provide list of detectors they want to execute on input and output separately.
4. We understand user would likely need to have a bit of background to use some detectors of different types, where the answer is not quite intuitive at first, like use of generated type detectors along with HAP / PII, where response objects look different.
5. Since some of these detectors may support optional parameters, we need each of the detector that the user is requesting to be an object, where one can pass parameters.

Acceptance Criteria

• Proposed API merged in orchestrator docs: https://github.com/foundation-model-stack/fms-guardrails-orchestrator/tree/main/docs/api

[declark1]

For reference, OpenAI Chat/Completions API implementation for TGIS router on the following branch (not merged): https://github.com/IBM/text-generation-router/tree/openai-api
https://github.com/IBM/text-generation-router/tree/openai-api/fmaas-router/src/openai