Provider/Generator Refactor

[sixlive]

Provider Response Type Refactor

To Do

• Update Providers
• Remove Generators / Update Provider Contract
• Prism::fake updates
• Update docs
  • Custom Provider Docs
  • Testing docs with Prism::fake

Overview

This refactor aims to improve Prism's architecture by moving request handling and response generation logic from the Generator classes to the individual providers. This change will make Prism more flexible and allow providers to implement their own request/response flow in a way that best suits their specific needs.

Current Implementation

Currently, our Provider interface uses a generic ProviderResponse type:

interface Provider
{
    public function text(TextRequest $request): ProviderResponse;
    public function structured(StructuredRequest $request): ProviderResponse;
    public function embeddings(EmbeddingsRequest $request): EmbeddingsResponse;
}

The Generator classes are responsible for:

• Managing the request loop
• Invoking tools
• Appending tool results
• Iterating additional calls
• Building the final response

This creates a rigid structure where all providers must conform to the same request/response pattern, even when their APIs might work differently.

Proposed Changes

We'll update the Provider interface to use specific response types:

interface Provider
{
    public function text(TextRequest $request): TextResponse;
    public function structured(StructuredRequest $request): StructuredResponse;
    public function embeddings(EmbeddingsRequest $request): EmbeddingsResponse;
}

Benefits

1. Providers can implement their own request/response flow
2. More flexible architecture for different API patterns
3. Clearer separation of concerns

Implementation Strategy

To make this transition manageable, we'll:

1. Update the Provider interface to temporarily accept union types:

public function text(TextRequest $request): TextResponse|ProviderResponse;

2. Create new response types for each operation type:

• TextResponse
• StructuredResponse

3. Migrate providers one at a time to return the new response types

4. Update Generator classes to handle both response types during transition

5. Once all providers are migrated, remove union types and old response types

Help Needed

This is a major refactor that will touch many parts of the codebase. We need help with:

1. Updating individual providers
2. Writing tests for new response types
3. Updating documentation
4. Code review and testing

Getting Started

Pick a provider to migrate:

1. Create a fork and new branch refactor/provider-responses
2. Update the provider to return the new response type
3. Add tests
4. Submit a PR

Testing Requirements

• All existing tests should pass
• New tests should cover the new response types
• Integration tests should verify tool handling still works

Questions?

Feel free to ask questions in the comments. This is a significant change to Prism's architecture, so we want to make sure everyone understands the goals and implementation strategy.

[ChrisB-TL]

I'll start with Anthropic and Bedrock and pick whatever's left after 😆

[pushpak1300]

I'll pick Gemini, Deepseek, Mistral & XAI in same order.

[sixlive]

I just merged the Ollama refactor as an example of how I think providers could/should look.

[ChrisB-TL]

I'll pick Gemini, Deepseek, Mistral & XAI in same order.

@pushpak1300 have you started on Gemini? If not I have some time this morning and can take on.

[pushpak1300]

i have not yet. @ChrisB-TL

[ChrisB-TL]

@sixlive Bedrock package also refactored. I am assuming we want to still keep it out of the main repo given the AWS SDK dependency? Or would you rather I PR it in?

[sixlive]

@ChrisB-TL lets keep is separate. Before we release the Bedrock package I want to complete #197. I'll prioritize it tomorrow!

[sixlive]

Shipped!!

https://github.com/echolabsdev/prism/releases/tag/v0.40.0

Thank you SO MUCH @pushpak1300 and @ChrisB-TL