Ai/initial

packages/ai-sdk/AI_SDK_INTEGRATION_PLAN.md

@@ -0,0 +1,265 @@
+# AI SDK Integration with Temporal Workflows
+
+## Overview
+
+This package provides seamless integration between Temporal workflows and the Vercel AI SDK, enabling AI model calls within Temporal workflows while maintaining deterministic execution and proper isolation.
+
+## Problem Statement
+
+Temporal workflows must be deterministic - they cannot make non-deterministic calls like HTTP requests to AI models directly. However, developers want to use AI models within their workflows for various use cases like content generation, decision making, and data processing.
+
+## Solution Architecture
+
+### Core Components
+
+1. **AI Model Provider Wrapper** - A custom model provider that intercepts AI SDK calls
+2. **Activity-Based Model Execution** - Routes model calls through Temporal activities
+3. **Workflow-Safe AI Interface** - Provides deterministic AI model access within workflows
+4. **Type-Safe Integration** - Maintains full TypeScript support for AI SDK models
+
+### Implementation Strategy
+
+#### 1. Custom Model Provider (`TemporalModelProvider`)
+
+```typescript
+interface TemporalModelProvider {
+  // Wraps any existing model provider (OpenAI, Anthropic, etc.)
+  // Intercepts model calls and routes them through activities
+  wrapProvider<T extends ModelProvider>(provider: T): T;
+}
+```
+
+**Key responsibilities:**
+- Intercept all model generation calls
+- Serialize model requests for activity execution
+- Handle streaming responses through activities
+- Maintain model provider interface compatibility
+
+#### 2. Activity-Based Model Execution
+
+```typescript
+// Activity that executes the actual AI model call
+async function executeAIModelActivity(
+  modelRequest: SerializedModelRequest
+): Promise<SerializedModelResponse> {
+  // Deserialize request
+  // Execute original model call outside workflow
+  // Return serialized response
+}
+```
+
+**Key features:**
+- Executes actual AI model calls in activities (non-deterministic environment)
+- Handles retries and timeouts appropriately
+- Supports both streaming and non-streaming responses
+- Manages API rate limits and failures
+
+#### 3. Workflow Integration
+
+```typescript
+// Usage in workflows
+import { createTemporalAI } from '@temporalio/ai-sdk';
+import { openai } from 'ai/openai';
+
+export async function myWorkflow() {
+  const ai = createTemporalAI(openai('gpt-4'));
+
+  const result = await generateText({
+    model: ai,
+    prompt: 'Generate a summary...'
+  });
+
+  return result.text;
+}
+```
+
+## Technical Implementation Plan
+
+### Phase 1: Core Infrastructure
+- [ ] Create base `TemporalModelProvider` class
+- [ ] Implement model request/response serialization
+- [ ] Create core AI execution activity
+- [ ] Add basic error handling and retries
+
+### Phase 2: AI SDK Integration  
+- [ ] Implement `generateText` support
+- [ ] Add `generateObject` functionality
+- [ ] Support streaming responses
+- [ ] Handle embeddings generation
+
+### Phase 3: Advanced Features
+- [ ] Multi-model provider support (OpenAI, Anthropic, etc.)
+- [ ] Custom activity configurations (timeouts, retries)
+- [ ] Response caching mechanisms
+- [ ] Metrics and observability
+
+### Phase 4: Developer Experience
+- [ ] Comprehensive TypeScript types
+- [ ] Documentation and examples
+- [ ] Testing utilities
+- [ ] Migration guides
+
+## API Design
+
+### Main Entry Points
+
+```typescript
+// Primary factory function
+export function createTemporalAI<T extends ModelProvider>(
+  provider: T,
+  options?: TemporalAIOptions
+): T;
+
+// Configuration options
+interface TemporalAIOptions {
+  activityOptions?: ActivityOptions;
+  caching?: CachingOptions;
+  retryPolicy?: RetryPolicy;
+}
+```
+
+### Activity Configuration
+
+```typescript
+interface AIActivityOptions extends ActivityOptions {
+  // AI-specific activity options
+  maxTokens?: number;
+  timeout?: Duration;
+  retryOnRateLimit?: boolean;
+}
+```
+
+## Usage Examples
+
+### Basic Text Generation
+
+```typescript
+import { generateText } from 'ai';
+import { createTemporalAI } from '@temporalio/ai-sdk';
+import { openai } from 'ai/openai';
+
+export async function contentGenerationWorkflow(topic: string) {
+  const ai = createTemporalAI(openai('gpt-4'));
+
+  const content = await generateText({
+    model: ai,
+    prompt: `Write a blog post about ${topic}`,
+    maxTokens: 1000
+  });
+
+  return content.text;
+}
+```
+
+### Structured Data Generation
+
+```typescript
+import { generateObject } from 'ai';
+import { z } from 'zod';
+
+const summarySchema = z.object({
+  title: z.string(),
+  keyPoints: z.array(z.string()),
+  sentiment: z.enum(['positive', 'negative', 'neutral'])
+});
+
+export async function documentAnalysisWorkflow(document: string) {
+  const ai = createTemporalAI(openai('gpt-4'));
+
+  const analysis = await generateObject({
+    model: ai,
+    schema: summarySchema,
+    prompt: `Analyze this document: ${document}`
+  });
+
+  return analysis.object;
+}
+```
+
+### Multi-Step AI Workflow
+
+```typescript
+export async function contentReviewWorkflow(content: string) {
+  const ai = createTemporalAI(openai('gpt-4'));
+
+  // Step 1: Generate initial review
+  const review = await generateText({
+    model: ai,
+    prompt: `Review this content for quality: ${content}`
+  });
+
+  // Step 2: Generate improvement suggestions
+  const suggestions = await generateObject({
+    model: ai,
+    schema: improvementSchema,
+    prompt: `Based on this review, suggest improvements: ${review.text}`
+  });
+
+  return {
+    review: review.text,
+    suggestions: suggestions.object
+  };
+}
+```
+
+## Implementation Challenges
+
+### 1. Response Streaming
+- Challenge: AI SDK supports streaming responses, but activities return single values
+- Solution: Implement custom streaming mechanism through activity heartbeats or signals
+
+### 2. Large Context Windows
+- Challenge: Large prompts/responses may exceed activity payload limits
+- Solution: Implement chunking and external storage for large payloads
+
+### 3. Rate Limiting
+- Challenge: AI providers have rate limits that can cause activity failures
+- Solution: Implement intelligent retry policies and backoff strategies
+
+### 4. Model Provider Compatibility
+- Challenge: Different AI providers have different interfaces and capabilities
+- Solution: Create abstraction layer that normalizes provider differences
+
+## Testing Strategy
+
+### Unit Tests
+- Model provider wrapping functionality
+- Request/response serialization
+- Error handling scenarios
+
+### Integration Tests
+- End-to-end workflow execution with real AI models
+- Activity timeout and retry behavior
+- Multiple model provider compatibility
+
+### Performance Tests
+- Large payload handling
+- Concurrent AI model calls
+- Memory usage under load
+
+## Security Considerations
+
+- API key management through Temporal's secret handling
+- Input sanitization for AI model prompts
+- Output validation and content filtering
+- Rate limiting and cost control mechanisms
+
+## Migration and Adoption
+
+### For Existing AI SDK Users
+1. Minimal code changes required - mostly import statement updates
+2. Existing AI SDK patterns remain the same
+3. Additional configuration options for Temporal-specific features
+
+### For Temporal Users
+1. Familiar activity-based pattern with AI-specific optimizations
+2. Standard Temporal retry and timeout mechanisms
+3. Integration with existing Temporal observability tools
+
+## Future Enhancements
+
+- Support for fine-tuned models
+- Integration with Temporal's data converter for custom serialization
+- Support for AI model chaining and pipelines
+- Integration with vector databases for RAG patterns
+- Cost tracking and budgeting features