swift-llm

Simple Swift library for interacting with Large Language Models (LLMs), featuring support for streaming responses and tool integration.

swift-llm provides a modern, async/await-based interface to communicate with LLM APIs, making it easy to integrate advanced AI capabilities into your Swift applications.

Features

• Core LLM Interaction: Send requests and receive responses from LLMs using ChatMessage arrays for rich conversational context.
• Streaming Support: Handle real-time streaming of LLM responses for a more interactive user experience (AsyncThrowingStream).
• Multi-Standard SSE Parsing: Supports Server-Sent Events (SSE) streams conforming to both OpenAI (openAi) and Vercel AI SDK (vercel) standards via the SSETokenizer.
• Tool Integration: Empower your LLM to use predefined tools to perform actions in parallel and gather information, enabling more complex and capable assistants (TooledLLMClient).
• Customizable Requests: Modify outgoing URLRequest objects using a requestTransformer closure, allowing for custom headers, body modifications, or different endpoints for stream/non-stream calls.
• Typed Models: Clear, Codable Swift structures for requests, responses, and tool definitions.
• Modern Swift: Built with Swift 6.1+ and leverages modern concurrency features.
• Easy Integration: Designed as a Swift Package Manager library.

Requirements

• Swift 6.0 or later
• iOS 13.0 or later

Installation

Add swift-llm as a dependency to your Package.swift file:

import PackageDescription

let package = Package(
    name: "YourProjectName",
    platforms: [.iOS(.v13)],
    dependencies: [
        .package(url: "https://github.com/getgrinta/swift-llm.git", from: "0.1.4")
    ],
    targets: [
        .target(
            name: "YourProjectTarget",
            dependencies: ["swift-llm"]
        )
    ]
)

Usage

1. Basic Chat

import SwiftLLM

let endpoint = "your_llm_api_endpoint"
let modelName = "your_model_name"
let bearerToken = "your_api_key"

// Construct messages - an array of ChatMessage objects
// Assuming ChatMessage(role: .user, content: "...")
let messages = [ChatMessage(role: .user, content: "What is the capital of France?")]

// Initialize the LLMClient with your endpoint, model, and API key
// Default standard is .openAi. For Vercel, specify: SSETokenizer.Standard.vercel
let llmClient = LLMClient(endpoint: endpoint, model: modelName, apiKey: bearerToken)

Task {
    do {
        print("Sending request...")
        // Send messages and optionally set temperature (e.g., 0.7 for some creativity)
        let response = try await llmClient.send(messages: messages, temperature: 0.7)
        print("LLM Response: \(response.message)")
    } catch {
        print("Error during non-streaming chat: \(error.localizedDescription)")
    }
}

2. Streaming

import SwiftLLM

// Initialize the LLMClient with your endpoint, model, and API key
let llmClient = LLMClient(endpoint: endpoint, model: modelName, apiKey: bearerToken)
let endpoint = "your_llm_api_endpoint"
let modelName = "your_model_name"
let bearerToken = "your_api_key"

let messages = [ChatMessage(role: .user, content: "Tell me a short story, stream it part by part.")]

Task {
    do {
        // Stream messages and optionally set temperature
        let stream = try await llmClient.stream(messages: messages, temperature: 0.7)

        print("Streaming response:")
        for try await chunk in stream {
            print(chunk.message, terminator: "") // Append chunks as they arrive
        }
        print() // Newline after stream finishes
    } catch {
        print("Error during streaming chat: \(error.localizedDescription)")
    }
}

3. Using TooledLLMClient

First, define your tools:

import SwiftLLM

// Example Tool: Get Current Weather
let weatherTool = LLMTool(
    name: "getCurrentWeather",
    description: "Gets the current weather for a given location. Arguments should be a plain string with the location name.",
    execute: { argumentsAsLocationString in
        // In a real scenario, you might parse 'arguments' if it's structured (e.g., XML/JSON)
        // For this example, assume 'argumentsAsLocationString' is the location directly.
        print("Tool 'getCurrentWeather' called with arguments: \(argumentsAsLocationString)")
        
        // Simulate API call or logic
        if argumentsAsLocationString.lowercased().contains("paris") {
            return "The weather in Paris is sunny, 25°C."
        } else if argumentsAsLocationString.lowercased().contains("london") {
            return "It's currently cloudy with a chance of rain in London, 18°C."
        } else {
            return "Sorry, I don't know the weather for \(argumentsAsLocationString)."
        }
    }
)

let tools = [weatherTool]

Then, use TooledLLMClient:

import SwiftLLM

// Initialize the LLMClient first (used by TooledLLMClient)
// The LLMClient itself now uses ChatMessage and supports temperature, 
// though TooledLLMClient might manage this internally for its specific flow.
let llmClient = LLMClient(endpoint: endpoint, model: modelName, apiKey: bearerToken)
let tooledClient = TooledLLMClient(llmClient: llmClient) // Pass the LLMClient instance

let endpoint = "your_llm_api_endpoint_supporting_tools" // Ensure this endpoint supports tool use
let modelName = "your_tool_capable_model_name"
let bearerToken = "your_api_key"
let userInput = "What's the weather like in Paris today?"

Task {
    do {
        print("User Input: \(userInput)")
        let stream = try await tooledClient.processWithTools(
            userInput: userInput,
            tools: tools
        )

        print("\nFinal LLM Response (after potential tool use):")
        var fullResponse = ""
        for try await chunk in stream {
            print(chunk.message, terminator: "")
            fullResponse += chunk.message
        }
       
        print("\n--- Full Assembled Response ---")
        print(fullResponse)
    } catch let error as TooledLLMClientError {
        print("TooledLLMClientError: \(error)")
    } catch {
        print("An unexpected error occurred: \(error.localizedDescription)")
    }
}

Core Components

LLMClient

The primary client for all interactions with an LLM, supporting both non-streaming (single request/response) and streaming (continuous updates) communication. It handles the direct network requests to the LLM API. Interactions are based on ChatMessage arrays, allowing for conversational history to be passed to the LLM. It also supports a temperature parameter to control response randomness.

ChatMessage Structure (Conceptual): Your ChatMessage objects would typically include a role (e.g., .user, .assistant, .system) and content (the text of the message).

Initializer: public init(standard: SSETokenizer.Standard = .openAi, endpoint: String, model: String, apiKey: String, sessionConfiguration: URLSessionConfiguration = .default, requestTransformer: (@Sendable (URLRequest, _ isStream: Bool) -> URLRequest)? = nil)

• standard: (Optional) The SSE parsing standard to use. Defaults to .openAi. Can be set to .vercel for Vercel AI SDK compatibility.
• endpoint: The base URL for the LLM API.
• model: The identifier for the LLM model to be used.
• apiKey: Your API key for authentication.
• sessionConfiguration: (Optional) A URLSessionConfiguration for the underlying network session. Defaults to .default.
• requestTransformer: (Optional) A closure that allows you to modify the URLRequest before it's sent.

Example of requestTransformer:

let transformer: @Sendable (URLRequest, Bool) -> URLRequest = { request, isStream in
    var mutableRequest = request
    // Add a custom header
    mutableRequest.setValue("my-custom-value", forHTTPHeaderField: "X-Custom-Header")
    
    // Potentially change endpoint based on stream type
    if isStream {
        // mutableRequest.url = URL(string: "your_streaming_specific_endpoint")
    } else {
        // mutableRequest.url = URL(string: "your_non_streaming_specific_endpoint")
    }
    return mutableRequest
}

let llmClient = LLMClient(
    standard: .openAi,
    endpoint: vercelEndpoint,
    model: vercelModelName,
    apiKey: vercelBearerToken,
    requestTransformer: transformer
)

Key methods:

• public func send(messages: [ChatMessage], temperature: Double? = nil) async throws -> ChatOutput (for non-streaming requests)
• public func stream(messages: [ChatMessage], temperature: Double? = nil) -> AsyncThrowingStream<ChatOutput, Error> (for setting up a streaming connection)

TooledLLMClient

Manages interactions with an LLM that can utilize a predefined set of tools. It orchestrates a multi-pass conversation:

1. Sends user input (often initially as a string, which it converts to ChatMessage for the LLM) and tool descriptions to the LLM.
2. Parses the LLM's decision and executes the identified tools.
3. Sends the tool execution results back to the LLM (as ChatMessage objects) to generate a final, user-facing response.

Initializer: public init(llmClient: LLMClient)

Key method:

• public func processWithTools(userInput: String, tools: [LLMTool]) async throws -> AsyncThrowingStream<ChatOutput, Error>

Note on Tool Argument Formatting:

The TooledLLMClient includes a default prompt that instructs the LLM to provide arguments like {"toolsToUse": [{"name": "tool_name", "arguments": "<tool_specific_xml_args_or_empty_string>"}]}. The arguments field from this JSON is what gets passed to your tool's execute closure. You'll need to:

1. Ensure the LLM you use can follow this JSON instruction for its tool_calls response.
2. Adapt your tool's execute closure to parse the arguments string as needed (e.g., if it's plain text, XML, or a JSON string itself). The example above simplifies this for clarity.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request or open an Issue if you find a bug or have a feature request.

(Optional: Add guidelines for commit messages, code style, running tests, etc.)

License

swift-llm is released under the Apache License 2.0. See LICENSE file for details.