AI-native (edge and LLM) proxy for agents. Handles all the pesky heavy lifting in building agentic apps -- fast ⚡️ query routing, seamless integration of prompts with business APIs, and unified access and observabilty of LLMs. Built by the contributors of Envoy proxy.
proxy routing gateway prompt proxy-server openai envoy envoyproxy llms generative-ai llmops llm-inference ai-gateway llm-gateway llm-routing
-
Updated
Feb 21, 2025 - Rust