TrustGate

A powerful, AI Gateway designed from scratch for AI

Documentation | Quick Start | Community

✨ Features

• 🚀 High Performance: Built in Go for maximum efficiency and minimal latency
• 🌍 Multi-Provider: Multiple LLM provider support
• 🤖 AI-Ready: Optimized for AI model providers (OpenAI, Anthropic, etc.)
• 🔄 Fallback Ready: Built-in model fallback capabilities
• 🔄 Load Balancing: Advanced algorithms including round-robin, weighted round-robin, and IP hash
• 🔒 Security: Built-in authentication, rate limiting, and token management
• 🔌 Plugin System: Extensible architecture for custom functionality
• ⚡ Real-time Config: Dynamic configuration without restarts
• ☁️ Cloud Agnostic: Deploy anywhere with cloud-agnostic architecture

🚀 Quick Start

Using Docker Compose

# Clone the repository
git clone https://github.com/NeuralTrust/TrustGate.git
cd TrustGate

# Start the services
docker compose -f docker-compose.prod.yaml up -d

Using Kubernetes

Coming soon

Local Development

# Start dependencies
docker compose up -d redis postgres

# Run the servers
./scripts/run_local.sh

🏗️ Architecture

TrustGate consists of two main components:

1. Admin API (Port 8080)

   • Gateway management
   • Configuration management
   • API key management
   • Plugin configuration

2. Proxy API (Port 8081)

   • Request routing
   • Load balancing
   • Plugin execution

📊 Monitoring

TrustGate provides comprehensive monitoring through Prometheus metrics:

# config.yaml
metrics:
  enabled: true                # Enable metrics collection
  enable_latency: true        # Basic latency metrics
  enable_upstream: true       # Upstream latency tracking
  enable_connections: true    # Connection tracking
  enable_per_route: true     # Per-route metrics

Key Metrics

• trustgate_requests_total - Request counts by gateway, method, and status
• trustgate_latency_ms - Overall request processing time
• trustgate_detailed_latency_ms - Granular latency by service/route
• trustgate_upstream_latency_ms - Upstream service latency
• trustgate_connections - Active connection tracking

Prometheus Configuration

scrape_configs:
  - job_name: 'trustgate'
    static_configs:
      - targets: ['localhost:9090']

For detailed metrics documentation, dashboards, and advanced queries, see our Monitoring Guide.

🔌 Plugins

Extend functionality with plugins:

type Plugin interface {
    Name() string
    Stages() []types.Stage
    AllowedStages() []types.Stage
    Execute(ctx *Context) error
}

Current Plugins:

• Rate Limiter
• Token Rate Limiter
• External API Call
• Data Masking (Pre-defined entities, custom keywords, regex)
• Prompt Moderation (keywords, regex)
• Toxicity Detection (Open AI Moderation API)

🔜 Work in progress (Coming Soon)

Security

• Jailbreak Protection

  • Azure Prompt Guard
  • AWS Prompt Guard
  • GCP Prompt Guard

• Toxicity Detection

  • Azure Content Safety API

• Prompt Moderation

  • Topic detection (accepted/denied)

• Network Security

  • CORS protection
  • SQL Injection prevention
  • Cross-site injection protection

🤝 Contributing

We love contributions! Please see our Contributing Guide for details.

1. Fork the repository
2. Create your feature branch
3. Commit your changes
4. Push to the branch
5. Open a Pull Request

📜 License

TrustGate is licensed under the Apache License 2.0 - see the LICENSE file for details.

🌟 Star History

📫 Community & Support

• Documentation
• Slack Community
• GitHub Issues
• Twitter
• Blog

Made with ❤️ by NeuralTrust