#

vllm

Here are 6 public repositories matching this topic...

bricks-cloud / BricksLLM

🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.

api docker golang open-source security privacy ai azure rest-api postgresql self-hosted artificial-intelligence ycombinator openai gpt llm generative-ai anthropic vllm

Updated Jan 5, 2025
Go

substratusai / kubeai

AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.

kubernetes ai k8s whisper autoscaler openai-api llm vllm faster-whisper ollama vllm-operator ollama-operator inference-operator

Updated Feb 26, 2025
Go

InftyAI / llmaz

☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!

kubernetes huggingface llm modelscope llamacpp vllm llm-inference text-generation-inference ollama sglang

Updated Feb 24, 2025
Go

llmariner / llmariner

Extensible generative AI platform on Kubernetes with OpenAI-compatible APIs.

kubernetes ai gpu ml inference openai operator k8s autoscaling fine-tuning multi-cluster llm vllm

Updated Feb 26, 2025
Go

Climatik-Project / Climatik-Project

Carbon Limiting Auto Tuning for Kubernetes

kubernetes sustainability kepler kubernetes-operator power-capping green-computing keda kserve llm vllm llm-inference

Updated Nov 11, 2024
Go

TimeSurgeLabs / promptproxy

Call many AIs from a single API.

docker ai openai llama huggingface openai-api llm vllm openai-api-proxy llama2

Updated Mar 28, 2024
Go

Improve this page

Add a description, image, and links to the vllm topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the vllm topic, visit your repo's landing page and select "manage topics."