GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
-
Updated
Sep 5, 2024 - C++
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
A programming framework for agentic AI 🤖
OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
Official inference library for Mistral models
The easiest way to serve AI apps and models - Build reliable Inference APIs, LLM apps, Multi-model chains, RAG service, and much more!
Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud.
Superduper: Integrate AI models and machine learning workflows with your database to implement custom AI applications, without moving your data. Including streaming inference, scalable model hosting, training and vector search.
High-speed Large Language Model Serving on PCs with Consumer-grade GPUs
Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
Code examples and resources for DBRX, a large language model developed by Databricks
⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡
Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.
Sparsity-aware deep learning inference runtime for CPUs
📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.
irresponsible innovation. Try now at https://chat.dev/
Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads
Morpheus - A Network For Powering Smart Agents - Compute + Code + Capital + Community
Add a description, image, and links to the llm-inference topic page so that developers can more easily learn about it.
To associate your repository with the llm-inference topic, visit your repo's landing page and select "manage topics."