Burn is a new comprehensive dynamic Deep Learning Framework built using Rust with extreme flexibility, compute efficiency and portability as its primary goals.

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models

Postgres with GPUs for ML/AI apps.

All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI tools & agents, with access to OpenAI, Claude, Gemini, Ollama, Groq, and more.

Blazingly fast LLM inference.

A blazing fast inference solution for text embeddings models

A CLI tool to convert your codebase into a single LLM prompt with source tree, prompt templating, and token counting.

LSP-AI is an open-source language server that serves as a backend for AI-powered functionality, designed to assist and empower software engineers, not replace them.

AICI: Prompts as (Wasm) Programs

Scalable, Low-latency and Hybrid-enabled Vector Search in Postgres. Revolutionize Vector Search, not Database.

A program that provides LLMs with the ability to complete complex tasks using plugins.

All-in-one infrastructure for search, recommendations, RAG, and analytics offered via API

BAML is a language that helps you get structured data from LLMs, with the best DX possible. Works with all languages. Check out the promptfiddle.com playground

⚙️🦀 Build portable, modular & lightweight Fullstack Agents

Instant, controllable, local pre-trained AI models in Rust

《构筑大语言模型应用：应用开发与架构设计》一本关于 LLM 在真实世界应用的开源电子书，介绍了大语言模型的基础知识和应用，以及如何构建自己的模型。其中包括Prompt的编写、开发和管理，探索最好的大语言模型能带来什么，以及LLM应用开发的模式和架构设计。

`llm-chain` is a powerful rust crate for building chains in large language models allowing you to summarise text and complete complex tasks

TensorZero creates a feedback loop for optimizing LLM applications — turning production data into smarter, faster, and cheaper models.

Korvus is a search SDK that unifies the entire RAG pipeline in a single database query. Built on top of Postgres with bindings for Python, JavaScript, Rust and C.

A realtime serving engine for Data-Intensive Generative AI Applications