LLM inference in C/C++

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

Stable Diffusion and Flux in pure C/C++

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

Suno AI's Bark model in C/C++ for fast text-to-speech generation

Whisper Dart is a cross platform library for dart and flutter that allows converting audio to text / speech to text / inference from Open AI models

Port of MiniGPT4 in C++ (4bit, 5bit, 6bit, 8bit, 16bit CPU inference with GGML)

CLIP inference in plain C/C++ with no extra dependencies

WIP Library Text To Speech From Suno AI's Bark in C/C++ for fast inference

GENERAL Ai Library For DART & Flutter

Inference Vision Transformer (ViT) in plain C/C++ with ggml

llama.cpp (GGUF LLMs) and llava.cpp (GGUF VLMs) for ROS 2

A ggml (C++) re-implementation of tortoise-tts

Booster - open accelerator for LLM models. Better inference and debugging for AI hackers

LangCommand is a local inference command-line tool that transforms natural language descriptions into shell commands.

C++17 port of Demucs v3 (hybrid) and v4 (hybrid transformer) models with ggml and Eigen3

Your customized AI assistant - Personal assistants on any hardware! With llama.cpp, whisper.cpp, ggml, LLaMA-v2.

Speech-to-Text based on SileroVAD + whisper.cpp (GGML Whisper) for ROS 2

C++ inference wrappers for running blazing fast embedding services on your favourite serverless like AWS Lambda. By Prithivi Da, PRs welcome.

CLIP model deploy in plain C/C++ using ggml machine learning library