Self-evaluating interview for AI coders

Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.

This repo is to showcase how you can run a model locally and offline, free of OpenAI dependencies.

LLaMA Server combines the power of LLaMA C++ with the beauty of Chatbot UI.

Langport is a language model inference service

BabyAGI-🦙: Enhanced for Llama models (running 100% local) and persistent memory, with smart internet search based on BabyCatAGI and document embedding in langchain based on privateGPT

llm-inference is a platform for publishing and managing llm inference, providing a wide range of out-of-the-box features for model deployment, such as UI, RESTful API, auto-scaling, computing resource management, monitoring, and more.

📚 Local PDF-Integrated Chat Bot: Secure Conversations and Document Assistance with LLM-Powered Privacy

◉ Universal Intelligence: AI made simple.

Local character AI chatbot with chroma vector store memory and some scripts to process documents for Chroma

✨ Your Custom Offline Role Play with LLM and Stable Diffusion on Mac and Linux (for now) 🧙‍♂️

DocMind AI is a powerful, open-source Streamlit application leveraging LangChain and local Large Language Models (LLMs) via Ollama for advanced document analysis. Analyze, summarize, and extract insights from a wide array of file formats—securely and privately, all offline.

Experimental interface environment for open source LLM, designed to democratize the use of AI. Powered by llama-cpp, llama-cpp-python and Gradio.

A quant trading system platform based on FinGPT, demonstrating new applications of large pre-trained Language Models in quantitative finance.

使用numpy实现DeepSeek-R1-Distill-Qwen-1.5B的推理过程，易于学习LLM推理与移植到其它编程语言加速。 Implementing the inference process of DeepSeek-R1-Distill-Qwen-1.5B using numpy, making it easy to learn LLM (Large Language Model) inference and to port to other programming languages for acceleration.

Powering Agent Chains by Constraining LLM Outputs

A simple AI chat using FastAPI, Langchain and llama.cpp

Auto Complete anything using a gguf model

Email Auto-ReplAI is a Python tool that uses AI to automate drafting responses to unread Gmail messages, streamlining email management tasks.

✨ LLM CMD is a toolbox allowing you to use LLM in daily developer commands 💻