AI-powered text compression library for RAG systems and API calls. Reduce token usage by up to 50-60% while preserving semantic meaning with advanced compression strategies.

Offline Dictionary-based Compression (Re-Pair, Recursive Pairing)

LLMA = LLM + Arithmetic coder, which use LLM to do insane text data compression. LLMA=大模型+算术编码，它能使用LLM对文本数据进行暴力的压缩，达到极高的压缩率。

Extreme inline text compression for HTML / JS. A custom pipeline that generates stand-alone HTML or JS files which embed competitively compressed self-extracting text, with file sizes of 25% - 40% the original.

🍶 llm-distillery ⇢ use LLMs to run map-reduce summarization tasks on large documents until a target token size is met.

This project is about the implementation of the LZ77 and Deflate COMPRESSION algorithm.

A simple visualization for huffman tree, which is used in encoding text.

Text compression tool ⚡

Cross platform diff engine for .net core used to compare string values and generate a comparison output

Using Huffman code to encode and compress the secret text data along with using histogram shifting to encode the data into the image.

LZ77 algorithm, providing efficient lossless data compression and decompression capabilities. It offers a reliable and flexible solution for reducing file sizes in various programming languages.

An attempt to compress the enwik8 file

A text compression tool & library

Huffman code

Data driven approach for compressing text using an encoder-decoder architecture and word-level embeddings.

A simple python implementation for the well-known compression algorithm LZW

PAQ8PX compression archiver

Official Implementation of Paper "A Novel Metric for Evaluating Semantics Preservation"

Compressor for UTF-8 encoded text