microsoft/llmlingua is a Python project that compresses prompts fed into large language models (LLMs). The repository description states that it reduces prompts and KV-Cache to speed up inference, and the latest release is v0.2.2. Prompt compression is a method of trimming the unnecessary parts of the tokens that enter model input so they can be passed along more briefly. This project uses a smaller language model to judge what should be removed, taking charge of the compression step.
Several branches are gathered under the name LLMLingua Series. The repository description states that LLMLingua achieves up to 20x compression with only small performance loss. LongLLMLingua covers long-context handling and improved RAG performance, while LLMLingua-2 covers task-agnostic prompt compression. SecurityLingua is a safety guardrail model for security-aware prompt compression that detects LLM jailbreak attacks and helps generate safe responses. The implementation language is Python and the license is MIT, and it was first released on July 7, 2023. At the very front of the repository description, EMNLP'23 and ACL'24 are listed, and the star count is 6762.
However, the evidence stays within the scope revealed by the repository description and README. The up-to-20x compression and small performance loss are only values written in the description text; no per-condition benchmarks or comparison figures against other compression methods are provided. What v0.2.2 changed must be checked directly in the release notes. Among the collected metrics, momentum is 2.1 on a 10-point scale, and the novelty, technical, and quality items are empty. The star count shows interest only and does not prove performance. It is an option worth referencing for teams looking to reduce input tokens by compressing long RAG documents and conversation logs, but until you verify directly how much loss remains on which models and tasks, it is difficult to treat as an expectation.