Model Memory Compression
Runtime techniques that reduce the memory footprint of loaded model weights or state with minimal quality loss; it excludes offline quantization-format research and inference benchmarking suites.
| Project | What it does | Stars | Language | License | Latest release |
|---|---|---|---|---|---|
| surya-koritala/glyd | A lossless runtime codec for BF16 weights and KV-cache data that reduces GPU memory and reconstructs exact values. | ★ 255 | Rust | NOASSERTION | v0.32.1 |
| srkrz23/vramkit | Provides experimental compression for LLM inference weights and KV caches to reduce VRAM use. | ★ 3 | Python | — | — |