surya-koritala/glyd is a Rust compression implementation that explains how it stores bf16 weights and the KV cache in GPU memory with fewer bits and restores the original values at execution time. The latest release is v0.31.1. However, the available material does not state what changed in this version. Besides the lossless path, it also offers a lossy path, with :kestrel at 6.5 bits and :swift at 5.5 bits.
The purpose of this approach is to reduce the GPU memory required to run a model. According to the measured table in the README, when running Qwen3.5-9B in vLLM, weight memory drops from 17.66 GiB to 12.68 GiB (-28.2%) on an RTX 4090, and from 13.89 GiB (-21.3%) on an RTX 5090. On an RTX 4090 24GB, a context that could not handle even 4,096 tokens is said to have grown to 189,440 tokens. Qwen3.8-27B shrank from 50.22 GiB to 38.77 GiB (-22.8%) on an H100, and in a comparison that restores hierarchical matrices bit by bit it recorded 49.52 GB to 33.28 GB (-32.8%). The 33% in the description corresponds to this comparison value.
The execution requirements are Linux x86_64 or aarch64, an NVIDIA Ampere or newer GPU, and driver 580 or later. The installation methods given are a shell script and a glyd quantization backend for vLLM. Since the published performance figures are limited to the measured table in the project README, no basis is given for generalizing them to other models or environments. The memory-saving potential is therefore concrete, but the results need to be checked in the relevant environment before applying it.