srkrz23/vramkit is a Python repository that compresses weights and the KV cache separately during LLM inference to cut memory usage. No separate release version is given in FACTS. The README states support for ROCm, CUDA, MPS and CPU. It is an approach aimed at handling larger models on laptops with unified memory, but its actual scope of applicability should be judged against the measurements and implementation presented.
The weight codec uses a 32-bit range coder and a two-dimensional adaptive context model. According to the README, self_attn.o_proj(int4) of Qwen2.5-0.5B recorded 2.676 bits per weight losslessly. That is 33.1% below raw 4-bit, and also below the 2.717 bits of zstd-19. The average across all tensors was 2.92 bits, described as a 27% reduction against 4-bit. The KV cache quantizes keys per channel and values per token. The README reports a 2x compression with 0.1% loss relative to int8 at a reference perplexity of about 18. The per-token approach under the same memory condition shows 4.6% loss, and int4 keys show 25.6% loss.
All of these numbers are measurements taken by the author on an Apple Silicon laptop, and no independent verification is presented in FACTS. The author states that an earlier version evaluated quality with cosine similarity and that this error was corrected. The repository has no test directory and no continuous integration setup. It is therefore more appropriate to treat the benchmark numbers as author-reported results that need reproduction and further verification.