Projects

kvmem/kvmem

KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.

Posts

NewOCT 18

kvmem/kvmem v0.18.0 is a C++ project that keeps the KV cache in host memory and brings it back to the GPU when needed, for long-context inference.