cipheraxat/nanoserve is a small language model inference engine built with PyTorch. The latest registered release is not in FACTS, and no update stating a version could be confirmed. According to the repository description, it implements page-based key-value caching, continuous batching, chunked prefill, prefix caching, and an OpenAI-compatible server. It also includes recomputation-based preemption and a Triton paged-attention kernel.
The engine provides Qwen3 models through a compatible interface. Key-value caching is how a model retains information about previous tokens, and this repository manages it in fixed-size blocks. Continuous batching and chunked prefill are the paths that handle multiple requests and inputs, while prefix caching reuses those results across requests that share the same beginning. It also implements page-based attention computation kernels in Triton and provides an OpenAI-compatible server. The license is MIT and the implementation language is Python.
According to the README, automated tests check whether generated tokens match the results of a reference transformer under several conditions, such as continuous batching and cache reuse. This is a description of token-result consistency; FACTS presents no figures that demonstrate performance. Therefore, what can currently be confirmed is limited to the scope of feature implementation and result-consistency testing, and actual throughput or latency cannot be judged.