fronteagle739/streaming-llama-stack is a C-language inference stack built on a llama.cpp fork. The README describes a setup that streams weights from NVMe to run a 483 GiB-class model in the 32 GiB of graphics memory on an RTX 5090. The repository description also cites streaming and distributed inference for low-memory systems that leverage fast SSD arrays. What is introduced here is the configuration and feature set presented in the repository README; no official release version is published in FACTS.
This configuration handles large-model inference by streaming from NVMe instead of loading the entire model weights into graphics memory. The README documents a streaming prefill that processes prompts and a decode kernel for DeepSeek V4.1 Flash. An OpenAI-compatible API, MCP tools, and a web interface are included as well. It also describes an optional setup that splits prefill and decode work with a Mac Studio. So this is closer to a stack bundling both execution paths and invocation methods than to a single inference kernel. The license is MIT, and the repository language is listed as C.
The README's performance table presents text-based measurements together with the hardware configuration, and also discloses the constraints on network connectivity and memory. This is a record for a specific setup, not evidence that the same results will appear on other hardware. The repository was created on 2026-10-25, and the star count recorded in FACTS is 6. No official release is registered either. To the extent that the README describes it as an experimental setup, the scope currently confirmed is limited to the presented execution path and measurement environment.