Devotd Github
NewInference & serving

masherdon/siege-ladder — DeepSeek-V4.1-Flash Runtime Research

masherdon/siege-ladder is a repository containing llama.cpp patches and research notes for running DeepSeek-V4.1-Flash-1017 on a single PC.

Repositorymasherdon/siege-ladder ↗

masherdon/siege-ladder is a repository containing llama.cpp patches and research notes for running DeepSeek-V4.1-Flash-1017 on a single PC.

GitHub description: DeepSeek-V4.1-Flash (1017) on one 16 GB GPU + 64 GB RAM, experts streamed from NVMe: llama.cpp patches, a weight assembler, benchmarks and notes. Work in progress.

stars
★ 3
Forks
0
Language
Python
License
MIT-0
Read original source ↗View on GitHub ↗

masherdon/siege-ladder is a research repository that attempts to run DeepSeek-V4.1-Flash-1017 on a single PC with 16 GB of graphics card memory and 64 GB of system memory. There is no latest public release. The repository states that it is work in progress rather than a finished engine, and it provides llama.cpp patches, a weight combination script, measurement tools, and notes. So this is less a new version release than a look at the run method and research materials published in the repository.

The core is a method that streams the model's expert weights from NVMe to divide memory usage. An expert is a bundle of parameters that the model selects to use depending on the input. The README explains that it uses graphics card memory, fixed system memory, and NVMe in a ladder-like arrangement and predicts routing to selectively skip experts. The primary language is Python and the license is MIT-0. This is closer to a concrete implementation record of running a large model under limited memory than a general introduction to an inference engine.

The performance case recorded in the README is a 1M document experiment dated November 4, 2026. That record lists 946,058 tokens processed, a processing speed of 365.78 tok/s, and an answer generation speed of 14.38 tok/s. This is a single run result, and the conditions differ from other performance tables. The README notes that reproducibility is limited and that some components have not been published. Therefore these figures are better regarded as a reference record of ongoing research than as a comparable performance evaluation.