Devotd Github
NewInference & serving

musicurgy/delta-inference-engine 0.92 — Windows inference server

musicurgy/delta-inference-engine is a repository that the README describes as running DeepSeek-V4.1-Flash on Windows and providing an OpenAI-compatible API.

Repositorymusicurgy/delta-inference-engine ↗

musicurgy/delta-inference-engine is a repository that the README describes as running DeepSeek-V4.1-Flash on Windows and providing an OpenAI-compatible API.

GitHub description: Delta Inference Engine: DeepSeek-V4.1-Flash on one NVIDIA GPU, natively on Windows (OpenAI-compatible server)

stars
★ 7
Forks
2
Language
C++
License
MIT
Latest release
0.92
Read original source ↗View on GitHub ↗

musicurgy/delta-inference-engine is a C++ repository that the README describes as running DeepSeek-V4.1-Flash locally on a single NVIDIA GPU under Windows and providing an OpenAI-compatible API. The latest release recorded in FACTS is 0.92. However, since no specific items are given for what changed in 0.92, it cannot be asserted as a change point. The repository publishes CUDA, SYCL and HIP code along with model execution tools, format converters and verification code.

According to the README, this port is based on Xolotl and Strata. It loads the expert units of the 552B-parameter model into memory at 2-bit precision, and reads the required units from storage when memory runs short. The README also puts forward up to 1M tokens of context. The install batch file downloads or builds the engine and prepares the model files. The license is MIT.

However, the performance results in the README were measured on a specific RTX 3090 setup and are not independently verified results. The single-GPU execution claim and the stated context length alone make it difficult to judge the memory requirements or processing speed on other PCs. This repository therefore shows a Windows execution path, but the published verification is limited as a basis for general-purpose performance.