github.devotd.
NewInference & serving

deepseek-ai/deepinfer — an inference framework for DeepSeek-V4.1-Flash

deepseek-ai/deepinfer is an offline inference framework that names DeepSeek-V4.1-Flash as its supported target.

Repositorydeepseek-ai/deepinfer ↗

deepseek-ai/deepinfer is an offline inference framework that names DeepSeek-V4.1-Flash as its supported target.

GitHub description: DeepInfer: High-performance, large-scale inference framework for DeepSeek models

stars
★ 3.3K
Forks
240
Language
Python
License
Apache-2.0
Read original source ↗View on GitHub ↗

deepseek-ai/deepinfer is an offline inference framework for DeepSeek models. The repository README names DeepSeek-V4.1-Flash as its supported target. FACTS contains no latest release information, so it cannot be described as a specific version's release or change.

This project provides execution paths and scheduling features for running DeepSeek model inference. The README presents optimized kernels, data parallel attention and expert parallelism, draft token verification, asynchronous batch scheduling, deterministic sampling and structured output as its features. Draft token verification (speculative verification) is an inference method that verifies tokens produced as drafts. NVIDIA SM100 and Huawei Ascend 950 are also named as supported targets. It is a Python project under the Apache-2.0 license.

However, the README states that a separate key-value cache and work disaggregation, among other things, are required for a complete serving configuration. The measurement environment of the performance charts is presented as SM100 with 4 prefill and 64 decode, but no reproduction procedure or independent measurement is confirmed in FACTS. It is therefore better to distinguish the implementation scope for a specific model from unverified performance evidence than to judge a performance lead from this material alone.