introspective-diffusion/dlmrt is a single-request inference engine for block diffusion language models. Instead of generating tokens one at a time, block diffusion language models denoise whole token blocks. dlmrt applies this process to one request at a time. According to the repository description, it supports the QwenDLM, LLaDA2.1-mini, and DiffusionGemma checkpoints, and is the inference engine for On-Policy Residual Learning for Diffusion Language Models (ORL). There is no publicly released version.
The core of the implementation is running the entire denoising loop in the conditional nodes of a GPU CUDA graph. The host is described as starting one block and reading the resulting tokens. When multiple GPUs are used, tensor parallelism is performed with fused NVLink reduction to avoid host synchronization. It also includes an overlapped execution method using programmatic dependent execution in Triton and DeepGEMM. There are an FP8 block-scale projection and a Blackwell NVFP4 expert path, and it can run on a single GPU or an NVLink node. It also provides an HTTP server and a HumanEval benchmark tool. It is a Cuda project licensed under Apache-2.0.
The performance figures are hard to take as grounds for judgment. The TPS and round latency table is a value reported by the project itself, and the test suite is for checking whether the numbers match the reference implementation. Therefore, actual throughput or latency performance cannot be verified from this material alone. There is also no public release, so what can be confirmed today is limited to the inference structure specific to block diffusion models and its implementation.