Devotd Github
NewMultimodal

diffuspace/clover-35b-a3b-preview — Block Diffusion Generation Preview

Clover-35B-A3B in DiffuSpace is a language model preview that combines a mixture-of-experts architecture with block-level diffusion generation.

Repositorydiffuspace/clover-35b-a3b-preview ↗

Clover-35B-A3B in DiffuSpace is a language model preview that combines a mixture-of-experts architecture with block-level diffusion generation.

stars
★ 70
Forks
3
Read original source ↗View on GitHub ↗

DiffuSpace introduces the Clover-35B-A3B preview model in the diffuspace/clover-35b-a3b-preview repository. According to the README, the model combines a mixture-of-experts (MoE) architecture with block-level diffusion generation. The README describes the Hugging Face weights and the Transformers, SGLang, and llama.cpp inference paths. This is not a changelog for an official release, but an introduction to the model preview and the ways to run it.

The model has 35B total parameters, of which roughly 3B are activated per token. Block-level diffusion generation differs from writing a sentence one token at a time: it generates tokens in blocks. The README explains that bidirectional attention is used within blocks, and also points to a context of up to 131072 tokens. The core of the repository, then, is that it distinguishes the large total model scale from the per-token active scale, and presents the block generation approach together with its inference paths.

The available materials are the README and the diagrams; the training code and training instructions are stated to be released in the future. So while the described architecture and execution paths can be understood, implementation details or generation quality cannot be verified with the code in the repository. Rather than judging performance on the basis of the material at this stage, it is more reasonable to treat this as a research-oriented model introduction until the implementation and evaluation evidence are released.