github.devotd.
NewInference & serving

deepseek-ai/deepgemm v2.1.1.post3 — GPU Tensor Core Kernel Library

deepseek-ai/deepgemm v2.1.1.post3 is the latest release of a CUDA kernel library that compiles and runs FP8, FP4, and BF16 matrix multiplication and MoE operations at runtime on NVIDIA GPUs.

Repositorydeepseek-ai/deepgemm ↗

deepseek-ai/deepgemm v2.1.1.post3 is the latest release of a CUDA kernel library that compiles and runs FP8, FP4, and BF16 matrix multiplication and MoE operations at runtime on NVIDIA GPUs.

GitHub description: DeepGEMM: clean and efficient BLAS kernel library on GPU

stars
★ 9.1K
Forks
1.4K
Language
Cuda
License
MIT
Latest release
v2.1.1.post3
Read original source ↗View on GitHub ↗

The latest release of deepseek-ai/deepgemm is v2.1.1.post3. It is a BLAS kernel library for NVIDIA GPUs that accelerates matrix multiplication (GEMM) and Mixture of Experts (MoE) operations. Its core operations are FP8, FP4, and BF16 matrix multiplication and MoE, with the kernels compiled at runtime and optimized for the target GPU.

The README describes the repository as an integrated high-performance tensor core kernel library. Beyond FP8, FP4, and BF16 GEMM, the scope includes Mega MoE, a fused MoE that overlaps communication with computation, MQA scoring used by the lightning indexer, and HyperConnection, all in a single CUDA codebase. Every kernel is compiled at runtime with DeepJIT, so no CUDA compilation is needed during installation. The license is MIT, the primary language is Cuda, and the recorded star count is 9050.

However, the changes in v2.1.1.post3 are not included in the provided material. What was added or modified in this version has to be checked directly in the repository's release notes. No performance benchmark figures or test environment are given either, so it is too early to judge how fast it is on which GPU and model size. The research record assigns only a momentum score of 6.3 and does not compute novelty, technology, or quality scores. It is therefore hard to make an adoption decision premised on unverified performance numbers.