guqiong96/lsglang

guqiong96/lsglang 1.7.2 是一个发布提交，将 GPU expert 缓存填充路径拆分为两条，并开启了推测性验证的默认值。

GitHub 项目说明: Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parallel architecture, suitable for MOE model hybrid inference.

星标

★ 147

复刻

17

语言

Python

许可证

Apache-2.0

最新版本

lsglang-v1.6.0