Devotd Github
NewLLM infrastructure

afdep/afdep-comm v1.0.0 — an expert-parallel communication layer for AFD

afdep/afdep-comm v1.0.0 provides a communication layer that exchanges tokens and results in a configuration that separates Attention and FFN.

Repositoryafdep/afdep-comm ↗

afdep/afdep-comm v1.0.0 provides a communication layer that exchanges tokens and results in a configuration that separates Attention and FFN.

GitHub description: Expert-parallel communication for Attention–FFN Disaggregation (AFD), built on DeepEP V2

stars
★ 30
Forks
0
Language
Python
License
Apache-2.0
Latest release
v1.0.0
Read original source ↗View on GitHub ↗

afdep/afdep-comm is the expert-parallel communication layer for AFD (Attention–FFN Disaggregation) configurations that place Attention and FFN (feed-forward network) on different GPUs. The latest release is v1.0.0. However, the provided repository description does not list anything as newly changed in this version.

This repository handles the communication that sends tokens to experts and returns the computation results. It is a research implementation that adapts the DeepEP V2 kernel to these two roles, written in Python. Routing, expert computation, and service scheduling are left to the caller. So the focus is on the communication path inside a full MoE inference system rather than on the system as a whole. The repository presents comparative measurements based on the same kernel and execution procedure, and describes separately the paths within a single NVLink domain and the paths that connect multiple domains. The license is Apache-2.0.

The scope covered by the measurements is limited to the reference implementation and the execution procedure presented in the repository. It is not data showing the performance of the whole system including routing or expert computation, so it is difficult to use as a basis for performance judgments beyond that. The specific change log for v1.0.0 is also absent from the verifiable description, so at present it is appropriate to treat this as material for understanding the implementation of the communication layer and the scope of the measurements.