Projects

ipads-sai/mobiinfer-android

MobiInfer runs 35B-A3B-class MoE models fully on-device on the Qualcomm Hexagon NPU, streaming experts from flash with lookahead scheduling (LES) and DSpark speculation.

View on GitHub ↗

stars: ★ 16
Forks: 1
Language: C++
License: MIT
Latest release: v0.1-alpha

Posts

NewOCT 19

ipads-sai/mobiinfer-android — On-device MoE for Snapdragon NPUs

ipads-sai/mobiinfer-android is a C++ Android repository that runs 35B-A3B-class MoE models on device on the Qualcomm Hexagon NPU.

ipads-sai/mobiinfer-android Inference & servingC++