Devotd Github
NewInference & serving

alibaba/mnn — a lightweight inference framework for mobile and embedded devices

alibaba/mnn is a lightweight deep learning framework that supports model inference and training on mobile and embedded devices.

Repositoryalibaba/mnn ↗

alibaba/mnn is a lightweight deep learning framework that supports model inference and training on mobile and embedded devices.

GitHub description: MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.

stars
★ 16.3K
Forks
2.5K
Language
C++
License
Apache-2.0
Latest release
3.6.1
Read original source ↗View on GitHub ↗

The latest release of alibaba/mnn is 3.6.1. MNN is a lightweight deep learning framework optimized for mobile and embedded devices, supporting both model inference and training. What exactly changed in 3.6.1 cannot be determined from the provided material. Based on the repository description and README, the core is supporting multiple models and compute environments so that inference can be performed on the device.

The README says it supports Tensorflow, Caffe, ONNX and Torchscripts models, along with CNN, RNN, GAN and Transformer. It also supports hybrid computation that uses CPU and GPU together. MNN-LLM is a large language model runtime based on MNN, and MNN-Diffusion is a Stable Diffusion model runtime. It also provides model conversion, compression and training tools. The research material introduces on-device LLM and multimodal inference support through Android apps and backends such as CPU, GPU and Hexagon. The implementation language is C++ and the license is Apache-2.0.

The provided material contains no changelog for 3.6.1 and no performance benchmarks. Therefore the scope of support can be confirmed, but performance on specific devices and models, and the concrete improvements in this release, are difficult to judge from this material alone.