github.devotd.
NewInference & serving

mw00/project-maya v1.1.15 — a GLM-5.3-Flash engine for older GPUs

mw00/project-maya is a C++ engine that splits experts across GPU, RAM and NVMe SSD to run GLM-5.3-Flash on a personal GPU.

Repositorymw00/project-maya ↗

Project Maya - run GLM-5.3-Flash (321B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

stars
★ 1K
Forks
136
Language
C++
License
MIT
Latest release
v1.1.15
Read original source ↗View on GitHub ↗

The latest release of mw00/project-maya is v1.1.15. This project is a dedicated engine for running the 321B MoE model GLM-5.3-Flash on a personal GPU. The provided information contains no changelog for v1.1.15, so it is not possible to tell what changed in this version.

The core is a hierarchical memory approach that splits the model's experts across GPU, RAM and NVMe SSD, and moves them between tiers during a conversation. Frequently used experts stay on the GPU, the next tier sits in RAM, and the rest go to the SSD. It ships five of its own quantization formats; the four formats other than M-Derisked are described in the repository as reaching 97.7–99.2% zero-shot accuracy relative to the FP8 model. It also provides OpenAI- and Anthropic-compatible APIs, up to 8 concurrent requests, a browser chat interface and a monitoring dashboard. It is an MIT-licensed C++ project.

The performance table consists of user-submitted measurements:118 tok/s decode and 4129 tok/s prefill on 4x RTX 4090, 23–30+ tok/s on a single RTX 5090, 26–27 tok/s on a single RTX 3090, and up to 29.6 tok/s on AMD Radeon 8065S. Support for older NVIDIA Pascal GPUs is experimental. These figures are therefore reference values that show what is possible on each type of hardware, and real performance may vary depending on the configuration.