Projects

andrei-borcea/liyab

🔥 Run large open language models on your phone. A C++20 GGUF inference engine that streams models larger than RAM from flash, paces itself by the device's thermal forecast, and powers a private on-device assistant. Android · iOS · macOS · Linux.

View on GitHub ↗

stars: ★ 7
Forks: 0
Language: C++
License: Apache-2.0
Latest release: —

Posts

NewOCT 18

andrei-borcea/liyab — a GGUF inference engine for phones

andrei-borcea/liyab is a C++20 inference engine that streams GGUF models larger than RAM from flash storage so they run on phones.

andrei-borcea/liyab Inference & servingC++