Embedded Device Inference
Compact inference runtimes for microcontrollers, phones, consoles, and other constrained devices; it excludes full server clusters and general embedded agent frameworks without local model execution.
| Project | What it does | Stars | Language | License | Latest release |
|---|---|---|---|---|---|
| andrei-borcea/liyab | Runs GGUF language models on phones by streaming models larger than RAM from flash and adapting to thermal conditions. | ★ 7 | C++ | Apache-2.0 | — |