Model Runtime Tooling
Utilities for selecting models that fit hardware, launching runtimes, monitoring use, and managing local model files; it excludes the inference engine itself and general LLM API routing.
Subfields
Hardware Fitters and Selectors
Tools that estimate memory, speed, compatibility, or feasible model sizes for particular hardware; it excludes engines that execute models and leaderboards that only compare quality.
1 projectsModel Launchers and Managers
Desktop apps, launchers, and control utilities that start, configure, switch, or monitor local inference engines and model files; it excludes standalone inference kernels and hosted model gateways.
1 projects| Project | What it does | Stars | Language | License | Latest release |
|---|---|---|---|---|---|
| alexsjones/llmfit | LLMFit checks local hardware and recommends quantized models that fit available CPU, memory, GPU, and VRAM. | ★ 38.1K | Rust | MIT | v1.1.17 |
| receptron/laya | Runs the Laya ONNX decision model in Node.js to produce calibrated classification probabilities and scores. | ★ 960 | TypeScript | MIT | v0.2.0 |