New
magnitudedev/magnitude — a local inference engine tailored to the hardware
magnitudedev/magnitude is a Rust-based open-source engine that tunes kernels to the hardware and uses them for model inference.
Projects
magnitudedev/magnitude
Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, or just a CPU.
View on GitHub ↗- stars
- ★ 7K
- Forks
- 481
- Language
- Rust
- License
- Apache-2.0
- Latest release
- @magnitudedev/cli@0.2.9
Posts
1No matching stories.