github.devotd.
EN
English 한국어简体中文日本語
Projects

lateos-ai/reflex

A high-performance, GGUF-native Rust & CUDA inference engine optimized for cold-start latency and real-time 'System 1' agent decision loops.

View on GitHub ↗
stars
★ 62
Forks
1
Language
Rust
License
MIT
Latest release
v0.2.9-runpod-hub

Posts

1

More in this field

Local Inference Engines ↗
  • ollama/ollama

    A general-purpose inference engine that runs many open models locally or in the cloud.

  • ggml-org/llama.cpp

    A portable C/C++ inference engine executes large language models locally.

  • niko1221/strata

    A local engine for Qwen3.8-Flash-Next with one-click setup and OpenAI- or Anthropic-compatible localhost APIs.

  • antirez/ds4

    Runs selected DeepSeek, GLM, and Qwen models locally across Metal, CUDA, and ROCm with SSD streaming.

  • jundot/omlx

    OMLX is a local Apple Silicon LLM server with continuous batching, SSD and KV caching, and menu-bar management.

  • flashml-org/freetoken

    An engine that serves massive MoE models locally across GPU, CPU, and host memory.