Projects

jitllm/jitllm

Smart inference engine that adapts to your hardware to run models faster, even when they don’t fit in memory

Posts

NewOCT 18

jitllm/jitllm is a Go-based inference engine designed to generate kernels tailored to the hardware and to handle models larger than memory.