Projects

professorpalmer/bonsai-ada-surgery

Bonsai 2 27B: 262k context (trained max) on 12 GB, 54 to 97 tok/s decode, 2x prefill, q8_0 KV recipe, grammar tool calls 9/9. Same ternary weights. Windows bundle + receipts.

View on GitHub ↗

stars: ★ 70
Forks: 6
Language: Python
License: NOASSERTION
Latest release: bundle-20261017b

Posts

NewOCT 18

professorpalmer/bonsai-ada-surgery bundle-20261017b — a 12GB serving recipe

A repository documenting a recipe for serving Bonsai 2 27B on a 12GB RTX 4070, along with a hierarchical KV cache and tool-grammar optimization.

professorpalmer/bonsai-ada-surgery Inference & servingPython