Model Engineering
Tools and methods for model training, offline compression, general evaluation, analysis, provenance, and access routing. Compression-oriented distillation belongs under model-compression-and-quantization; general adaptation under training. Excludes model releases, runtime execution, and security-primary analysis or evaluation, which belong under ai-security.
Subfields
Model Access and Routing
Gateways and routers that provide model-compatible APIs, select providers, enforce usage policy, or relay subscriptions; it excludes local inference engines and MCP tool proxies.
4 projectsModel Evaluation
Benchmarks, datasets, judges, and harnesses primarily measuring general model or agent capability, quality, reliability, or efficiency. Safety-, jailbreak-, injection-, harmfulness-, or security-focused evaluation belongs under ai-safety-evaluation regardless of benchmark or harness packaging. Excludes training tools and production execution systems.
4 projectsModel Training and Fine-Tuning
Frameworks, recipes, and tools that optimize model weights through pretraining continued, fine-tuning, preference learning, or distillation; it excludes inference serving and prompt-level agent skills.
2 projects| Project | What it does | Stars | Language | License | Latest release |
|---|---|---|---|---|---|
| diegosouzapw/omniroute | Routes requests across 359 providers and 1,200+ models through one endpoint with quota-aware fallback. | ★ 78K | TypeScript | MIT | v3.8.51 |
| yetone/magpie | A local gateway translates four model protocols and routes agent requests across accounts and models using quotas and weights. | ★ 15.6K | Go | MIT | v0.1.1285 |
| yinsongxu/llm2jev | Turns existing local LLMs and MLLMs into Jev-style structured decision models without fine-tuning, returning judgments from one prefill. | ★ 449 | Python | Apache-2.0 | v0.2.0 |
| scaleapi/agentenv-framework | A framework for researchers, engineers, and domain experts to jointly build realistic, reproducible RL environments. | ★ 322 | Python | Apache-2.0 | v0.9.1334 |
| deepseek-ai/deepep-ascend | A low-level library that accelerates all-to-all communication for distributed MoE training and inference on Huawei Ascend NPUs. | ★ 260 | C++ | — | — |
| zihan-su/self_gradient_forcing_plus | Trains autoregressive video generators by separating context-writing and denoising parameters to reduce gradient conflict. | ★ 196 | Python | Apache-2.0 | — |
| qwenlm/blenderworld | 223-task Blender benchmark with 2,651 checks evaluates agents creating editable 3D scenes from text, images and video. | ★ 124 | Python | MIT | — |
| composo-ai/dishwasherbench | Benchmarks language-model agents on goal completion in a physics simulation of a real dishwasher. | ★ 70 | TypeScript | MIT | — |
| jevagentharness/jevagentharness | A local OpenAI-compatible proxy that classifies coding-agent steps and routes them to fast or strong models. | ★ 11 | Python | MIT | — |
| cmu-l3/reuf2f | A benchmark where coding agents prove theorems or counterexamples in Lean-formalized undergraduate research problems. | ★ 9 | Lean | NOASSERTION | — |