Projects

xorbitsai/inference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

View on GitHub ↗

stars: ★ 9.6K
Forks: 884
Language: Python
License: Apache-2.0
Latest release: v4.0.1

Posts

NewOCT 20

xorbitsai/inference v4.0.1 — Language, Speech and Multimodal Model Service

Introduces the model service features of xorbitsai/inference (Xinference) and its latest release, v4.0.1.

xorbitsai/inference Inference & servingPython