The latest release of triton-inference-server/server is v2.73.0. This repository is open-source inference service software that simplifies AI inference, and it is built for inference in cloud and edge environments. Its core purpose is to bundle models built with different machine learning frameworks into a single service and deploy them together.
The support scope laid out in the README is broad. It can deploy models built with TensorRT, PyTorch, ONNX, OpenVINO, Python, and RAPIDS FIL, and the targets it runs on are NVIDIA GPUs, x86 and ARM CPUs, and AWS Inferentia. Inference modes are divided into real-time, batch, ensemble, and audio/video streaming. An ensemble is a configuration that combines the results of several models into one. Clients call models over two protocols, HTTP/REST and gRPC, and installation and usage are documented around Docker, with tutorials and examples included. The license is BSD-3-Clause and the primary language is Python. The repository was created on October 4, 2018, and it has 11099 GitHub stars.
However, the collected records contain no changelog for v2.73.0. With nothing corresponding to release notes, it is not possible to tell whether this version carries any specific feature improvement or only adds to the maintenance history. The momentum score reaches just 0.7 on a 10-point scale, and the novelty, technology, and quality scores are empty. Within what the documentation alone confirms, it is a repository worth checking for setups that need to gather models from several frameworks behind one server.