Triton Inference Server is an open-source inference server that bundles models from several machine learning frameworks, such as TensorRT, PyTorch, and ONNX, behind one server and takes inference requests over HTTP/REST and gRPC, and the repository's latest release is v2.73.0.
Triton Inference Server is an open-source inference server that bundles models from several machine learning frameworks, such as TensorRT, PyTorch, and ONNX, behind one server and takes inference requests over HTTP/REST and gRPC, and the repository's latest release is v2.73.0.