Devotd Github
NewInference & serving

nvidia/tensorrt v11.4 — Public Components of TensorRT SDK

nvidia/tensorrt holds the open-source components of TensorRT, the deep learning inference SDK for NVIDIA GPUs, and the latest release is v11.4.

Repositorynvidia/tensorrt ↗

nvidia/tensorrt holds the open-source components of TensorRT, the deep learning inference SDK for NVIDIA GPUs, and the latest release is v11.4.

GitHub description: NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.

stars
★ 13.4K
Forks
2.4K
Language
C++
License
Apache-2.0
Latest release
v11.4
Read original source ↗View on GitHub ↗

nvidia/tensorrt holds the open-source components of TensorRT, the SDK for running deep learning inference at high performance on NVIDIA GPUs. The latest release is v11.4. What is public includes the source of the TensorRT plugins and the ONNX parser (the component that reads ONNX model files), plus sample applications that show platform usage and features. The README states that this OSS component is part of the TensorRT GA (general availability) release and includes some extensions and bug fixes. What is open is this component, not the whole SDK.

Even the public part alone shows what the structure of a CUDA GPU-specialized inference engine configuration looks like. The engine leads into model optimization, plugins and the runtime path, the implementation language is C++, and the license is Apache-2.0. The repository was created on May 2, 2019 and has 13,409 stars. For those who work directly with inference pipelines on NVIDIA GPUs, the plugin and parser sources remain in code, so it is a good starting point to look.

However, there is no material that breaks down item by item what changed in v11.4. The README only describes the public scope and the included extensions and bug fixes, and it gives no benchmark or measured figures to compare performance. There is also no score compared against other repositories, so how much this version actually moved real inference performance is hard to judge from this repository alone. It is worth reading as reference material for those already working with TensorRT, but anyone looking for something new to check should know first that the changelog cannot be seen separately.