nvidia/model-optimizer

nvidia/model-optimizer 是一个 Python 库，通过多种模型优化技术生成可部署到推理框架的检查点。

GitHub 项目说明: A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.