The latest release of huggingface/trl, v1.16.0, has been published on GitHub. TRL (Transformers Reinforcement Learning) is a library for post-training foundation models on top of the Transformers ecosystem.
The repository description states that its goal is to train transformer language models with reinforcement learning. The README explains that it ships SFTTrainer, GRPOTrainer, DPOTrainer, KTOTrainer and a CLI together, so SFT, GRPO, DPO and KTO can be performed. SFT is supervised fine-tuning, which trains a model further on data with correct answers attached, and the remaining trainers cover preference optimization and reinforcement learning paths. It states that it can scale from a single GPU to multi-node clusters. It is a Python repository published under the Apache-2.0 license with 19561 stars.
However, the individual changes in v1.16.0, the performance benchmarks and the adoption cases are not given. The recorded momentum score is 3.3 out of 10, and the novelty, tech and quality fields are empty. Teams that already run a post-training pipeline on top of Transformers may want to look at this release, but to decide on adoption based on the gains of this version, it is better to read the release notes directly.