lightning-ai/litgpt is a Python project that provides recipes for large-scale pretraining, finetuning, and deployment of 20+ high-performance LLMs. It does not stop at training and covers inference execution as well, and the latest release is v0.5.13. What changed in this release, however, is not included in the given record.
Pretraining is the stage where a model is given general knowledge, and finetuning is the stage of continued training for a specific task, and this repository brings both stages and deployment together in one place. Each model is implemented separately from scratch without a common abstraction layer, and it provides the run commands and the training workflow (the sequence of steps) for each model. Training paths such as full, LoRA, and QLoRA are laid out so that a finetuning method can be chosen, and Flash attention, FSDP, Adapter, and quantization are supported as well. It covers fast inference and running on GPUs with little memory, and the license is Apache-2.0, so it can be used in enterprises too. It has 13720 stars.
However, there are no performance figures or benchmarks in the record, so it is not possible to confirm what level of quality and processing speed it delivers. The repository was created on May 4, 2023, and of the evaluation scores only momentum is given, 1.6 out of 10, while the freshness, technology, and quality scores are not given. Rather than adopting it right now, it is more realistic to keep it as a reference for training pipeline recipes.