github.devotd.
NewLLM infrastructure

mosaicml/composer v0.32.1 — a PyTorch library that simplifies distributed training

mosaicml/composer, a PyTorch-based training library from MosaicML, folds multi-node distributed training with FSDP, DDP, and elastic sharded checkpointing into a single Trainer.

Repositorymosaicml/composer ↗

mosaicml/composer, a PyTorch-based training library from MosaicML, folds multi-node distributed training with FSDP, DDP, and elastic sharded checkpointing into a single Trainer.

GitHub description: Supercharge Your Model Training

stars
★ 5.5K
Forks
473
Language
Python
License
Apache-2.0
Latest release
v0.32.1
Read original source ↗View on GitHub ↗

mosaicml/composer is an open-source deep learning training library built on PyTorch by MosaicML. It is published on GitHub at github.com/mosaicml/composer, and its purpose is to simplify the distributed training workflow, the kind of training that is split across multiple nodes in large clusters, into a single Trainer. It is written in Python and released under the Apache-2.0 license. The latest public release is v0.32.1.

It is a library for scalable PyTorch model training and for composing training algorithms, bundling multi-node training, checkpointing, and training optimization algorithms into one Trainer. Inside the Trainer you configure the parallelism strategy, the data loader, metrics, and loggers, and it supports FSDP, DDP, and elastic sharded checkpointing. Callbacks and training algorithms let you adapt the training process to your own requirements. The repository was created on October 12, 2021, and it has 5507 GitHub stars.

What can be confirmed, however, is limited to the feature list described in the README. The v0.32.1 changelog and any figures comparing training speed or throughput are not part of the given information. As an option that brings distributed training and checkpoint management together in one place, it is worth considering, but how much actual training performance improves can only be known by measuring it directly in the target environment.