MosaicML Foundry

GitHub - mosaicml/llm-foundry: LLM training code for MosaicML foundation models

• A library for multi-GPU training.
• Built on Pytorch’s FSDP - Fully Sharded Data Parallel.