“data-parallelism”

ColossalAI

Making large AI models cheaper, faster and more accessible

DeepSpeed

Deep learning optimization library for efficient distributed training and inference