“fp8”

TransformerEngine

A library accelerating Transformer models on NVIDIA GPUs using low precision formats.

accelerate

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support