{"data":{"slug":"nvidia-transformerengine","name":"TransformerEngine","tagline":"A library for accelerating Transformer models on NVIDIA GPUs using low precision formats like FP8 and FP4.","github_url":"https://github.com/NVIDIA/TransformerEngine","owner":"NVIDIA","repo":"TransformerEngine","owner_avatar_url":"https://avatars.githubusercontent.com/u/1728152?v=4","primary_language":"Python","stars":3524,"forks":819,"topics":["cuda","deep-learning","fp4","fp8","gpu","jax","machine-learning","python","pytorch"],"archived":false,"github_pushed_at":"2026-09-05T13:13:46+00:00","maintenance_label":"Very active","stars_delta_30d":45,"url":"https://www.graphcanon.com/tools/nvidia-transformerengine","markdown_url":"https://www.graphcanon.com/tools/nvidia-transformerengine.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/nvidia-transformerengine","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=nvidia-transformerengine","description":"A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.","homepage_url":"https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/index.html","license":"Apache-2.0","open_issues":343,"watchers":36,"ai_summary":"TransformerEngine is a Python-based library by NVIDIA, focused on enhancing the performance of Transformer models through low-precision computation techniques (FP8 and FP4) on compatible GPUs. It supports frameworks such as PyTorch and JAX, aiming for better training and inference throughput with reduced memory footprint.","readme_excerpt":"..\n    Copyright (c) 2022-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.\n\n    See LICENSE for license information.\n\n|License|\n\nTransformer Engine\n==================\n\n`Quickstart <#examples>`_ | `Installation <#installation>`_ | `User Guide <https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/index.html>`_ | `Examples <https://github.com/NVIDIA/TransformerEngine/tree/main/examples>`_ | `Convergence <#convergence>`_ | `Integrations <#integrations>`_ | `Release notes <https://docs.nvidia.com/deeplearning/transformer-engine/documentation-archive.html>`_\n\nLatest News\n===========\n\n* [06/2026] `Boosting MoE Training Throughput with Advanced Fusion Kernels <https://developer.nvidia.com/blog/boosting-moe-training-throughput-with-advanced-fusion-kernels/>`_\n* [06/2026] `Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning <https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Ultra-Technical-Report.pdf>`_\n* [06/2026] `Train Models Faster with JAX and MaxText Using NVFP4 on NVIDIA Blackwell <https://developer.nvidia.com/blog/train-models-faster-with-jax-and-maxtext-using-nvfp4-on-nvidia-blackwell/>`_\n* [04/2026] `Run High-Throughput Reinforcement Learning Training with End-to-End FP8 Precision <https://developer.nvidia.com/blog/run-high-throughput-reinforcement-learning-training-with-end-to-end-fp8-precision/>`_\n* [02/2026] `Using NVFP4 Low-Precision Model Training for Higher Throughput Without Losing Accuracy <https://developer.nvidia.com/blog/using-nvfp4-low-precision-model-training-for-higher-throughput-without-losing-accuracy/>`_\n* [12/2025] `NVIDIA Nemotron 3: Efficient and Open Intelligence <https://arxiv.org/abs/2512.20856>`_ - trained with NVFP4 on Transformer Engine\n* [11/2025] `NVIDIA Blackwell Architecture Sweeps MLPerf Training v5.1 Benchmarks <https://developer.nvidia.com/blog/nvidia-blackwell-architecture-sweeps-mlperf-training-v5-1-benchmarks/>`_\n* [11/2025] `Scale Biology Transformer Models with PyTorch and NVIDIA BioNeMo Recipes <https://developer.nvidia.com/blog/scale-biology-transformer-models-with-pytorch-and-nvidia-bionemo-recipes/>`_\n* [11/2025] `FP8 Training of Large-Scale RL Models <https://lmsys.org/blog/2025-11-25-fp8-rl/>`_\n* [09/2025] `Pretraining Large Language Models with NVFP4 <https://www.arxiv.org/pdf/2509.25149>`_\n* [09/2025] `Native FP8 Mixed Precision Training for Ling 2.0, Open Sourced! <https://huggingface.co/blog/im0qianqian/ling-mini-2-fp8-mixed-precision-training-solution>`_\n* [09/2025] `Faster Training Throughput in FP8 Precision with NVIDIA NeMo <https://developer.nvidia.com/blog/faster-training-throughput-in-fp8-precision-with-nvidia-nemo/>`_\n* [08/2025] `How we built DeepL's next-generation LLMs with FP8 for training and inference <https://www.deepl.com/en/blog/tech/next-generation-llm-fp8-training>`_\n* [08/2025] `NVFP4 Trains with Precision of 16-bit and Speed and Efficiency of 4-bit <https://developer.nvidia.com/blog/nvfp4-trains-with-precision-of-16-bit-and-speed-and-efficiency-of-4-bit/>`_\n\n`Previous News <#previous-news>`_\n\nWhat is Transformer Engine?\n===========================\n.. overview-begin-marker-do-not-remove\n\nTransformer Engine (TE) is a library for accelerating Transformer models on NVIDIA GPUs, including\nusing 8-bit floating point (FP8) precision on Hopper, Ada, and Blackwell GPUs, to provide better\nperformance with lower memory utilization in both training and inference. On Blackwell GPUs, TE also\nsupports MXFP8 (Microscaling FP8) and NVFP4 formats for even greater efficiency. TE provides a collection\nof highly optimized building blocks for popular Transformer architectures and an automatic mixed\nprecision-like API that can be used seamlessly with your framework-specific code. TE also includes a\nframework agnostic C++ API that can be integrated with other deep learning libraries to enable FP8\nsupport for Transformers.\n\nAs Transformer models scale to hundreds of billions of parameters a","github_created_at":"2022-09-20T15:20:26+00:00","created_at":"2026-07-11T10:36:30.002767+00:00","updated_at":"2026-09-20T04:23:22.622318+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"cuda","name":"cuda"},{"slug":"deep-learning","name":"deep-learning"},{"slug":"fp4","name":"fp4"},{"slug":"fp8","name":"fp8"},{"slug":"gpu","name":"gpu"},{"slug":"jax","name":"jax"},{"slug":"machine-learning","name":"machine-learning"},{"slug":"pytorch","name":"pytorch"}],"trust":{"provenance":{"is_fork":false,"github_id":539057023,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-07T00:00:53.884Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":5,"days_since_push":1,"last_release_at":"2026-08-14T20:27:00Z","stars_delta_30d":45,"open_issues_delta_30d":33},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T10:36:31.347Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-07T00:00:54.364Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-09-07T00:00:54.364Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-07T00:00:54.364Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["If you need high-throughput training or inference of Transformer models specifically on compatible NVIDIA GPUs (Hopper, Ada, Blackwell).","For projects requiring low-precision computation techniques without significant loss in accuracy."],"when_not_to_use":["Avoid if your project is not running on NVIDIA's Hopper, Ada, or Blackwell GPUs.","If memory usage isn't a critical concern and you prefer higher precision over speed optimization."],"source":"enrich:decision_facts","observed_at":"2026-07-12T12:27:52.029Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"TransformerEngine optimizes Transformer model performance with FP8/FP4 precision on NVIDIA GPUs like Hopper, Ada, and Blackwell, boosting throughput and reducing memory usage."}]}}