{"data":{"slug":"nvidia-megatron-lm","name":"Megatron-LM","tagline":"Ongoing research training transformer models at scale","github_url":"https://github.com/NVIDIA/Megatron-LM","owner":"NVIDIA","repo":"Megatron-LM","owner_avatar_url":"https://avatars.githubusercontent.com/u/1728152?v=4","primary_language":"Python","stars":17938,"forks":4530,"topics":["large-language-models","model-para","transformers"],"archived":false,"github_pushed_at":"2026-09-18T08:33:53+00:00","maintenance_label":"Very active","stars_delta_30d":597,"url":"https://www.graphcanon.com/tools/nvidia-megatron-lm","markdown_url":"https://www.graphcanon.com/tools/nvidia-megatron-lm.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/nvidia-megatron-lm","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=nvidia-megatron-lm","description":"Ongoing research training transformer models at scale","homepage_url":"https://docs.nvidia.com/megatron-core/developer-guide/latest/get-started/quickstart.html","license":"Other","open_issues":1368,"watchers":172,"ai_summary":"Megatron-LM is a research project by NVIDIA for training large transformer models at scale, focusing on parallelism strategies across multiple GPUs.","readme_excerpt":"## Getting Started\n\n**Install from PyPI:**\n\n```bash\nuv pip install megatron-core\n```\n\n**Or clone and install from source:**\n\n```bash\ngit clone https://github.com/NVIDIA/Megatron-LM.git\ncd Megatron-LM\nuv pip install -e .\n```\n\n> **Note:** Building from source can use a lot of memory. If the build runs out of memory, limit parallel compilation jobs by setting `MAX_JOBS` (for example, `MAX_JOBS=4 uv pip install -e .`).\n\nFor NVIDIA GPU Cloud (NGC) container setup and all installation options, review the **[Installation Guide](https://docs.nvidia.com/megatron-core/developer-guide/latest/get-started/install.html)**.\n\n- **[Your First Training Run](https://docs.nvidia.com/megatron-core/developer-guide/latest/get-started/quickstart.html)** - End-to-end training examples with data preparation\n- **[Parallelism Strategies](https://docs.nvidia.com/megatron-core/developer-guide/latest/user-guide/parallelism-guide.html)** - Scale training across GPUs with TP, PP, DP, EP, and CP\n- **[Contribution Guide](https://docs.nvidia.com/megatron-core/developer-guide/latest/developer/contribute.html)** - How to contribute to Megatron Core\n- **[Style Guide](style-guide.md)** - Python style overrides","github_created_at":"2019-03-21T16:15:52+00:00","created_at":"2026-07-07T17:33:33.267662+00:00","updated_at":"2026-09-20T04:33:26.547311+00:00","categories":[{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"gpu","name":"gpu"},{"slug":"large-language-models","name":"large-language-models"},{"slug":"model-para","name":"model-para"},{"slug":"nvidia","name":"nvidia"},{"slug":"parallelism","name":"parallelism"},{"slug":"scale","name":"scale"},{"slug":"training","name":"training"},{"slug":"transformers","name":"transformers"}],"trust":{"provenance":{"is_fork":false,"github_id":176982014,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-18T08:47:49.269Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":3,"days_since_push":0,"last_release_at":"2026-08-19T19:02:15Z","stars_delta_30d":597,"open_issues_delta_30d":256},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-09-18T08:47:51.688Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-18T08:47:48.177Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-09-18T08:47:48.177Z"},"license_spdx":{"value":"Other","source":"github.license","observed_at":"2026-09-18T08:47:48.177Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["Building from source can be memory-intensive, and users may need to limit parallel compilation jobs by setting MAX_JOBS to manage memory usage."]},"constraints":null,"when_to_use":["Use Megatron-LM when you need to train large transformer models and have access to multiple GPUs, as it excels in parallelism strategies like tensor parallelism (TP), pipeline parallelism (PP), data-2",">// The text was cut off, but based on the context, it seems to suggest using Megatron-LM for its advanced parallelism strategies which are particularly beneficial for large-scale model training on a多","If your project involves research in scaling transformer models and you are working with NVIDIA GPUs, Megatron-LM provides specialized tools and strategies to optimize training."],"when_not_to_use":["Avoid using Megatron-LM if your project does not require large-scale model training or if you do not have access to multiple GPUs, as its benefits are most pronounced in these scenarios.","Do not use Megatron-LM if your team lacks the expertise to handle complex parallelism strategies or if your infrastructure cannot support the memory and computational demands of building from source."],"source":"enrich:decision_facts","observed_at":"2026-09-18T10:17:25.083Z"},"constraint_facets":null,"decision_summary":[{"label":"Requirements","value":"Building from source can be memory-intensive, and users may need to limit parallel compilation jobs by setting MAX_JOBS to manage memory usage."},{"label":"Adopt for","value":"Megatron-LM is a research project by NVIDIA for training large transformer models at scale, emphasizing parallelism strategies across multiple GPUs."},{"label":"License detail","value":"The license for Megatron-LM is categorized as 'Other', indicating that it may have specific terms and conditions that differ from standard open-source licenses."}]}}