Comparison
horovod vs Megatron-LM
Verdict
Pick horovod when tags unique to horovod: baidu, deep-learning, deeplearning, keras; pick Megatron-LM when requirements: Min 32 GB RAM; Requires NVIDIA GPUs for optimized performance. Non-GPU usage is not supported or recommended.; Installation from source can be resource-intensive and may require limiting parallel compilation jobs to avoid running out of memory..
Markdown twin · horovod alternatives · Megatron-LM alternatives
GraphCanon updated today
vs
Trust & integrity
| Signal | horovod | Megatron-LM |
|---|---|---|
| Maintenance | Active (21d since push) As of today · github_public_v1 | Very active (0d since push) As of 1d · github_public_v1 |
| Provenance | Not a fork · Organization account As of today · github_public_v1 | Not a fork · Organization account As of 1d · github_public_v1 |
| Security (OSV) | No lockfile As of today · none | No lockfile As of 1d · none |
Tagline
- horovod
- Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
- Megatron-LM
- Ongoing research training transformer models at scale
Stars
- horovod
- 15k
- Megatron-LM
- 17k
Forks
- horovod
- 2.2k
- Megatron-LM
- 4.2k
Open issues
- horovod
- 406
- Megatron-LM
- 988
Language
- horovod
- Python
- Megatron-LM
- Python
Adopt for
- horovod
- -
- Megatron-LM
- Megatron-LM from NVIDIA is a research-focused tool for developing and training large-scale language models with transformer architectures, emphasizing efficient parallelism across multiple GPUs.
Persona
- horovod
- -
- Megatron-LM
- -
Runtime
- horovod
- -
- Megatron-LM
- -
License
- horovod
- Other
- Megatron-LM
- Other
Last pushed
- horovod
- Jun 20, 2026
- Megatron-LM
- Jul 11, 2026
Categories
- horovod
- Model Training
- Megatron-LM
- Model Training
Trust and health
Maintenance
- horovod
- Active (82%)
- Megatron-LM
- Very active (96%)
Days since push
- horovod
- 21d
- Megatron-LM
- 0d
Open issues (now)
- horovod
- 406
- Megatron-LM
- 988
Full report
- horovod
- Trust report
- Megatron-LM
- Trust report
Choose horovod if…
- Tags unique to horovod: baidu, deep-learning, deeplearning, keras.
- Leaner open-issue backlog (406).
When NOT to use horovod
- Model Training: Try prompting and RAG first; fine-tuning is the answer to style/format, not missing knowledge.
Choose Megatron-LM if…
- Requirements: Min 32 GB RAM; Requires NVIDIA GPUs for optimized performance. Non-GPU usage is not supported or recommended.; Installation from source can be resource-intensive and may require limiting parallel compilation jobs to avoid running out of memory..
- Tags unique to Megatron-LM: large-language-models, model-para, transformers.
- The tool is particularly beneficial when your project is GPU-centric and benefits from advanced parallelism techniques such as Tensor, Pipeline, Data, Expert, and Cluster Parallelisms (TP, PP, DP, EP,
When NOT to use Megatron-LM
- Avoid Megatron-LM if your computational setup does not include NVIDIA GPUs as it leverages GPU-specific features and parallelisms that may not be available or efficient on non-NVIDIA hardware.
- If you need portability across various hardware without depending on proprietary optimizations, other tools might better serve your needs.
Explore
Sources
Every stat on this page traces to a dated GitHub sync, license file, enrichment field, or trust scan.
- GitHub stars (horovod/horovod) · observed Jul 11, 2026
- GitHub forks (horovod/horovod) · observed Jul 11, 2026
- Last push (horovod/horovod) · observed Jun 20, 2026
- License file (Other) · observed Jul 11, 2026
- Trust scan (lockfile / OSV) · observed Jul 11, 2026
- GitHub stars (NVIDIA/Megatron-LM) · observed Jul 11, 2026
- GitHub forks (NVIDIA/Megatron-LM) · observed Jul 11, 2026
- Last push (NVIDIA/Megatron-LM) · observed Jul 11, 2026
- License file (Other) · observed Jul 11, 2026
- Decision facts (enrichment) · observed Jul 11, 2026
- Trust scan (lockfile / OSV) · observed Jul 11, 2026
GitHub stars on cards: horovod 15k · Megatron-LM 17k (synced Jul 11, 2026).
Common questions
- What is the difference between horovod and Megatron-LM?
- horovod: Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.. Megatron-LM: Ongoing research training transformer models at scale. See the comparison table for live GitHub stats and shared categories.
- When should I choose horovod over Megatron-LM?
- Choose horovod over Megatron-LM when Tags unique to horovod: baidu, deep-learning, deeplearning, keras; Leaner open-issue backlog (406).
- When should I choose Megatron-LM over horovod?
- Choose Megatron-LM over horovod when Requirements: Min 32 GB RAM; Requires NVIDIA GPUs for optimized performance. Non-GPU usage is not supported or recommended.; Installation from source can be resource-intensive and may require limiting parallel compilation jobs to avoid running out of memory.; Tags unique to Megatron-LM: large-language-models, model-para, transformers; The tool is particularly beneficial when your project is GPU-centric and benefits from advanced parallelism techniques such as Tensor, Pipeline, Data, Expert, and Cluster Parallelisms (TP, PP, DP, EP,.
- When should I avoid horovod?
- Model Training: Try prompting and RAG first; fine-tuning is the answer to style/format, not missing knowledge.
- When should I avoid Megatron-LM?
- Avoid Megatron-LM if your computational setup does not include NVIDIA GPUs as it leverages GPU-specific features and parallelisms that may not be available or efficient on non-NVIDIA hardware. If you need portability across various hardware without depending on proprietary optimizations, other tools might better serve your needs.
- Is horovod or Megatron-LM more popular on GitHub?
- Megatron-LM has more GitHub stars (17,020 vs 14,692). Stars measure visibility, not whether either tool fits your constraints.
- Are horovod and Megatron-LM open source?
- Yes - both are open-source projects on GitHub (horovod: Other, Megatron-LM: Other).
- Where can I find alternatives to horovod or Megatron-LM?
- GraphCanon lists graph-backed alternatives at horovod alternatives and Megatron-LM alternatives (horovod markdown twin, Megatron-LM markdown twin), ranked by typed relationship edges rather than popularity votes.
- Is there a machine-readable version of this comparison?
- Yes. The markdown twin at this comparison mirrors this page for agents and LLM crawlers, with the same stats table and FAQ answers.
- Which is better maintained, horovod or Megatron-LM?
- horovod: Active. Megatron-LM: Very active. Compare maintenance labels, days since push, and release cadence in the trust section below - stars alone do not measure maintenance.
- Where are the full trust reports for horovod and Megatron-LM?
- GraphCanon publishes per-repo trust reports with dated maintenance, provenance, and scan summaries: horovod trust report; Megatron-LM trust report.