GraphCanon updated today · GitHub synced today · 26 views this month
Decision brief
Scalable toolkit for data pre-processing tailored to LLMs, featuring deduplication and quality checks.
Good fit when
- You're working with NVIDIA NeMo models and require seamless integration.
- Your project demands semantic-based deduplication techniques.
Avoid when
- Your dataset doesn't align with NVIDIA hardware specifications.
- You prefer data curation tools that do not emphasize semantic processing.
Observed Jul 12, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (2d since push)
- As of today
- Provenance
- Not a fork · Organization account
- As of today
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install CuratorSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
NVIDIA-NeMo/Curator is a Python-based repository offering tools for scalable data processing and curation specifically designed for Large Language Models (LLMs). It includes functionalities such as deduplication, quality checks, and preparation pipelines which are crucial steps before the training phase of AI models.
Capability facts
- Languages
- python
Source: github.language+pyproject.toml · Aug 24, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Aug 24, 2026)
Supported Python and dependency versions are defined in [`pyproject.toml`](pyproject.toml); the PSource link
Tags
README
Quick Start
Three paths, depending on what you're trying to do. Each path is self-contained.
NeMo Curator uses uv for installation. Install it once:
curl -LsSf https://astral.sh/uv/install.sh | sh
Path C — Docker (recommended for video and audio)
Video and audio pipelines depend on system codec libraries; the published container ships them preconfigured.
- Container: nemo-curator on NGC
- Setup instructions: Installation Guide
Full setup for all paths: Installation Guide • Tutorials
Installation and Deployment
| Resource | Link |
|---|---|
| Installation guide (CPU, GPU, Docker, source) | docs.nvidia.com/nemo/curator/latest/get-started/installation |
| Container image | nemo-curator on NGC |
| Infrastructure (Slurm, Kubernetes, multi-node) | Infrastructure docs |
| API reference | API docs |
| Concepts | Concepts |
Supported Python and dependency versions are defined in pyproject.toml; the Python versions are also shown on the PyPI badge above. The README does not duplicate them to avoid drift.
License
NeMo Curator is released under the Apache License 2.0. See LICENSE for the full text.
For agents
This page has a .md twin and JSON over the API.