--- title: "HRM" type: "tool" slug: "sapientinc-hrm" canonical_url: "https://www.graphcanon.com/tools/sapientinc-hrm" github_url: "https://github.com/sapientinc/HRM" homepage_url: "https://sapient.inc" stars: 12587 forks: 1833 primary_language: "Python" license: "Apache-2.0" categories: ["llm-frameworks", "model-training"] tags: ["deep-learning", "reasoning", "large-language-models", "brain-inspired-ai"] updated_at: "2026-07-07T18:31:48.132331+00:00" --- # HRM > Hierarchical Reasoning Model Official Release HRM is a brain-inspired hierarchical recurrent architecture that excels in reasoning tasks with minimal data and parameters. ## Facts - Repository: https://github.com/sapientinc/HRM - Homepage: https://sapient.inc - Stars: 12,587 · Forks: 1,833 · Open issues: 76 · Watchers: 268 - Primary language: Python - License: Apache-2.0 - Last pushed: 2026-03-31T23:08:31+00:00 ## Categories - [LLM Frameworks](/categories/llm-frameworks.md) - [Model Training](/categories/model-training.md) ## Tags deep-learning, reasoning, large language models, brain-inspired-ai ## Related tools - [ollama](/tools/ollama-ollama.md) - Get up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. (★ 175,659) - [prompts.chat](/tools/f-prompts-chat.md) - The world's largest open-source prompt library for AI (★ 165,019) - [transformers](/tools/huggingface-transformers.md) - 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models (★ 162,347) - [open-webui](/tools/open-webui-open-webui.md) - User-friendly AI Interface (Supports Ollama, OpenAI API, ...) (★ 144,575) - [awesome-llm-apps](/tools/shubhamsaboo-awesome-llm-apps.md) - 100+ AI Agent & RAG apps you can actually run — clone, customize, ship. (★ 116,702) - [LLMs-from-scratch](/tools/rasbt-llms-from-scratch.md) - Implement a ChatGPT-like LLM in PyTorch from scratch (★ 98,711) - [TradingAgents](/tools/tauricresearch-tradingagents.md) - TradingAgents: Multi-Agents LLM Financial Trading Framework (★ 91,610) - [caveman](/tools/juliusbrussee-caveman.md) - Cuts 65% of tokens in AI coding agent responses. (★ 86,150) ## README (excerpt) ```text # Hierarchical Reasoning Model Reasoning, the process of devising and executing complex goal-oriented action sequences, remains a critical challenge in AI. Current large language models (LLMs) primarily employ Chain-of-Thought (CoT) techniques, which suffer from brittle task decomposition, extensive data requirements, and high latency. Inspired by the hierarchical and multi-timescale processing in the human brain, we propose the Hierarchical Reasoning Model (HRM), a novel recurrent architecture that attains significant computational depth while maintaining both training stability and efficiency. HRM executes sequential reasoning tasks in a single forward pass without explicit supervision of the intermediate process, through two interdependent recurrent modules: a high-level module responsible for slow, abstract planning, and a low-level module handling rapid, detailed computations. With only 27 million parameters, HRM achieves exceptional performance on complex reasoning tasks using only 1000 training samples. The model operates without pre-training or CoT data, yet achieves nearly perfect performance on challenging tasks including complex Sudoku puzzles and optimal path finding in large mazes. Furthermore, HRM outperforms much larger models with significantly longer context windows on the Abstraction and Reasoning Corpus (ARC), a key benchmark for measuring artificial general intelligence capabilities. These results underscore HRM’s potential as a transformative advancement toward universal computation and general-purpose reasoning systems. Read Our Paper: [https://arxiv.org/abs/2506.21734](https://arxiv.org/abs/2506.21734) **Join Our Discord Community: [https://discord.gg/sapient](https://discord.gg/sapient)** ## Quick Start Guide 🚀 ### Prerequisites ⚙️ Ensure PyTorch and CUDA are installed. The repo needs CUDA extensions to be built. If not present, run the following commands: ```bash # Install CUDA 12.6 CUDA_URL=https://developer.download.nvidia.com/compute/cuda/12.6.3/local_installers/cuda_12.6.3_560.35.05_linux.run wget -q --show-progress --progress=bar:force:noscroll -O cuda_installer.run $CUDA_URL sudo sh cuda_installer.run --silent --toolkit --override export CUDA_HOME=/usr/local/cuda-12.6 # Install PyTorch with CUDA 12.6 PYTORCH_INDEX_URL=https://download.pytorch.org/whl/cu126 pip3 install torch torchvision torchaudio --index-url $PYTORCH_INDEX_URL # Additional packages for building extensions pip3 install packaging ninja wheel setuptools setuptools-scm ``` Then install FlashAttention. For Hopper GPUs, install FlashAttention 3 ```bash git clone git@github.com:Dao-AILab/flash-attention.git cd flash-attention/hopper python setup.py install ``` For Ampere or earlier GPUs, install FlashAttention 2 ```bash pip3 install flash-attn ``` ## Install Python Dependencies 🐍 ```bash pip install -r requirements.txt ``` ## W&B Integration 📈 This project uses [Weights & Biases](https://wandb.ai/) for experiment tracking and metric visualization. Ensure you're logged in: ```bash wandb login ``` ## Run Experiments ### Quick Demo: Sudoku Solver 💻🗲 Train a master-level Sudoku AI capable of solving extremely difficult puzzles on a modern laptop GPU. 🧩 ```bash # Download and build Sudoku dataset python dataset/build_sudoku_dataset.py --output-dir data/sudoku-extreme-1k-aug-1000 --subsample-size 1000 --num-aug 1000 # Start training (single GPU, smaller batch size) OMP_NUM_THREADS=8 python pretrain.py data_path=data/sudoku-extreme-1k-aug-1000 epochs=20000 eval_interval=2000 global_batch_size=384 lr=7e-5 puzzle_emb_lr=7e-5 weight_decay=1.0 puzzle_emb_weight_decay=1.0 ``` Runtime: ~10 hours on a RTX 4070 laptop GPU ## Trained Checkpoints 🚧 - [ARC-AGI-2](https://huggingface.co/sapientinc/HRM-checkpoint-ARC-2) - [Sudoku 9x9 Extreme (1000 examples)](https://huggingface.co/sapientinc/HRM-checkpoint-sudoku-extreme) - [Maze 30x30 Hard (1000 examples)](https://huggingface.co/sapientinc/HRM-checkpoint-maze-30 ``` --- **Machine-readable endpoints** - JSON: [`/api/graphcanon/tools/sapientinc-hrm`](/api/graphcanon/tools/sapientinc-hrm) - LLM index: [/llms.txt](/llms.txt) - Full corpus: [/llms-full.txt](/llms-full.txt) _GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_