--- title: "reasoning-from-scratch" type: "tool" slug: "rasbt-reasoning-from-scratch" canonical_url: "https://www.graphcanon.com/tools/rasbt-reasoning-from-scratch" github_url: "https://github.com/rasbt/reasoning-from-scratch" homepage_url: "https://mng.bz/lZ5B" stars: 4693 forks: 699 primary_language: "Jupyter Notebook" license: "Apache-2.0" categories: ["llm-frameworks"] tags: ["inference-time-scaling", "deep-learning", "chain-of-thought", "ai", "artificial-intelligence", "distillation", "large-language-models", "grpo"] updated_at: "2026-07-07T18:40:39.680074+00:00" --- # reasoning-from-scratch > Implement a reasoning LLM in PyTorch from scratch, step by step This repository contains the source code and detailed steps for developing a large language model (LLM) with reasoning capabilities using PyTorch. It's designed as an educational tool to help learners understand how reasoning models function. ## Facts - Repository: https://github.com/rasbt/reasoning-from-scratch - Homepage: https://mng.bz/lZ5B - Stars: 4,693 · Forks: 699 · Open issues: 2 · Watchers: 61 - Primary language: Jupyter Notebook - License: Apache-2.0 - Last pushed: 2026-07-06T23:18:59+00:00 ## Categories - [LLM Frameworks](/categories/llm-frameworks.md) ## Tags inference-time-scaling, deep-learning, chain-of-thought, ai, artificial-intelligence, distillation, large language models, grpo ## Related tools - [ollama](/tools/ollama-ollama.md) - Get up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. (★ 175,659) - [prompts.chat](/tools/f-prompts-chat.md) - The world's largest open-source prompt library for AI (★ 165,019) - [transformers](/tools/huggingface-transformers.md) - 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models (★ 162,347) - [open-webui](/tools/open-webui-open-webui.md) - User-friendly AI Interface (Supports Ollama, OpenAI API, ...) (★ 144,575) - [awesome-llm-apps](/tools/shubhamsaboo-awesome-llm-apps.md) - 100+ AI Agent & RAG apps you can actually run — clone, customize, ship. (★ 116,702) - [LLMs-from-scratch](/tools/rasbt-llms-from-scratch.md) - Implement a ChatGPT-like LLM in PyTorch from scratch (★ 98,711) - [TradingAgents](/tools/tauricresearch-tradingagents.md) - TradingAgents: Multi-Agents LLM Financial Trading Framework (★ 91,610) - [caveman](/tools/juliusbrussee-caveman.md) - Cuts 65% of tokens in AI coding agent responses. (★ 86,150) ## README (excerpt) ```text # Build A Reasoning Model (From Scratch) This repository contains the code for developing an LLM reasoning model and is the official code repository for the book [*Build a Reasoning Model (From Scratch)*](https://mng.bz/lZ5B).

(Printed in color.)
In [*Build a Reasoning Model (From Scratch)*](https://mng.bz/lZ5B), you will learn and understand how a reasoning large language model (LLM) works. Reasoning is one of the most exciting and important recent advances in improving LLMs, but it’s also one of the easiest to misunderstand if you only hear the term reasoning and read about it in theory. This is why this book takes a hands-on approach. We will start with a pre-trained base LLM and then add reasoning capabilities ourselves, step by step in code, so you can see exactly how it works. The methods described in this book walk you through the process of developing your own small-but-functional reasoning model for educational purposes. It mirrors the approaches used in creating large-scale reasoning models such as DeepSeek R1, GPT-5 Thinking, and others. In addition, this book includes code for loading the weights of existing, pretrained models. - Link to the official [source code repository](https://github.com/rasbt/reasoning-from-scratch) - Link to the [book at Manning](https://mng.bz/lZ5B) (the publisher's website) - Link to the book page on Amazon.com (TBD) - ISBN 9781633434677

To download a copy of this repository, click on the [Download ZIP](https://github.com/rasbt/reasoning-from-scratch/archive/refs/heads/main.zip) button or execute the following command in your terminal: ```bash git clone --depth 1 https://github.com/rasbt/reasoning-from-scratch.git ```
> **Tip:** > Chapter 2 provides additional tips on installing Python, managing Python packages, and setting up your coding environment.

## Table of Contents (In Progress) - [Troubleshooting Guide](./troubleshooting.md) | Chapter Title | Main Code | | ----------------------------------------------------------- | ------------------------------------------------------------ | | Ch 1: Understanding reasoning Models | No code | | Ch 2: Generating Text with a Pre-trained LLM | - [ch02_main.ipynb](ch02/01_main-chapter-code/ch02_main.ipynb)
- [ch02_exercise-solutions.ipynb](ch02/01_main-chapter-code/ch02_exercise-solutions.ipynb) | | Ch 3: Evaluating Reasoning Models | - [ch03_main.ipynb](ch03/01_main-chapter-code/ch03_main.ipynb)
- [ch03_exercise-solutions.ipynb](ch03/01_main-chapter-code/ch03_exercise-solutions.ipynb) | | Ch 4: Improving Reasoning with Inference-Time Scaling | - [ch04_main.ipynb](ch04/01_main-chapter-code/ch04_main.ipynb)
- [ch04_exercise-solutions.ipynb](ch04/01_main-chapter-code/ch04_exercise-solutions.ipynb) | | Ch 5: Inference-Time Scaling via Self-Refinement | - [ch05_main.ipynb](ch05/01_main-chapter-code/ch05_main.ipynb)
- [ch05_exercise-solutions.ipynb](ch05/01_main-chapter-code/ch05_exercise-solutions.ipynb) | | Ch 6: Training Reasoning Models with Reinforcement Learning | - [ch06_main.ipynb](ch06/01_main-chapter-code/ch06_main.ipynb)
- [ch06_exercise-solutions.ipynb](ch06/01_main-chapter-code/ch06_exercise-solutions.ipynb) | | Ch 7: Improving GRPO for Reinforcement Learning | - [ch07_main.ipynb](ch07/01_main-chapter-code/ch07_main.ipynb)
- [ch07_exercise-solutions.ipynb](ch07/01_main-chapter-code/ch07_exercise-solutions.ipynb) | | Ch 8: Distilling Reasoning Models for Efficient Reasoning | - [ch08_main.ipynb](ch08/01_main-chapter-code/ch08_main.ipynb)
- [ch08_exercise-solutions.ipynb] ``` --- **Machine-readable endpoints** - JSON: [`/api/graphcanon/tools/rasbt-reasoning-from-scratch`](/api/graphcanon/tools/rasbt-reasoning-from-scratch) - LLM index: [/llms.txt](/llms.txt) - Full corpus: [/llms-full.txt](/llms-full.txt) _GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_