{"data":{"slug":"datawhalechina-happy-llm","name":"happy-llm","tagline":"从零开始构建大模型","github_url":"https://github.com/datawhalechina/happy-llm","owner":"datawhalechina","repo":"happy-llm","owner_avatar_url":"https://avatars.githubusercontent.com/u/46047812?v=4","primary_language":"Jupyter Notebook","stars":33889,"forks":3210,"topics":["agent","llm","rag"],"archived":false,"github_pushed_at":"2026-08-08T17:18:48+00:00","maintenance_label":"Steady","stars_delta_30d":902,"url":"https://www.graphcanon.com/tools/datawhalechina-happy-llm","markdown_url":"https://www.graphcanon.com/tools/datawhalechina-happy-llm.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/datawhalechina-happy-llm","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=datawhalechina-happy-llm","description":"📚 从零开始构建大模型","homepage_url":"https://datawhalechina.github.io/happy-llm/","license":"Other","open_issues":70,"watchers":105,"ai_summary":"一个系统性的 LLM 学习教程，涵盖 NLP 基础、Transformer 架构、预训练语言模型、大语言模型的定义与训练策略，以及动手搭建大模型的全过程。","readme_excerpt":"<div align='center'>\n    <img src=\"./images/head.jpg\" alt=\"alt text\" width=\"100%\">\n    <h1>Happy-LLM</h1>\n</div>\n\n<div align=\"center\">\n  <img src=\"https://img.shields.io/github/stars/datawhalechina/happy-llm?style=flat&logo=github\" alt=\"GitHub stars\"/>\n  <img src=\"https://img.shields.io/github/forks/datawhalechina/happy-llm?style=flat&logo=github\" alt=\"GitHub forks\"/>\n  <img src=\"https://img.shields.io/badge/language-Chinese-brightgreen?style=flat\" alt=\"Language\"/>\n  <a href=\"https://github.com/datawhalechina/happy-llm\"><img src=\"https://img.shields.io/badge/GitHub-Project-blue?style=flat&logo=github\" alt=\"GitHub Project\"></a>\n  <a href=\"https://swanlab.cn/@kmno4/Happy-LLM/overview\"><img src=\"https://raw.githubusercontent.com/SwanHubX/assets/main/badge1.svg\" alt=\"SwanLab\"></a>\n</div>\n\n<div align=\"center\">\n  <a href=\"https://trendshift.io/repositories/14175\" target=\"_blank\"><img src=\"https://trendshift.io/api/badge/repositories/14175\" alt=\"datawhalechina%2Fhappy-llm | Trendshift\" style=\"width: 250px; height: 55px;\" width=\"250\" height=\"55\"/></a>\n</div>\n\n<div align=\"center\">\n\n[中文](./README.md) | [English](./README_en.md)\n\n</div>\n\n<div align=\"center\">\n  <p><a href=\"https://datawhalechina.github.io/happy-llm/\">📚 在线阅读地址</a></p>\n  <h3>📚 从零开始构建大模型</h3>\n  <p><em>深入理解 LLM 核心原理，动手实现你的第一个大模型</em></p>\n</div>\n\n---\n\n## 🎯 项目介绍\n\n> &emsp;&emsp;*很多小伙伴在看完 Datawhale开源项目： [self-llm 开源大模型食用指南](https://github.com/datawhalechina/self-llm) 后，感觉意犹未尽，想要深入了解大语言模型的原理和训练过程。于是我们（Datawhale）决定推出《Happy-LLM》项目，旨在帮助大家深入理解大语言模型的原理和训练过程。*\n\n&emsp;&emsp;本项目是一个**系统性的 LLM 学习教程**，将从 NLP 的基本研究方法出发，根据 LLM 的思路及原理逐层深入，依次为读者剖析 LLM 的架构基础和训练过程。同时，我们会结合目前 LLM 领域最主流的代码框架，演练如何亲手搭建、训练一个 LLM，期以实现授之以鱼，更授之以渔。希望大家能从这本书开始走入 LLM 的浩瀚世界，探索 LLM 的无尽可能。\n\n### ✨ 你将收获什么？\n\n- 📚 **Datawhale 开源免费** 完全免费的学习本项目所有内容\n- 🔍 **深入理解** Transformer 架构和注意力机制\n- 📚 **掌握** 预训练语言模型的基本原理\n- 🧠 **了解** 现有大模型的基本结构\n- 🏗️ **动手实现** 一个完整的 LLaMA2 模型\n- ⚙️ **掌握训练** 从预训练到微调的全流程\n- 🚀 **实战应用** RAG、Agent 等前沿技术\n\n## 📖 内容导航\n\n| 章节 | 关键内容 | 状态 |\n| --- | --- | --- |\n| [学习与环境准备](./docs/学习与环境准备.md) | 分章依赖、硬件建议与实践入口 | ✅ |\n| [前言](./docs/前言.md) | 本项目的缘起、背景及读者建议 | ✅ |\n| [第一章 NLP 基础概念](./docs/chapter1/第一章%20NLP基础概念.md) | 什么是 NLP、发展历程、任务分类、文本表示演进 | ✅ |\n| [第二章 Transformer 架构](./docs/chapter2/第二章%20Transformer架构.md) | 注意力机制、Encoder-Decoder、手把手搭建 Transformer | ✅ |\n| [第三章 预训练语言模型](./docs/chapter3/第三章%20预训练语言模型.md) | Encoder-only、Encoder-Decoder、Decoder-Only 模型对比 | ✅ |\n| [第四章 大语言模型](./docs/chapter4/第四章%20大语言模型.md) | LLM 定义、训练策略、涌现能力分析 | ✅ |\n| [第五章 动手搭建大模型](./docs/chapter5/第五章%20动手搭建大模型.md) | 实现 LLaMA2、训练 Tokenizer、预训练小型 LLM | ✅ |\n| [第六章 大模型训练实践](./docs/chapter6/第六章%20大模型训练流程实践.md) | 预训练、有监督微调、LoRA/QLoRA 高效微调 | ✅ |\n| [第七章 大模型应用](./docs/chapter7/第七章%20大模型应用.md) | 模型评测、RAG 检索增强、Agent 智能体 | ✅ |\n| [第八章 Agentic-RL ](./docs/chapter8/第八章%20大模型强化学习.md) | GRPO、OPD、Search-R1、ReTool（Coding Agent-RL） | ✅ |\n| [Extra Chapter LLM Blog](./Extra-Chapter/) | 优秀的大模型 学习笔记/Blog ，欢迎大家来 PR ！| 🚧 |\n\n> 第六章正文已覆盖 Pretrain、SFT 与 PEFT 等核心训练流程，建议结合 [第六章实践说明](./docs/chapter6/readme.md) 和 [学习与环境准备](./docs/学习与环境准备.md) 一起阅读。\n\n> 第八章聚焦 GRPO、OPD、Search-R1 与 ReTool。更多 Agentic RL 算法、训练代码与实验实践，可以前往作者持续维护的另一个仓库 [agentic-rl-lab](https://github.com/KMnO4-zx/agentic-rl-lab)；该仓库更新频率更高，会持续跟进新的算法与环境。\n\n### Extra Chapter LLM Blog\n\n- [大模型都这么厉害了，微调0.6B的小模型有什么意义？](./Extra-Chapter/why-fine-tune-small-large-language-models/readme.md) @[不要葱姜蒜](https://github.com/KMnO4-zx) 2025-7-11\n\n- [Transformer 整体模块设计解读](./Extra-Chapter/transformer-architecture/) @[ditingdapeng](https://github.com/ditingdapeng) 2025-7-14\n\n- [文本数据处理详解](./Extra-Chapter/text-data-processing/readme.md) @[蔡鋆捷](https://github.com/xinala-781) 2025-7-14\n\n- [Qwen3-\"VL\"——超小中文多模态模型的“拼接微调”之路](./Extra-Chapter/vlm-concatenation-finetune/README.md) @[ShaohonChen](https://github.com/ShaohonChen) 2025-7-30\n\n- [S1: Thinking Budget with vLLM](./Extra-Chapter/s1-vllm-thinking-budget/readme.md) @[不要葱姜蒜](https://github.com/kmno4-zx) 2025-8-03\n\n\n- [CDDRS: 使用细粒度语义信息指导增强的RAG检索方法](./Extra-Chapter/CDDRS/readme.md) @[Hongru0306](https://github.com/Hongru0","github_created_at":"2024-05-28T03:22:50+00:00","created_at":"2026-07-07T17:31:55.550349+00:00","updated_at":"2026-09-20T04:32:44.991876+00:00","categories":[{"slug":"developer-tools","name":"Developer Tools","url":"https://www.graphcanon.com/categories/developer-tools","markdown_url":"https://www.graphcanon.com/categories/developer-tools.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/developer-tools"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"agent","name":"agent"},{"slug":"llm","name":"llm"},{"slug":"rag","name":"rag"}],"trust":{"provenance":{"is_fork":false,"github_id":806854629,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-18T08:42:06.114Z","maintenance":{"label":"Steady","score":60,"methodology":"github_public_v1","releases_90d":0,"days_since_push":40,"last_release_at":"2026-01-29T06:35:12Z","stars_delta_30d":902,"open_issues_delta_30d":6},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-09-18T08:42:08.020Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-18T08:42:04.783Z"},"languages":{"value":["jupyter notebook"],"source":"github.language","observed_at":"2026-09-18T08:42:04.783Z"},"license_spdx":{"value":"Other","source":"github.license","observed_at":"2026-09-18T08:42:04.783Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["Hardware recommendations and setup instructions are provided in the '学习与环境准备' section."],"min_ram_gb":8,"requires_docker":false},"constraints":{"min_ram_gb":8,"requires_docker":false},"when_to_use":["When you need a detailed guide to understand the principles and training process of large language models, especially if you are interested in hands-on implementation of LLaMA2.","If you are looking for a resource that covers the entire process from NLP basics to advanced topics like RAG and Agent technologies.","For those who prefer learning materials in Chinese, as the content is primarily available in this language."],"when_not_to_use":["If you are seeking a tool for immediate deployment of large language models without the need for understanding the underlying principles.","When you require a resource that focuses solely on the latest research papers and cutting-edge algorithms without a systematic learning path.","If you prefer English-language resources, as the primary content is in Chinese, although an English version is available."],"source":"enrich:decision_facts","observed_at":"2026-09-18T12:08:10.028Z"},"constraint_facets":{"min_ram_gb":8,"requires_docker":false},"decision_summary":[{"label":"Requirements","value":"Min 8 GB RAM; Hardware recommendations and setup instructions are provided in the '学习与环境准备' section."},{"label":"Adopt for","value":"Happy-LLM is a comprehensive tutorial for learning about large language models, focusing on NLP basics, Transformer architecture, and practical implementation of LLaMA2."},{"label":"License detail","value":"The license is not specified as open source, but the content is free to use and learn from."}]}}