{"data":{"slug":"rasbt-llms-from-scratch","name":"LLMs-from-scratch","tagline":"Implement a ChatGPT-like LLM in PyTorch from scratch, step by step","github_url":"https://github.com/rasbt/LLMs-from-scratch","owner":"rasbt","repo":"LLMs-from-scratch","owner_avatar_url":"https://avatars.githubusercontent.com/u/5618407?v=4","primary_language":"Jupyter Notebook","stars":102733,"forks":15748,"topics":["ai","artificial-intelligence","attention-mechanism","deep-learning","finetuning","from-scratch","generative-ai","gpt","instruction-tuning","language-model","large-language-models","llm","machine-learning","natural-language-processing","pretraining","python","pytorch","tokenizer","transformers"],"archived":false,"github_pushed_at":"2026-08-10T01:11:40+00:00","maintenance_label":"Very active","stars_delta_30d":3541,"url":"https://www.graphcanon.com/tools/rasbt-llms-from-scratch","markdown_url":"https://www.graphcanon.com/tools/rasbt-llms-from-scratch.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/rasbt-llms-from-scratch","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=rasbt-llms-from-scratch","description":"Implement a ChatGPT-like LLM in PyTorch from scratch, step by step","homepage_url":"https://amzn.to/4fqvn0D","license":"Other","open_issues":2,"watchers":838,"ai_summary":"A repository focused on building a ChatGPT-like language model using PyTorch with detailed steps.","readme_excerpt":"## Hardware Requirements\n\nThe code in the main chapters of this book is designed to run on conventional laptops within a reasonable timeframe and does not require specialized hardware. This approach ensures that a wide audience can engage with the material. Additionally, the code automatically utilizes GPUs if they are available. (Please see the [setup](https://github.com/rasbt/LLMs-from-scratch/blob/main/setup/README.md) doc for additional recommendations.)\n\n\n ","github_created_at":"2023-07-23T18:15:57+00:00","created_at":"2026-07-07T17:30:31.144588+00:00","updated_at":"2026-08-16T00:01:21.960528+00:00","categories":[{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"ai","name":"ai"},{"slug":"artificial-intelligence","name":"artificial-intelligence"},{"slug":"attention-mechanism","name":"attention-mechanism"},{"slug":"deep-learning","name":"deep-learning"},{"slug":"finetuning","name":"finetuning"},{"slug":"from-scratch","name":"from-scratch"},{"slug":"generative-ai","name":"generative-ai"},{"slug":"gpt","name":"gpt"}],"trust":{"provenance":{"is_fork":false,"github_id":669879380,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-16T00:01:21.284Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":0,"days_since_push":5,"last_release_at":null,"stars_delta_30d":3541,"open_issues_delta_30d":-1},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T10:55:56.330Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-16T00:01:21.704Z"},"languages":{"value":["jupyter notebook","python"],"source":"github.language+pyproject.toml","observed_at":"2026-08-16T00:01:21.704Z"},"license_spdx":{"value":"Other","source":"github.license","observed_at":"2026-08-16T00:01:21.704Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["- You are an advanced practitioner aiming to fully understand the underpinnings of LLMs using PyTorch as your primary framework.","- Your goal is not just to deploy a model but also to deeply comprehend each component and tweak it accordingly, allowing for high customization.","- You need educational materials or tutorials that walk through the entire process from pre-training to fine-tuning stages to educate others in advanced AI concepts."],"when_not_to_use":["- If you are looking for a rapid deployment of an LLM without understanding its intricate structure - this tool requires extensive manual and conceptual work.","- You prefer frameworks with automatic model generation or other high-level abstractions that simplify the process. This repository emphasizes manual creation, which is more time-consuming but offers丰","a deeper learning experience."],"source":"enrich:decision_facts","observed_at":"2026-07-11T11:51:24.273Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"LLMs-from-scratch is a project-oriented repository aimed at building PyTorch-based language models from the ground up, with detailed step-by-step instructions."}]}}