---
title: "ml-engineering"
type: "tool"
slug: "stas00-ml-engineering"
canonical_url: "https://www.graphcanon.com/tools/stas00-ml-engineering"
github_url: "https://github.com/stas00/ml-engineering"
homepage_url: "https://stasosphere.com/machine-learning/"
stars: 18259
forks: 1158
primary_language: "Python"
license: "CC-BY-SA-4.0"
categories: ["model-training", "inference-serving", "developer-tools"]
tags: ["llm", "ai", "machine-learning", "debugging", "large-language-models", "gpus", "mlops", "inference"]
updated_at: "2026-07-07T18:27:46.733757+00:00"
---

# ml-engineering

> Machine Learning Engineering Open Book

An open-source repository containing methodologies, tools, and step-by-step instructions for training, fine-tuning, and inference of large language models (LLMs) and multimodal models. It includes technical details on hardware setup, orchestration systems like SLURM, and guidance on cloud resource management.

## Facts

- Repository: https://github.com/stas00/ml-engineering
- Homepage: https://stasosphere.com/machine-learning/
- Stars: 18,259 · Forks: 1,158 · Open issues: 2 · Watchers: 148
- Primary language: Python
- License: CC-BY-SA-4.0
- Last pushed: 2026-07-01T23:28:52+00:00

## Categories

- [Model Training](/categories/model-training.md)
- [Inference & Serving](/categories/inference-serving.md)
- [Developer Tools](/categories/developer-tools.md)

## Tags

llm, ai, machine-learning, debugging, large language models, gpus, mlops, inference

## Related tools

- [ECC](/tools/affaan-m-ecc.md) - The agent harness performance optimization system (★ 226,962)
- [AutoGPT](/tools/significant-gravitas-autogpt.md) - AutoGPT: Build, Deploy, and Run AI Agents (★ 185,417)
- [prompts.chat](/tools/f-prompts-chat.md) - The world's largest open-source prompt library for AI (★ 165,019)
- [transformers](/tools/huggingface-transformers.md) - 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models (★ 162,347)
- [JavaGuide](/tools/snailclimb-javaguide.md) - Snailclimb/JavaGuide: 面试 & 后端通用面试指南，覆盖计算机基础、数据库、分布式、高并发、系统设计与 AI 应用开发 (★ 156,863)
- [open-webui](/tools/open-webui-open-webui.md) - User-friendly AI Interface (Supports Ollama, OpenAI API, ...) (★ 144,575)
- [browser-use](/tools/browser-use-browser-use.md) - 🌐 Make websites accessible for AI agents. Automate tasks online with ease. (★ 103,315)
- [LLMs-from-scratch](/tools/rasbt-llms-from-scratch.md) - Implement a ChatGPT-like LLM in PyTorch from scratch (★ 98,711)

## README (excerpt)

```text
# Machine Learning Engineering Open Book

This is an open collection of methodologies, tools and step by step instructions to help with successful training and fine-tuning of large language models and multi-modal models and their inference.

This is a technical material suitable for LLM/VLM training engineers and operators. That is the content here contains lots of scripts and copy-n-paste commands to enable you to quickly address your needs.

This repo is an ongoing brain dump of my experiences training Large Language Models (LLM) (and VLMs); a lot of the know-how I acquired while training the open-source [BLOOM-176B](https://huggingface.co/bigscience/bloom) model in 2022 and [IDEFICS-80B](https://huggingface.co/HuggingFaceM4/idefics-80b-instruct) multi-modal model in 2023, and RAG models at [Contextual.AI](https://contextual.ai/) in 2024.

I've been compiling this information mostly for myself so that I could quickly find solutions I have already researched in the past and which have worked, but as usual I'm happy to share these notes with the wider ML community.


## Table of Contents


**Part 1. Insights**

1. **[The AI Battlefield Engineering](./insights/ai-battlefield.md)** - what you need to know in order to succeed.

1. **[How to Choose a Cloud Provider](./insights/how-to-choose-cloud-provider.md)** - these questions will empower you to have a successful compute cloud experience.

**Part 2. Hardware**

1. **[Compute](compute)** - accelerators, CPUs, CPU memory.

1. **[Storage](storage)** - local, distributed and shared file systems.

1. **[Network](network)** - intra- and inter-node networking.


**Part 3. Orchestration**

1. **[Orchestration Systems](orchestration)** - managing containers and resources
1. **[SLURM](orchestration/slurm)** - Simple Linux Utility for Resource Management


**Part 4. Training**

1. **[Training](training)** - model training-related guides


**Part 5. Inference**

1. **[Inference](inference)** - model inference insights


**Part 6. Development**

1. **[Debugging and Troubleshooting](debug)** - how to debug easy and difficult issues

1. **[And more debugging](https://github.com/stas00/the-art-of-debugging)**

1. **[Testing](testing)** - numerous tips and tools to make test writing enjoyable


**Part 7. Miscellaneous**

1. **[Resources](resources)** - LLM/VLM chronicles


## Updates

I announce any significant updates on my twitter channel [https://twitter.com/StasBekman](https://twitter.com/StasBekman).

## Ebook versions of the book

You can download various ebook formats of this book:
* [PDF](https://huggingface.co/stas/ml-engineering-book/resolve/main/Stas%20Bekman%20-%20Machine%20Learning%20Engineering.pdf?download=true)
* [EPUB](https://huggingface.co/stas/ml-engineering-book/resolve/main/Stas%20Bekman%20-%20Machine%20Learning%20Engineering.epub?download=true)


I will try to rebuild these once in a few weeks or so, but if you want the latest ebook versions, the instructions for building are [here](build).

Thanks to HuggingFace for giving me permission to host my book's ebook formats at the [HF hub](https://huggingface.co/stas/ml-engineering-book).


## SKILL.md for AI agents

I maintain a [SKILL.md](./SKILL.md) file that you can use to teach your AI agent to train and operate large-scale ML models better.

See also the companion skills: [The Art of Debugging](https://github.com/stas00/the-art-of-debugging/blob/master/SKILL.md) and [Stas' Python Cookbook](https://github.com/stas00/python-cookbook/blob/master/SKILL.md).


## Lectures/Talks

- [Building resilient ML Engineering skills](https://www.youtube.com/watch?v=IBJUt9JPKHk) given on 2026-01-10 for the [GPU Mode community](https://github.com/gpu-mode). Only had time to discuss performance reality of accelerators, network and storage and how each of them can be crucial to the ensemble's performance. Thanks to [Mark Saroufim](https://github.com/msaroufim) for organizing and providing an awesome support during the talk.

## Discussions
```

---

**Machine-readable endpoints**

- JSON: [`/api/graphcanon/tools/stas00-ml-engineering`](/api/graphcanon/tools/stas00-ml-engineering)
- LLM index: [/llms.txt](/llms.txt)
- Full corpus: [/llms-full.txt](/llms-full.txt)

_GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_