---
title: "AI-Infra-from-Zero-to-Hero"
type: "tool"
slug: "huaizhengzhang-ai-infra-from-zero-to-hero"
canonical_url: "https://www.graphcanon.com/tools/huaizhengzhang-ai-infra-from-zero-to-hero"
github_url: "https://github.com/HuaizhengZhang/AI-Infra-from-Zero-to-Hero"
homepage_url: "https://huaizheng.xyz/"
stars: 4163
forks: 399
primary_language: null
license: "MIT"
categories: ["model-training", "inference-serving", "developer-tools", "ai-agents"]
tags: ["llmsys", "model-training", "mlsys", "genai", "large-language-models", "model-serving", "ai-infra"]
updated_at: "2026-07-07T18:41:40.75573+00:00"
---

# AI-Infra-from-Zero-to-Hero

> 🚀 Awesome System for Machine Learning ⚡️ AI System Papers and Industry Practice

A comprehensive collection of resources, papers, and industry practices related to machine learning systems, large language models (LLMs), and generative AI. The repository is organized into categories such as ML/DL infrastructure, LLM infrastructure, and domain-specific infrastructures, including video tutorials and links to recent conference proceedings.

## Facts

- Repository: https://github.com/HuaizhengZhang/AI-Infra-from-Zero-to-Hero
- Homepage: https://huaizheng.xyz/
- Stars: 4,163 · Forks: 399 · Open issues: 14 · Watchers: 133
- License: MIT
- Last pushed: 2025-07-25T02:24:35+00:00

## Categories

- [Model Training](/categories/model-training.md)
- [Inference & Serving](/categories/inference-serving.md)
- [Developer Tools](/categories/developer-tools.md)
- [AI Agents](/categories/ai-agents.md)

## Tags

llmsys, model-training, mlsys, genai, large-language-models, model-serving, ai-infra

## Related tools

- [ECC](/tools/affaan-m-ecc.md) - The agent harness performance optimization system (★ 226,962)
- [hermes-agent](/tools/nousresearch-hermes-agent.md) - The self-improving AI agent built by Nous Research (★ 210,880)
- [AutoGPT](/tools/significant-gravitas-autogpt.md) - AutoGPT: Build, Deploy, and Run AI Agents (★ 185,417)
- [ollama](/tools/ollama-ollama.md) - Get up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. (★ 175,659)
- [prompts.chat](/tools/f-prompts-chat.md) - The world's largest open-source prompt library for AI (★ 165,019)
- [transformers](/tools/huggingface-transformers.md) - 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models (★ 162,347)
- [JavaGuide](/tools/snailclimb-javaguide.md) - Snailclimb/JavaGuide: 面试 & 后端通用面试指南，覆盖计算机基础、数据库、分布式、高并发、系统设计与 AI 应用开发 (★ 156,863)
- [langflow](/tools/langflow-ai-langflow.md) - Langflow is a powerful platform for building and deploying AI-powered agents and workflows. (★ 151,298)

## README (excerpt)

```text
# AI System School 

### 💫💫💫 System for Machine Learning, LLM (Large Language Model), GenAI (Generative AI)

### Updates: 

- Video Tutorials [[YouTube]](https://youtu.be/ChD1_aVZJ0g?si=Kg-yB3F4Iea0Xp9J) [[bilibili]](https://www.bilibili.com/video/BV1ZwYUerEtL/) [[小红书]](http://xhslink.com/MmrjcT)
- We are preparing a new website [[Lets Go AI]](https://letsgoai.pro/) for this repo!!!

### *Path to System for AI* [[Whitepaper You Must Read]](./paper/mlsys-whitepaper.pdf)

A curated list of research in machine learning systems. Link to the code if available is also present. Now we have a [team](#maintainer) to maintain this project. *You are very welcome to pull request by using our template*.



## System for AI (Ordered by Category)

### ML / DL Infra

- [Data Processing](data_processing.md#data-processing)
- [Training System](training.md#training-system)
- [Inference System](inference.md#inference-system)
- [Machine Learning Infrastructure](infra.md#machine-learning-infrastructure)

### LLM Infra

- [LLM Training](llm_training.md#llm_training)
- [LLM Serving](llm_serving.md#llm_serving)

### Domain-Specific Infra

- [Video System](video_system.md#video-system)
- [AutoML System](AutoML_system.md#automl-system)
- [Edge AI](edge_system.md#edge-or-mobile-papers)
- [GNN System](GNN_system.md#system-for-gnn-traininginference)
- [Federated Learning System](federated_learning_system.md#federated-learning-system)
- [Deep Reinforcement Learning System](drl_system.md#deep-reinforcement-learning-system)

## System for ML/LLM Conference

### Conference

- OSDI
- SOSP
- SIGCOMM
- NSDI
- MLSys
- ATC
- Eurosys 
- Middleware
- SoCC
- TinyML

## General Resources

- [Survey](#survey)
- [Book](#book)
- [Video](#video)
- [Course](#course)
- [Blog](#blog)

## Survey

- Toward Highly Available, Intelligent Cloud and ML Systems [[Slide]](http://sysnetome.com/Talks/cguo_netai_2018.pdf)
- A curated list of awesome System Designing articles, videos and resources for distributed computing, AKA Big Data. [[GitHub]](https://github.com/madd86/awesome-system-design)
- awesome-production-machine-learning: A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning [[GitHub]](https://github.com/EthicalML/awesome-production-machine-learning)
- Opportunities and Challenges Of Machine Learning Accelerators In Production [[Paper]](https://www.usenix.org/system/files/opml19papers-ananthanarayanan.pdf)
  - Ananthanarayanan, Rajagopal, et al. "
  - 2019 {USENIX} Conference on Operational Machine Learning (OpML 19). 2019.
- How (and How Not) to Write a Good Systems Paper [[Advice]](https://www.usenix.org/legacy/events/samples/submit/advice_old.html)
- Applied machine learning at Facebook: a datacenter infrastructure perspective [[Paper]](https://research.fb.com/wp-content/uploads/2017/12/hpca-2018-facebook.pdf)
  - Hazelwood, Kim, et al. (*HPCA 2018*)
- Infrastructure for Usable Machine Learning: The Stanford DAWN Project
  - Bailis, Peter, Kunle Olukotun, Christopher Ré, and Matei Zaharia. (*preprint 2017*)
- Hidden technical debt in machine learning systems [[Paper]](https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf)
  - Sculley, David, et al. (*NIPS 2015*)
- End-to-end arguments in system design [[Paper]](http://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf)
  - Saltzer, Jerome H., David P. Reed, and David D. Clark. 
- System Design for Large Scale Machine Learning [[Thesis]](http://shivaram.org/publications/shivaram-dissertation.pdf)
- Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications [[Paper]](https://arxiv.org/pdf/1811.09886.pdf)
  - Park, Jongsoo, Maxim Naumov, Protonu Basu et al. *arXiv 2018*
  - Summary: This paper presents a characterizations of DL models and then shows the new design principle of DL hardware.
- A Berkeley View of Systems Challenges for AI [[Paper]](https://arxiv.org/pdf/
```

---

**Machine-readable endpoints**

- JSON: [`/api/graphcanon/tools/huaizhengzhang-ai-infra-from-zero-to-hero`](/api/graphcanon/tools/huaizhengzhang-ai-infra-from-zero-to-hero)
- LLM index: [/llms.txt](/llms.txt)
- Full corpus: [/llms-full.txt](/llms-full.txt)

_GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_