{"data":{"slug":"huaizhengzhang-ai-infra-from-zero-to-hero","name":"AI-Infra-from-Zero-to-Hero","tagline":"Awesome System for Machine Learning and LLM Infra","github_url":"https://github.com/HuaizhengZhang/AI-Infra-from-Zero-to-Hero","owner":"HuaizhengZhang","repo":"AI-Infra-from-Zero-to-Hero","owner_avatar_url":"https://avatars.githubusercontent.com/u/5894780?v=4","primary_language":null,"stars":4285,"forks":409,"topics":["ai-infra","genai","large-language-models","llmsys","mlsys","model-serving","model-training"],"archived":false,"github_pushed_at":"2025-07-25T02:24:35+00:00","maintenance_label":"Dormant","stars_delta_30d":87,"url":"https://www.graphcanon.com/tools/huaizhengzhang-ai-infra-from-zero-to-hero","markdown_url":"https://www.graphcanon.com/tools/huaizhengzhang-ai-infra-from-zero-to-hero.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/huaizhengzhang-ai-infra-from-zero-to-hero","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=huaizhengzhang-ai-infra-from-zero-to-hero","description":"🚀 Awesome System for Machine Learning ⚡️ AI System Papers and Industry Practice. ⚡️ System for Machine Learning, LLM (Large Language Model), GenAI (Generative AI). 🍻 OSDI, NSDI, SIGCOMM, SoCC, MLSys, etc. 🗃️ Llama3, Mistral, etc. 🧑‍💻 Video Tutorials. ","homepage_url":"https://huaizheng.xyz/","license":"MIT","open_issues":14,"watchers":134,"ai_summary":"Curated list of research papers, industry practices, video tutorials, and resources related to AI system design, covering areas such as data processing, training systems, inference systems, large language model infrastructures.","readme_excerpt":"# AI System School \n\n### 💫💫💫 System for Machine Learning, LLM (Large Language Model), GenAI (Generative AI)\n\n### Updates: \n\n- Video Tutorials [[YouTube]](https://youtu.be/ChD1_aVZJ0g?si=Kg-yB3F4Iea0Xp9J) [[bilibili]](https://www.bilibili.com/video/BV1ZwYUerEtL/) [[小红书]](http://xhslink.com/MmrjcT)\n- We are preparing a new website [[Lets Go AI]](https://letsgoai.pro/) for this repo!!!\n\n### *Path to System for AI* [[Whitepaper You Must Read]](./paper/mlsys-whitepaper.pdf)\n\nA curated list of research in machine learning systems. Link to the code if available is also present. Now we have a [team](#maintainer) to maintain this project. *You are very welcome to pull request by using our template*.\n\n\n\n## System for AI (Ordered by Category)\n\n### ML / DL Infra\n\n- [Data Processing](data_processing.md#data-processing)\n- [Training System](training.md#training-system)\n- [Inference System](inference.md#inference-system)\n- [Machine Learning Infrastructure](infra.md#machine-learning-infrastructure)\n\n### LLM Infra\n\n- [LLM Training](llm_training.md#llm_training)\n- [LLM Serving](llm_serving.md#llm_serving)\n\n### Domain-Specific Infra\n\n- [Video System](video_system.md#video-system)\n- [AutoML System](AutoML_system.md#automl-system)\n- [Edge AI](edge_system.md#edge-or-mobile-papers)\n- [GNN System](GNN_system.md#system-for-gnn-traininginference)\n- [Federated Learning System](federated_learning_system.md#federated-learning-system)\n- [Deep Reinforcement Learning System](drl_system.md#deep-reinforcement-learning-system)\n\n## System for ML/LLM Conference\n\n### Conference\n\n- OSDI\n- SOSP\n- SIGCOMM\n- NSDI\n- MLSys\n- ATC\n- Eurosys \n- Middleware\n- SoCC\n- TinyML\n\n## General Resources\n\n- [Survey](#survey)\n- [Book](#book)\n- [Video](#video)\n- [Course](#course)\n- [Blog](#blog)\n\n## Survey\n\n- Toward Highly Available, Intelligent Cloud and ML Systems [[Slide]](http://sysnetome.com/Talks/cguo_netai_2018.pdf)\n- A curated list of awesome System Designing articles, videos and resources for distributed computing, AKA Big Data. [[GitHub]](https://github.com/madd86/awesome-system-design)\n- awesome-production-machine-learning: A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning [[GitHub]](https://github.com/EthicalML/awesome-production-machine-learning)\n- Opportunities and Challenges Of Machine Learning Accelerators In Production [[Paper]](https://www.usenix.org/system/files/opml19papers-ananthanarayanan.pdf)\n  - Ananthanarayanan, Rajagopal, et al. \"\n  - 2019 {USENIX} Conference on Operational Machine Learning (OpML 19). 2019.\n- How (and How Not) to Write a Good Systems Paper [[Advice]](https://www.usenix.org/legacy/events/samples/submit/advice_old.html)\n- Applied machine learning at Facebook: a datacenter infrastructure perspective [[Paper]](https://research.fb.com/wp-content/uploads/2017/12/hpca-2018-facebook.pdf)\n  - Hazelwood, Kim, et al. (*HPCA 2018*)\n- Infrastructure for Usable Machine Learning: The Stanford DAWN Project\n  - Bailis, Peter, Kunle Olukotun, Christopher Ré, and Matei Zaharia. (*preprint 2017*)\n- Hidden technical debt in machine learning systems [[Paper]](https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf)\n  - Sculley, David, et al. (*NIPS 2015*)\n- End-to-end arguments in system design [[Paper]](http://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf)\n  - Saltzer, Jerome H., David P. Reed, and David D. Clark. \n- System Design for Large Scale Machine Learning [[Thesis]](http://shivaram.org/publications/shivaram-dissertation.pdf)\n- Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications [[Paper]](https://arxiv.org/pdf/1811.09886.pdf)\n  - Park, Jongsoo, Maxim Naumov, Protonu Basu et al. *arXiv 2018*\n  - Summary: This paper presents a characterizations of DL models and then shows the new design principle of DL hardware.\n- A Berkeley View of Systems Challenges for AI [[Paper]](https://arxiv.org/pdf/","github_created_at":"2019-01-07T12:59:19+00:00","created_at":"2026-07-07T17:35:27.43766+00:00","updated_at":"2026-08-17T18:01:14.99012+00:00","categories":[{"slug":"developer-tools","name":"Developer Tools","url":"https://www.graphcanon.com/categories/developer-tools","markdown_url":"https://www.graphcanon.com/categories/developer-tools.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/developer-tools"},{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"ai-infra","name":"ai-infra"},{"slug":"genai","name":"genai"},{"slug":"large-language-models","name":"large language models"},{"slug":"llmsys","name":"llmsys"},{"slug":"mlsys","name":"mlsys"}],"trust":{"provenance":{"is_fork":false,"github_id":164433070,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-17T18:01:12.732Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":388,"last_release_at":null,"stars_delta_30d":87,"open_issues_delta_30d":0},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T11:05:22.865Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-17T18:01:13.947Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-08-17T18:01:13.947Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When you are aiming to understand the foundational research papers, industry practices, video tutorials specific to ML systems and LLM infrastructures without requiring implementation details.","For those interested in attending specialized conferences like MLSys or OSDI, this repository highlights key themes."],"when_not_to_use":["If you need step-by-step implementations for AI infrastructure setup as the repository focuses on resources rather than detailed technical instructions.","Avoid if seeking guidance specifically for real-time system deployment and tuning, since it does not cover operational tactics in depth."],"source":"enrich:decision_facts","observed_at":"2026-07-14T20:01:37.605Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"A curated resource list for AI system design focusing on large language models and various system aspects."}]}}