LLMSys-PaperList
Curated list of academic papers related to Large Language Model systems
GraphCanon updated today · GitHub synced today
Trust & integrity
Full report- Maintenance
- Very active (1d since push)
- As of today · Source: github_public_v1
- Provenance
- Not a fork · Personal account
- As of today · Source: github_public_v1
- Security (OSV)
- No lockfile
- As of today · Source: none
Public GitHub metadata and optional OSV dependency scans. Signals, not a guarantee. Trust methodology.
Overview
A repository containing a curated collection of academic papers, articles, tutorials, and projects related to Large Language Model (LLM) systems in categories such as training, serving, multi-modal systems, LLM frameworks, ML conferences, survey papers, benchmarks, and more.
Capability facts
No sourced capability facts yet. Facts appear after ingest scans repo manifests (Dockerfile, package.json, MCP configs).
Categories
Tags
README
Awesome LLM Systems Papers
A curated list of Large Language Model systems related academic papers, articles, tutorials, slides and projects. Star this repository, and then you can keep abreast of the latest developments of this booming research field.
Table of Contents
- LLM Systems
- Training
- Pre-training
- Post Training
- Fault Tolerance / Straggler Mitigation
- Serving
- LLM serving
- Agent Systems
- Serving at the edge
- System Efficiency Optimization - Model Co-design
- Multi-Modal Training Systems
- Multi-Modal Serving Systems
- Training
- LLM for Systems
- Industrial LLM Technical Report
- ML Conferences
- NeurIPS 2025
- LLM Frameworks
- Training
- Post-Training
- Serving
- ML Systems
- Survey Paper
- LLM Benchmark / Leaderboard / Traces
- Related ML Readings
- MLSys Courses
- Other Reading
LLM Systems
Training
Pre-training
Before 2024
- Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM
- Reducing Activation Recomputation in Large Transformer Models
- Optimized Network Architectures for Large Language Model Training with Billions of Parameters | MIT
- Carbon Emissions and Large Neural Network Training | Google, UCB
2024
- Perseus: Removing Energy Bloat from Large Model Training | SOSP' 24
- MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs | ByteDance
- DISTMM: Accelerating distributed multimodal model training | NSDI' 24
- Pipeline Parallelism with Controllable Memory | Sea AI Lab
- Boosting Large-scale Parallel Training Efficiency with C4: A Communication-Driven Approach
- Scaling Beyond the GPU Memory Limit for Large Mixture-of-Experts Model Training | ICML' 24
- Alibaba HPN: A Data Center Network for Large Language ModelTraining
- The Llama 3 Herd of Models (Section 3)
- Enabling Parallelism Hot Switching for Efficient Training of Large Language Models | SOSP' 24
- Revisiting Reliability in Large-Scale Machine Learning Research Clusters
- ScheMoE: An Extensible Mixture-of-Experts Distributed Training System with Tasks Scheduling | EuroSys '24
- DynaPipe : Optimizing Multi-task Training through Dynamic Pipelines | EuroSys '24
- HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis | EuroSys'24
- Demystifying Workload Imbalances in Large Transformer Model Training over Variable-length Sequences | PKU
- [Improving