{"data":{"slug":"huangowen-awesome-llm-compression","name":"Awesome-LLM-Compression","tagline":"Awesome LLM compression research papers and tools to accelerate LLM training and inference.","github_url":"https://github.com/HuangOwen/Awesome-LLM-Compression","owner":"HuangOwen","repo":"Awesome-LLM-Compression","owner_avatar_url":"https://avatars.githubusercontent.com/u/24937399?v=4","primary_language":null,"stars":1883,"forks":134,"topics":[],"archived":false,"github_pushed_at":"2026-08-27T15:48:21+00:00","maintenance_label":"Steady","stars_delta_30d":17,"url":"https://www.graphcanon.com/tools/huangowen-awesome-llm-compression","markdown_url":"https://www.graphcanon.com/tools/huangowen-awesome-llm-compression.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/huangowen-awesome-llm-compression","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=huangowen-awesome-llm-compression","description":"Awesome LLM compression research papers and tools.","homepage_url":null,"license":"MIT","open_issues":5,"watchers":48,"ai_summary":"Compilation of research papers and tools focused on compressing large language models for improved computational efficiency during both training and serving phases.","readme_excerpt":"<div align=\"center\">\n    <h1>Awesome LLM Compression</h1>\n    <a href=\"https://awesome.re\"><img src=\"https://awesome.re/badge.svg\"/></a>\n    <img src=https://img.shields.io/github/stars/HuangOwen/Awesome-LLM-Compression.svg?style=social >\n    <img src=https://img.shields.io/github/watchers/HuangOwen/Awesome-LLM-Compression.svg?style=social >\n</div>\n\n\n\nAwesome LLM compression research papers and tools to accelerate LLM training and inference. \n\nPapers in each category are grouped by year in collapsible blocks, newest first — click a year to expand it. The current year is expanded by default.\n\n# Contents\n\n- [📑 Papers](#papers)\n  - [Survey](#survey)\n  - [Quantization](#quantization)\n  - [Pruning and Sparsity](#pruning-and-sparsity)\n  - [Distillation](#distillation)\n  - [Efficient Prompting](#efficient-prompting)\n  - [KV Cache Compression](#kv-cache-compression)\n  - [Other](#other)\n- [🔧 Tools](#tools)\n  - [Inference and Serving Engines](#inference-and-serving-engines)\n  - [Quantization Libraries and Kernels](#quantization-libraries-and-kernels)\n  - [Model Compression Toolkits](#model-compression-toolkits)\n  - [KV Cache and Prompt Compression](#kv-cache-and-prompt-compression)\n  - [Training and Fine-Tuning](#training-and-fine-tuning)\n  - [Analysis and Benchmarking](#analysis-and-benchmarking)\n  - [Other Tools](#other-tools)\n- [🙌 Contributing](#contributing)\n- [🌟 Star History](#star-history)\n\n## Papers\n\n### Survey\n\n<details open>\n<summary><b>2026</b> &nbsp;·&nbsp; 6 papers</summary>\n\n- Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey <br> Arxiv 2026 [[Paper]](https://arxiv.org/abs/2603.04445)\n\n- A Survey of On-Policy Distillation for Large Language Models <br> Arxiv 2026 [[Paper]](https://arxiv.org/abs/2604.00626)\n\n- Unifying Data, Memory, and Compute Efficiency in LLM training: A Survey <br> TAI 2026 [[Paper]](https://arxiv.org/abs/2606.10706)\n\n- From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Serving <br> Arxiv 2026 [[Paper]](https://arxiv.org/abs/2607.02574)\n\n- Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization <br> ACL Findings 2026 [[Paper]](https://arxiv.org/abs/2607.08057)\n\n- Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques <br> IJCAI-ECAI 2026 (Survey Track) [[Paper]](https://arxiv.org/abs/2607.12829)\n\n</details>\n\n<details>\n<summary><b>2025</b> &nbsp;·&nbsp; 14 papers</summary>\n\n- Compressed but Compromised? A Study of Jailbreaking in Compressed LLMs <br> NeurIPS Lock-LLM Workshop 2025 [[Paper]](https://openreview.net/pdf?id=OkNfb8SmLh) [[Blog]](https://namburisrinath.medium.com/compressed-but-compromised-a-study-of-jailbreaking-in-compressed-llms-02a6e40aaf17)\n\n- Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systems <br> EMNLP 2025 [[Paper]](https://arxiv.org/abs/2502.14305) \n\n- Key, Value, Compress: A Systematic Exploration of KV Cache Compression Techniques <br> CICC 2025 [[Paper]](https://arxiv.org/abs/2503.11816)\n\n- Are We There Yet? A Measurement Study of Efficiency for LLM Applications on Mobile Devices <br> Arxiv 2025 [[Paper]](https://arxiv.org/abs/2504.00002) \n\n- Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions <br> Arxiv 2025 [[Paper]](https://arxiv.org/abs/2504.14772) \n\n- An Empirical Study on Prompt Compression for Large Language Models <br> Building Trust Workshop @ ICLR 2025 2025 [[Paper]](https://arxiv.org/abs/2505.00019) [[PCToolkit]](https://github.com/3DAgentWorld/Toolkit-for-Prompt-Compression)\n\n- Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities <br> Arxiv 2025 [[Paper]](https://arxiv.org/abs/2505.01043) [[GitHub Page]](https://github.com/Hao840/Awesome-Low-Precision-Training)\n\n- Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques <br> COMPSAC 202","github_created_at":"2023-05-30T06:26:11+00:00","created_at":"2026-07-11T10:32:40.865569+00:00","updated_at":"2026-10-11T06:00:46.492355+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"}],"tags":[{"slug":"compression","name":"compression"},{"slug":"efficiency","name":"efficiency"},{"slug":"research-papers","name":"research papers"},{"slug":"training-acceleration","name":"training acceleration"}],"trust":{"provenance":{"is_fork":false,"github_id":647138126,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-10-11T06:00:45.601Z","maintenance":{"label":"Steady","score":60,"methodology":"github_public_v1","releases_90d":0,"days_since_push":44,"last_release_at":null,"stars_delta_30d":17,"open_issues_delta_30d":3},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T10:32:42.037Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-10-11T06:00:46.027Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-10-11T06:00:46.027Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["The repository provides curated listings but does not develop its own software; hence specific language requirements are not applicable."]},"constraints":null,"when_to_use":["When you need to explore the latest advancements in LLM compression techniques and their impact on both training and inference.","If your project requires a detailed survey of model compression for large language models covering various aspects like quantization, pruning, distillation, and efficient prompting.","For thorough understanding on how compression affects parametric knowledge, which can be critical before choosing specific compression methods."],"when_not_to_use":["Avoid relying solely on Awesome LLM-Compression if you require a hands-on toolset rather than theoretical frameworks and research papers, as it focuses more on consolidating the survey information.","If your immediate need is for proprietary or commercial tools that offer out-of-the-box functionality, since this resource mainly links to academic research and open-source projects."],"source":"enrich:decision_facts","observed_at":"2026-07-11T11:18:59.188Z"},"constraint_facets":null,"decision_summary":[{"label":"Requirements","value":"The repository provides curated listings but does not develop its own software; hence specific language requirements are not applicable."},{"label":"Adopt for","value":"Awesome LLM-Compression curates a comprehensive collection of research papers and tools aimed at compressing large language models, focusing on enhancing computational efficiency during both training and serving phases."},{"label":"License detail","value":"MIT License"}]}}