{"data":{"slug":"zlab-princeton-llm-pruning-collection","name":"llm-pruning-collection","tagline":"Collection of LLM pruning methods and training code for GPUs & TPUs.","github_url":"https://github.com/zlab-princeton/llm-pruning-collection","owner":"zlab-princeton","repo":"llm-pruning-collection","owner_avatar_url":"https://avatars.githubusercontent.com/u/237910146?v=4","primary_language":"Python","stars":72,"forks":9,"topics":["jax","llm-evaluation","llm-training","pruning","pruning-models","tpu"],"archived":false,"github_pushed_at":"2026-04-20T18:15:44+00:00","maintenance_label":"Slowing","stars_delta_30d":3,"url":"https://www.graphcanon.com/tools/zlab-princeton-llm-pruning-collection","markdown_url":"https://www.graphcanon.com/tools/zlab-princeton-llm-pruning-collection.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/zlab-princeton-llm-pruning-collection","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=zlab-princeton-llm-pruning-collection","description":"A collection of various llm pruning implementations, training code for GPUs & TPUs, and evaluation script.","homepage_url":null,"license":"Apache-2.0","open_issues":2,"watchers":0,"ai_summary":"Includes various implementations of LLM pruning, pretraining and fine-tuning codes, and evaluation scripts for GPU and TPU platforms.","readme_excerpt":"# Small LLMs: Pruning vs Training from Scratch\n\n[Yufeng Xu<sup>1</sup>](https://github.com/Zephyr271828), [Taiming Lu<sup>1</sup>](https://taiminglu.com/), [Jiachen Zhu<sup>2</sup>](https://jiachenzhu.github.io/), [Mingjie Sun<sup>3</sup>](https://eric-mingjie.github.io/), [Kunjun Li<sup>1</sup>](https://kunjun-li.github.io/), and [Zhuang Liu<sup>1</sup>](https://liuzhuang13.github.io/)\n\n1 Princeton. 2 NYU. 3 CMU.  \n\n---\n\n\nThis is a Jax-based repo for LLM Prunning, It contains:\n- the implementations of various LLM pruning methods of different granularity.\n- pretraining and fine-tuning code for both GPU and TPU platforms.\n- evaluation scripts for assessing model performance.\n\n\n**We gratefully acknowledge the generous support of the Google TPU Research Cloud (TRC), which provided the computational resources used to build this repository.**\n\nThe repo is organized as follows:\n\n```bash\n├── pruning\n│   ├── FLAP # including Wanda-sp and FLAP\n│   ├── LLM-Pruner\n│   ├── llmshearing # sheared llama\n│   ├── minitron # including shortgpt\n│   ├── SLEB # including sleb\n│   ├── SliceGPT # including slicegpt\n│   └── wanda # including sparsegpt and magnitude pruning\n├── training\n│   ├── fms_fsdp\n│   └── maxtext\n└── eval\n```\nwhere `pruning` is the collection of the pruning methods we experimented; `training` contains the LLM training frameworks we used, and we provided options for both TPU and GPU; `eval` contains JAX-compatible eval scripts we used to evaluate the pruned models.\n\n---\n\n\n\n## Supported Features\n**Pruning Methods**\n- [x] [Minitron](pruning/minitron/README.md#minitron-depth)\n- [x] [ShortGPT](pruning/minitron/README.md#shortgpt)\n- [x] [Wanda](pruning/wanda/README.md)\n- [x] [SliceGPT](pruning/SliceGPT/README.md)\n- [x] [SparseGPT](pruning/wanda/README.md)\n- [x] [Magnitude](pruning/wanda/README.md)\n- [x] [Sheared Llama](pruning/llmshearing/README.md)\n- [x] [SLEB](pruning/SLEB/README.md)\n- [x] [LLM Pruner](pruning/LLM-Pruner/README.md)\n- [x] [FLAP](pruning/FLAP/README.md)\n\n\n\n**Training Frameworks**\n- [x] [FMS-FSDP](training/fms_fsdp/README.md) \n- [x] [MaxText](training/maxtext/README.md)\n\n**Evaluation**\n- [x] accelerate [lm-eval-harness](eval/lm-evaluation-harness) for maxtext. (by 2-4x times!)\n\n\n\n## Get Started\n### Pruning\nIn order to reproduce the results of the different pruning methods, we need to set up separate environments for different methods. The installation and command guide can be found at `pruning/<method>/README.md`. Below is an overview:\n\n**Minitron**\n```bash\ncd pruning/minitron\nbash scripts/install.sh\nbash scripts/prune_llama3.1-8b.sh # contains minitron depth and width for llama3.1-8b\n```\n\n**ShortGPT**\n```bash\ncd pruning/minitron\nbash scripts/install.sh\nbash scripts/prune_llama2-7b.sh \n```\n\n**Wanda, SparseGPT, Magnitude**\n```bash\ncd pruning/wanda\nbash scripts/install.sh\nbash scripts/prune_llama3.1-8b.sh # contains wanda, sparsegpt, and magnitude for llama3.1-8b\nbash scripts/prune_llama2-7b.sh\nbash scripts/prune_llama-7b.sh\n```\n\n**LLM-Pruner**\n```bash\ncd pruning/LLM-Pruner\nbash scripts/install.sh\nbash scripts/prune_llama-7b.sh\nbash scripts/prune_llama2-7b.sh\nbash scripts/prune_llama3.1-8b.sh\n```\n\n**Sheared Llama**\n```bash\ncd pruning/llmshearing\nbash scripts/install.sh\n\nmkdir -p llmshearing/data/red_pajama && cd llmshearing/data/red_pajama\nhuggingface-cli download Zephyr271828/redpajama-for-prune --repo-type dataset --local-dir for_prune\ncd -\n\nbash scripts/hf2composer.sh\nbash scripts/prune_llama2-2.7b.sh\nbash scripts/prune_llama2-1.3b.sh\nbash scripts/prune_llama2-370m.sh\nbash scripts/composer2hf.sh\n```\n\n### Training\n**GPU**\nTo train on GPUs, please refer to the guide of [fms-fsdp](training/fms_fsdp/README.md) for details.\n\n**TPU**\nTo train on TPUs, please refer to guide of [MaxText](training/maxtext/README.md) for details.\n\n### Evaluation\n**GPU**  \nFor evaluation on GPUS, you may run the following evaluation script on your HF checkpoint:\n```bash\ncd training/fms_fsdp\nbash scripts/install.sh\n\ncd ../../eval\nbash scripts/eva","github_created_at":"2025-11-05T08:48:15+00:00","created_at":"2026-07-15T10:40:09.001957+00:00","updated_at":"2026-09-20T04:25:05.831588+00:00","categories":[{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"jax","name":"jax"},{"slug":"llm-evaluation","name":"llm-evaluation"},{"slug":"llm-training","name":"llm-training"},{"slug":"pruning","name":"pruning"},{"slug":"tpu","name":"tpu"}],"trust":{"provenance":{"is_fork":false,"github_id":1090120013,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-09T06:00:46.550Z","maintenance":{"label":"Slowing","score":36,"methodology":"github_public_v1","releases_90d":0,"days_since_push":141,"last_release_at":null,"stars_delta_30d":3,"open_issues_delta_30d":0},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-15T10:40:10.287Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-09T06:00:47.013Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-09-09T06:00:47.013Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-09T06:00:47.013Z"}},"decision_facts":{"hosting":null,"pricing":{"model":"freemium","summary":"The software is free and open-source, licensed under Apache-2.0, but users must provide their own hardware or use cloud services like Google TPU Research Cloud for computational resources."},"requirements":{"notes":["The repository includes pretraining and fine-tuning scripts for both GPU and TPU platforms.","A JAX-based environment is required to run the code in this repository."],"min_ram_gb":null,"requires_docker":false},"constraints":{"min_ram_gb":null,"pricing_model":"freemium","requires_docker":false},"when_to_use":["When you are working on reducing the size or improving inference speed of large language models using various pruning techniques available in this collection.","If you need specific fine-tuning and pretraining framework options for both GPU and TPU platforms, which support a variety of popular pruning methods and their implementations."],"when_not_to_use":["Avoid if your project requires a pruning method that is not included in the collection or if the current platform capabilities do not align with your hardware requirements.","Not suitable for those who need tools to train models from scratch rather than focusing on model pruning and optimization techniques."],"source":"enrich:decision_facts","observed_at":"2026-07-16T19:13:25.067Z"},"constraint_facets":{"min_ram_gb":null,"pricing_model":"freemium","requires_docker":false},"decision_summary":[{"label":"Pricing","value":"freemium - The software is free and open-source, licensed under Apache-2.0, but users must provide their own hardware or use cloud services like Google TPU Research Cloud for computational resources."},{"label":"Requirements","value":"The repository includes pretraining and fine-tuning scripts for both GPU and TPU platforms.; A JAX-based environment is required to run the code in this repository."},{"label":"Adopt for","value":"The llm-pruning-collection repository provides a comprehensive set of large language model pruning methods, along with the necessary training and evaluation scripts for GPUs and TPUs."}]}}