{"data":{"slug":"zlab-princeton-llm-pruning-collection","name":"llm-pruning-collection","tagline":"A collection of various llm pruning implementations, training code for GPUs & TPUs, and evaluation script.","github_url":"https://github.com/zlab-princeton/llm-pruning-collection","owner":"zlab-princeton","repo":"llm-pruning-collection","owner_avatar_url":"https://avatars.githubusercontent.com/u/237910146?v=4","primary_language":"Python","stars":69,"forks":8,"topics":["jax","llm-evaluation","llm-training","pruning","pruning-models","tpu"],"archived":false,"github_pushed_at":"2026-04-20T18:15:44+00:00","maintenance_label":"Steady","url":"https://www.graphcanon.com/tools/zlab-princeton-llm-pruning-collection","markdown_url":"https://www.graphcanon.com/tools/zlab-princeton-llm-pruning-collection.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/zlab-princeton-llm-pruning-collection","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=zlab-princeton-llm-pruning-collection","description":"A collection of various llm pruning implementations, training code for GPUs & TPUs, and evaluation script.","homepage_url":null,"license":"Apache-2.0","open_issues":2,"watchers":0,"ai_summary":null,"readme_excerpt":"# Small LLMs: Pruning vs Training from Scratch\n\n[Yufeng Xu<sup>1</sup>](https://github.com/Zephyr271828), [Taiming Lu<sup>1</sup>](https://taiminglu.com/), [Jiachen Zhu<sup>2</sup>](https://jiachenzhu.github.io/), [Mingjie Sun<sup>3</sup>](https://eric-mingjie.github.io/), [Kunjun Li<sup>1</sup>](https://kunjun-li.github.io/), and [Zhuang Liu<sup>1</sup>](https://liuzhuang13.github.io/)\n\n1 Princeton. 2 NYU. 3 CMU.  \n\n---\n\n\nThis is a Jax-based repo for LLM Prunning, It contains:\n- the implementations of various LLM pruning methods of different granularity.\n- pretraining and fine-tuning code for both GPU and TPU platforms.\n- evaluation scripts for assessing model performance.\n\n\n**We gratefully acknowledge the generous support of the Google TPU Research Cloud (TRC), which provided the computational resources used to build this repository.**\n\nThe repo is organized as follows:\n\n```bash\n├── pruning\n│   ├── FLAP # including Wanda-sp and FLAP\n│   ├── LLM-Pruner\n│   ├── llmshearing # sheared llama\n│   ├── minitron # including shortgpt\n│   ├── SLEB # including sleb\n│   ├── SliceGPT # including slicegpt\n│   └── wanda # including sparsegpt and magnitude pruning\n├── training\n│   ├── fms_fsdp\n│   └── maxtext\n└── eval\n```\nwhere `pruning` is the collection of the pruning methods we experimented; `training` contains the LLM training frameworks we used, and we provided options for both TPU and GPU; `eval` contains JAX-compatible eval scripts we used to evaluate the pruned models.\n\n---\n\n\n\n## Supported Features\n**Pruning Methods**\n- [x] [Minitron](pruning/minitron/README.md#minitron-depth)\n- [x] [ShortGPT](pruning/minitron/README.md#shortgpt)\n- [x] [Wanda](pruning/wanda/README.md)\n- [x] [SliceGPT](pruning/SliceGPT/README.md)\n- [x] [SparseGPT](pruning/wanda/README.md)\n- [x] [Magnitude](pruning/wanda/README.md)\n- [x] [Sheared Llama](pruning/llmshearing/README.md)\n- [x] [SLEB](pruning/SLEB/README.md)\n- [x] [LLM Pruner](pruning/LLM-Pruner/README.md)\n- [x] [FLAP](pruning/FLAP/README.md)\n\n\n\n**Training Frameworks**\n- [x] [FMS-FSDP](training/fms_fsdp/README.md) \n- [x] [MaxText](training/maxtext/README.md)\n\n**Evaluation**\n- [x] accelerate [lm-eval-harness](eval/lm-evaluation-harness) for maxtext. (by 2-4x times!)\n\n\n\n## Get Started\n### Pruning\nIn order to reproduce the results of the different pruning methods, we need to set up separate environments for different methods. The installation and command guide can be found at `pruning/<method>/README.md`. Below is an overview:\n\n**Minitron**\n```bash\ncd pruning/minitron\nbash scripts/install.sh\nbash scripts/prune_llama3.1-8b.sh # contains minitron depth and width for llama3.1-8b\n```\n\n**ShortGPT**\n```bash\ncd pruning/minitron\nbash scripts/install.sh\nbash scripts/prune_llama2-7b.sh \n```\n\n**Wanda, SparseGPT, Magnitude**\n```bash\ncd pruning/wanda\nbash scripts/install.sh\nbash scripts/prune_llama3.1-8b.sh # contains wanda, sparsegpt, and magnitude for llama3.1-8b\nbash scripts/prune_llama2-7b.sh\nbash scripts/prune_llama-7b.sh\n```\n\n**LLM-Pruner**\n```bash\ncd pruning/LLM-Pruner\nbash scripts/install.sh\nbash scripts/prune_llama-7b.sh\nbash scripts/prune_llama2-7b.sh\nbash scripts/prune_llama3.1-8b.sh\n```\n\n**Sheared Llama**\n```bash\ncd pruning/llmshearing\nbash scripts/install.sh\n\nmkdir -p llmshearing/data/red_pajama && cd llmshearing/data/red_pajama\nhuggingface-cli download Zephyr271828/redpajama-for-prune --repo-type dataset --local-dir for_prune\ncd -\n\nbash scripts/hf2composer.sh\nbash scripts/prune_llama2-2.7b.sh\nbash scripts/prune_llama2-1.3b.sh\nbash scripts/prune_llama2-370m.sh\nbash scripts/composer2hf.sh\n```\n\n### Training\n**GPU**\nTo train on GPUs, please refer to the guide of [fms-fsdp](training/fms_fsdp/README.md) for details.\n\n**TPU**\nTo train on TPUs, please refer to guide of [MaxText](training/maxtext/README.md) for details.\n\n### Evaluation\n**GPU**  \nFor evaluation on GPUS, you may run the following evaluation script on your HF checkpoint:\n```bash\ncd training/fms_fsdp\nbash scripts/install.sh\n\ncd ../../eval\nbash scripts/eva","github_created_at":"2025-11-05T08:48:15+00:00","created_at":"2026-07-15T10:40:09.001957+00:00","updated_at":"2026-07-15T10:40:11.69851+00:00","categories":[{"slug":"developer-tools","name":"Developer Tools","url":"https://www.graphcanon.com/categories/developer-tools","markdown_url":"https://www.graphcanon.com/categories/developer-tools.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/developer-tools"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"jax","name":"jax"},{"slug":"llm-evaluation","name":"llm-evaluation"},{"slug":"llm-training","name":"llm-training"},{"slug":"pruning","name":"pruning"},{"slug":"pruning-models","name":"pruning-models"},{"slug":"python","name":"python"},{"slug":"tpu","name":"tpu"}],"trust":{"provenance":{"is_fork":false,"github_id":1090120013,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-15T10:40:09.966Z","maintenance":{"label":"Steady","score":60,"methodology":"github_public_v1","releases_90d":0,"days_since_push":85,"last_release_at":null},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-15T10:40:10.287Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-15T10:40:09.732Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-07-15T10:40:09.732Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-07-15T10:40:09.732Z"}}}}