{"data":{"slug":"huggingface-peft","name":"peft","tagline":"State-of-the-art Parameter-Efficient Fine-Tuning","github_url":"https://github.com/huggingface/peft","owner":"huggingface","repo":"peft","owner_avatar_url":"https://avatars.githubusercontent.com/u/25720743?v=4","primary_language":"Python","stars":21704,"forks":2511,"topics":["adapter","diffusion","fine-tuning","llm","lora","parameter-efficient-learning","peft","python","pytorch","transformers"],"archived":false,"github_pushed_at":"2026-09-18T13:30:46+00:00","maintenance_label":"Very active","stars_delta_30d":261,"url":"https://www.graphcanon.com/tools/huggingface-peft","markdown_url":"https://www.graphcanon.com/tools/huggingface-peft.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/huggingface-peft","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=huggingface-peft","description":"🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.","homepage_url":"https://huggingface.co/docs/peft","license":"Apache-2.0","open_issues":84,"watchers":113,"ai_summary":"🤗 PEFT provides state-of-the-art methods for efficient parameter fine-tuning in large models.","readme_excerpt":"### High performance on consumer hardware\n\nConsider the memory requirements for training the following models on the [ought/raft/twitter_complaints](https://huggingface.co/datasets/ought/raft/viewer/twitter_complaints) dataset with an A100 80GB GPU with more than 64GB of CPU RAM.\n\n|   Model         | Full Finetuning | PEFT-LoRA PyTorch  | PEFT-LoRA DeepSpeed with CPU Offloading |\n| --------- | ---- | ---- | ---- |\n| bigscience/T0_3B (3B params) | 47.14GB GPU / 2.96GB CPU  | 14.4GB GPU / 2.96GB CPU | 9.8GB GPU / 17.8GB CPU |\n| bigscience/mt0-xxl (12B params) | OOM GPU | 56GB GPU / 3GB CPU | 22GB GPU / 52GB CPU |\n| bigscience/bloomz-7b1 (7B params) | OOM GPU | 32GB GPU / 3.8GB CPU | 18.1GB GPU / 35GB CPU |\n\nWith LoRA you can fully finetune a 12B parameter model that would've otherwise run out of memory on the 80GB GPU, and comfortably fit and train a 3B parameter model. When you look at the 3B parameter model's performance, it is comparable to a fully finetuned model at a fraction of the GPU memory.\n\n|   Submission Name        | Accuracy |\n| --------- | ---- |\n| Human baseline (crowdsourced) |\t0.897 |\n| Flan-T5 | 0.892 |\n| lora-t0-3b | 0.863 |\n\n> [!TIP]\n> The bigscience/T0_3B model performance isn't optimized in the table above. You can squeeze even more performance out of it by playing around with the input instruction templates, LoRA hyperparameters, and other training related hyperparameters. The final checkpoint size of this model is just 19MB compared to 11GB of the full bigscience/T0_3B model. Learn more about the advantages of finetuning with PEFT in this [blog post](https://www.philschmid.de/fine-tune-flan-t5-peft).","github_created_at":"2022-11-25T03:51:09+00:00","created_at":"2026-07-11T11:36:59.315716+00:00","updated_at":"2026-09-20T06:27:47.26692+00:00","categories":[{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"adapter","name":"adapter"},{"slug":"diffusion","name":"diffusion"},{"slug":"fine-tuning","name":"fine-tuning"},{"slug":"llm","name":"llm"},{"slug":"lora","name":"lora"},{"slug":"parameter-efficient-learning","name":"parameter-efficient-learning"},{"slug":"peft","name":"peft"},{"slug":"python","name":"python"}],"trust":{"provenance":{"is_fork":false,"github_id":570384908,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-20T06:27:44.606Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":2,"days_since_push":1,"last_release_at":"2026-09-15T15:31:52Z","stars_delta_30d":261,"open_issues_delta_30d":26},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T11:37:00.765Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-20T06:27:45.679Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-09-20T06:27:45.679Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-20T06:27:45.679Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When you need to fine-tune large language models but are constrained by compute resources or want to avoid overfitting.","If your project requires state-of-the-art parameter-efficient methods that integrate well with popular frameworks like Transformers and PyTorch."],"when_not_to_use":["If you require a tool that supports training from scratch, as PEFT is specifically designed for fine-tuning purposes only.","When working on models where the full fine-tuning of all parameters is feasible or preferred due to ample compute resources and no concern over overfitting."],"source":"enrich:decision_facts","observed_at":"2026-07-11T14:09:35.121Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"PEFT focuses on advanced techniques for efficiently tuning parameters in large models with Python."}]}}