{"data":{"slug":"artidoro-qlora","name":"qlora","tagline":"QLoRA finetuning of quantized LLMs","github_url":"https://github.com/artidoro/qlora","owner":"artidoro","repo":"qlora","owner_avatar_url":"https://avatars.githubusercontent.com/u/11949572?v=4","primary_language":"Jupyter Notebook","stars":11021,"forks":876,"topics":[],"archived":false,"github_pushed_at":"2024-06-10T19:20:16+00:00","maintenance_label":"Dormant","stars_delta_30d":42,"url":"https://www.graphcanon.com/tools/artidoro-qlora","markdown_url":"https://www.graphcanon.com/tools/artidoro-qlora.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/artidoro-qlora","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=artidoro-qlora","description":"QLoRA: Efficient Finetuning of Quantized LLMs","homepage_url":"https://arxiv.org/abs/2305.14314","license":"MIT","open_issues":207,"watchers":80,"ai_summary":"A repository for QLoRA, which provides tools for efficient fine-tuning of quantized large language models including the Guanaco model family.","readme_excerpt":"## License and Intended Use\nWe release the resources associated with QLoRA finetuning in this repository under MIT license.\nIn addition, we release the Guanaco model family for base LLaMA model sizes of 7B, 13B, 33B, and 65B. These models are intended for purposes in line with the LLaMA license and require access to the LLaMA models.\n\n---\n\n## Installation\nTo load models in 4bits with transformers and bitsandbytes, you have to install accelerate and transformers from source and make sure you have the latest version of the bitsandbytes library. After installing PyTorch (follow instructions [here](https://pytorch.org/get-started/locally/)), you can achieve the above with the following command:\n```bash\npip install -U -r requirements.txt\n```\n\n---\n\n## Getting Started\nThe `qlora.py` code is a starting point for finetuning and inference on various datasets.\nBasic command for finetuning a baseline model on the Alpaca dataset:\n```bash\npython qlora.py --model_name_or_path <path_or_name>\n```\n\nFor models larger than 13B, we recommend adjusting the learning rate:\n```bash\npython qlora.py –learning_rate 0.0001 --model_name_or_path <path_or_name>\n```\n\nTo replicate our Guanaco models see below.","github_created_at":"2023-05-11T09:30:23+00:00","created_at":"2026-07-11T23:22:17.364535+00:00","updated_at":"2026-09-26T06:00:17.072575+00:00","categories":[{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"fine-tuning","name":"fine-tuning"},{"slug":"guanaco","name":"guanaco"},{"slug":"llama-models","name":"llama models"},{"slug":"quantization","name":"quantization"}],"trust":{"provenance":{"is_fork":false,"github_id":639346169,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-26T06:00:16.036Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":837,"last_release_at":null,"stars_delta_30d":42,"open_issues_delta_30d":1},"security_summary":{"status":"findings","scanner":"osv@v1","low_count":48,"high_count":0,"last_scan_at":"2026-07-11T23:22:24.112Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-26T06:00:16.527Z"},"languages":{"value":["jupyter notebook"],"source":"github.language","observed_at":"2026-09-26T06:00:16.527Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-09-26T06:00:16.527Z"}},"decision_facts":{"hosting":null,"pricing":{"model":"freemium","summary":"Open source under MIT License; requires access to LLaMA base models"},"requirements":{"notes":["Installation involves installing PyTorch and specific packages from source","Works with model sizes ranging from 7B to 65B, includes recommendations for tuning different sizes"],"min_ram_gb":null,"requires_docker":false},"constraints":{"min_ram_gb":null,"pricing_model":"freemium","requires_docker":false},"when_to_use":["Need efficient fine-tuning for quantized LLaMA-based models","Working with limited computational resources, aiming to maintain performance"],"when_not_to_use":["Require native full-precision model tuning without efficiency constraints","Focusing on non-LLaMA-based language models where specific adaptations may not apply"],"source":"enrich:decision_facts","observed_at":"2026-07-17T12:56:06.728Z"},"constraint_facets":{"min_ram_gb":null,"pricing_model":"freemium","requires_docker":false},"decision_summary":[{"label":"Pricing","value":"freemium - Open source under MIT License; requires access to LLaMA base models"},{"label":"Requirements","value":"Installation involves installing PyTorch and specific packages from source; Works with model sizes ranging from 7B to 65B, includes recommendations for tuning different sizes"},{"label":"Adopt for","value":"QLoRA specializes in accelerating the fine-tuning process of quantized large language models like those in the Guanaco family."},{"label":"License detail","value":"MIT License; open-source tool for QLoRA fine-tuning process; LLaMA base models must be obtained legally as per their license terms"}]}}