{"data":{"slug":"hiyouga-llamafactory","name":"LlamaFactory","tagline":"Unified Efficient Fine-Tuning of 100+ LLMs & VLMs","github_url":"https://github.com/hiyouga/LlamaFactory","owner":"hiyouga","repo":"LlamaFactory","owner_avatar_url":"https://avatars.githubusercontent.com/u/16256802?v=4","primary_language":"Python","stars":74849,"forks":9170,"topics":["agent","ai","deepseek","fine-tuning","gemma","gpt","instruction-tuning","large-language-models","llama","llama3","llm","lora","moe","nlp","peft","qlora","quantization","qwen","rlhf","transformers"],"archived":false,"github_pushed_at":"2026-09-14T08:09:48+00:00","maintenance_label":"Very active","stars_delta_30d":717,"url":"https://www.graphcanon.com/tools/hiyouga-llamafactory","markdown_url":"https://www.graphcanon.com/tools/hiyouga-llamafactory.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/hiyouga-llamafactory","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=hiyouga-llamafactory","description":"Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)","homepage_url":"https://llamafactory.readthedocs.io","license":"Apache-2.0","open_issues":1156,"watchers":348,"ai_summary":"LlamaFactory is a Python-based tool for efficient fine-tuning of over 100 large language models and vision-language models, supporting various methods like LoRA, QLoRA, and quantization.","readme_excerpt":"### Hardware Requirement\n\n\\* *estimated*\n\n| Method                              | Bits |   7B  |  14B  |  30B  |   70B  |   `x`B  |\n| ----------------------------------- | ---- | ----- | ----- | ----- | ------ | ------- |\n| Full (`bf16` or `fp16`)             |  32  | 120GB | 240GB | 600GB | 1200GB | `18x`GB |\n| Full (`pure_bf16`)                  |  16  |  60GB | 120GB | 300GB |  600GB |  `8x`GB |\n| Freeze/LoRA/GaLore/APOLLO/BAdam/OFT |  16  |  16GB |  32GB |  64GB |  160GB |  `2x`GB |\n| QLoRA / QOFT                        |   8  |  10GB |  20GB |  40GB |   80GB |   `x`GB |\n| QLoRA / QOFT                        |   4  |   6GB |  12GB |  24GB |   48GB | `x/2`GB |\n| QLoRA / QOFT                        |   2  |   4GB |   8GB |  16GB |   24GB | `x/4`GB |\n\n---\n\n### Installation\n\n> [!IMPORTANT]\n> Installation is mandatory.\n\n#### Install from Source\n\n```bash\ngit clone --depth 1 https://github.com/hiyouga/LlamaFactory.git\ncd LlamaFactory\npip install -e .\npip install -r requirements/metrics.txt\n```\n\nOptional dependencies available: `metrics`, `deepspeed`. Install with: `pip install -e . && pip install -r requirements/metrics.txt -r requirements/deepspeed.txt`\n\nAdditional dependencies for specific features are available in `examples/requirements/`.\n\n#### Install from Docker Image\n\n```bash\ndocker run -it --rm --gpus=all --ipc=host hiyouga/llamafactory:latest\n```\n\nThis image is built on Ubuntu 22.04 (x86\\_64), CUDA 12.4, Python 3.11, PyTorch 2.6.0, and Flash-attn 2.7.4.\n\nFind the pre-built images: https://hub.docker.com/r/hiyouga/llamafactory/tags\n\nPlease refer to [build docker](#build-docker) to build the image yourself.\n\n<details><summary>Setting up a virtual environment with <b>uv</b></summary>\n\nCreate an isolated Python environment with [uv](https://github.com/astral-sh/uv):\n\n```bash\nuv run llamafactory-cli webui\n```\n\n</details>\n\n<details><summary>For Windows users</summary>\n\n#### Install PyTorch\n\nYou need to manually install the GPU version of PyTorch on the Windows platform. Please refer to the [official website](https://pytorch.org/get-started/locally/) and the following command to install PyTorch with CUDA support:\n\n```bash\npip uninstall torch torchvision torchaudio\npip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126\npython -c \"import torch; print(torch.cuda.is_available())\"\n```\n\nIf you see `True` then you have successfully installed PyTorch with CUDA support.\n\nTry `dataloader_num_workers: 0` if you encounter `Can't pickle local object` error.\n\n#### Install BitsAndBytes\n\nTo enable Quantized LoRA (QLoRA) on Windows, you need to install bitsandbytes.\n\nFor most users, it is recommended to install the latest official release:\n\n```bash\npip install bitsandbytes\n```\n\nIf you are using uv to manage your virtual environment, it is recommended to install bitsandbytes after installing the GPU-enabled version of PyTorch:\n\n```bash\nuv pip install bitsandbytes --no-deps\n```\n\n[!IMPORTANT]\nPay attention to the CUDA Toolkit version when installing bitsandbytes. Official bitsandbytes releases are built for specific CUDA Toolkit versions. On Windows x86-64, separate builds are currently provided for CUDA 11.8–12.6 and CUDA 12.8–12.9. Support for NVIDIA RTX 50 Series GPUs (e.g., RTX 5060 Ti, sm_120) requires the CUDA 12.8–12.9 builds.\n\nIf your environment uses an older CUDA version, or you need compatibility with older Windows / PyTorch combinations, you can install the third-party precompiled Windows wheel:\n\n```bash\npip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.2.post2-py3-none-win_amd64.whl\n```\n\n#### Install Flash Attention-2\n\nTo enable FlashAttention-2 on the Windows platform, please use the script from [flash-attention-windows-wheel](https://huggingface.co/lldacing/flash-attention-windows-wheel) to compile and install it by yourself.\n\n</details>\n\n<details><summary>For Ascend NPU users</summary>\n\nTo install LlamaFactory on Ascend NPU devices, plea","github_created_at":"2023-05-28T10:09:12+00:00","created_at":"2026-07-07T17:30:45.162339+00:00","updated_at":"2026-09-20T04:32:08.809202+00:00","categories":[{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"agent","name":"agent"},{"slug":"ai","name":"ai"},{"slug":"deepseek","name":"deepseek"},{"slug":"fine-tuning","name":"fine-tuning"},{"slug":"gpt","name":"gpt"},{"slug":"instruction-tuning","name":"instruction-tuning"},{"slug":"large-language-models","name":"large-language-models"},{"slug":"llama","name":"llama"}],"trust":{"provenance":{"is_fork":false,"github_id":646410686,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-18T08:37:08.390Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":0,"days_since_push":4,"last_release_at":"2026-05-30T15:57:58Z","stars_delta_30d":717,"open_issues_delta_30d":43},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-09-18T08:37:10.632Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-18T08:37:07.264Z"},"has_cli":{"value":true,"source":"pyproject.toml:[project.scripts]","observed_at":"2026-09-18T08:37:07.264Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-09-18T08:37:07.264Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-18T08:37:07.264Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["Hardware requirements vary based on the method and model size. For example, full fine-tuning of a 7B model requires 120GB of memory, while QLoRA with 2-bit can ","Installation is mandatory and can be done from source or via Docker. Additional dependencies like `metrics` and `deepspeed` are optional.","For Windows users, manual installation of PyTorch with CUDA support is required, and bitsandbytes must be installed for QLoRA support."],"min_ram_gb":4,"requires_docker":true},"constraints":{"min_ram_gb":4,"requires_docker":true},"when_to_use":["Use LlamaFactory when you need to fine-tune a wide range of large language models and vision-language models with methods like LoRA, QLoRA, and quantization.","LlamaFactory is ideal for users who require a unified framework for efficient fine-tuning across multiple models, as it supports over 100 different models.","Consider LlamaFactory if you are working with limited hardware resources, as it offers various quantization methods that can significantly reduce memory requirements."],"when_not_to_use":["Avoid LlamaFactory if you are working with models not included in its supported list of over 100 models, as it may not provide the necessary fine-tuning capabilities.","Do not use LlamaFactory if you are working in an environment where Python or specific dependencies like PyTorch and bitsandbytes cannot be installed or are not supported.","LlamaFactory may not be suitable if you require real-time or near-real-time fine-tuning capabilities, as the setup and installation process can be complex and time-consuming."],"source":"enrich:decision_facts","observed_at":"2026-09-18T11:08:14.376Z"},"constraint_facets":{"min_ram_gb":4,"requires_docker":true},"decision_summary":[{"label":"Requirements","value":"Min 4 GB RAM; Requires Docker; Hardware requirements vary based on the method and model size. For example, full fine-tuning of a 7B model requires 120GB of memory, while QLoRA with 2-bit can ; Installation is mandatory and can be done from source or via Docker. Additional dependencies like `metrics` and `deepspeed` are optional.; For Windows users, manual installation of PyTorch with CUDA support is required, and bitsandbytes must be installed for QLoRA support."},{"label":"Adopt for","value":"LlamaFactory is a Python-based tool for efficient fine-tuning of over 100 large language models and vision-language models, supporting various methods like LoRA, QLoRA, and quantization."}]}}