GraphCanon updated 4d · GitHub synced 4d · 29 views this month
Decision brief
LlamaFactory is a sophisticated tool for fine-tuning numerous large language models and visual language models efficiently using various methods such as LoRA, QLoRA, RLHF, and quantization.
Good fit when
- When you need to fine-tune over 100 different LLMs or VLMs with efficient methods like LoRA or QLoRA.
- For tasks requiring instruction tuning through the use of advanced techniques such as Reinforcement Learning from Human Feedback (RLHF).
Avoid when
- When you are looking to fine-tune less popular or niche models that are not supported within the 100+ models covered by LlamaFactory.
- If your project specifically requires custom fine-tuning methods not available in this repository, such as certain versions of PEFT (Parameter Efficient Fine-Tuning) techniques excluding LoRA and QLoa
Observed Jul 11, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (2d since push)
- As of 4d
- Provenance
- Not a fork · Personal account
- As of 4d
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install LlamaFactoryHow it fits your stack(38)
Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.
Alternative
Integrates
Depends on
Related
Relationship graph
Optional deeper exploration of typed edges and category neighbours.
Similar tools
Same-category neighbours not already linked as typed edges.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
A repository for efficient fine-tuning and instruction tuning of over 100 large language models (LLMs) and visual language models (VLMs). Supports various methods including LoRA, QLoRA, RLHF, and quantization.
Capability facts
- CLI
- CLI entrypoint
Source: pyproject.toml:[project.scripts] · Aug 16, 2026
- Languages
- python
Source: github.language+pyproject.toml · Aug 16, 2026
Categories
Graph entities
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Aug 16, 2026)
This image is built on Ubuntu 22.04 (x86\_64), CUDA 12.4, Python 3.11, PyTorch 2.6.0, and Flash-attn 2.7.4.Source link
Tags
README
Hardware Requirement
* estimated
| Method | Bits | 7B | 14B | 30B | 70B | xB |
|---|---|---|---|---|---|---|
Full (bf16 or fp16) | 32 | 120GB | 240GB | 600GB | 1200GB | 18xGB |
Full (pure_bf16) | 16 | 60GB | 120GB | 300GB | 600GB | 8xGB |
| Freeze/LoRA/GaLore/APOLLO/BAdam/OFT | 16 | 16GB | 32GB | 64GB | 160GB | 2xGB |
| QLoRA / QOFT | 8 | 10GB | 20GB | 40GB | 80GB | xGB |
| QLoRA / QOFT | 4 | 6GB | 12GB | 24GB | 48GB | x/2GB |
| QLoRA / QOFT | 2 | 4GB | 8GB | 16GB | 24GB | x/4GB |
Installation
[!IMPORTANT] Installation is mandatory.
Install from Source
git clone --depth 1 https://github.com/hiyouga/LlamaFactory.git
cd LlamaFactory
pip install -e .
pip install -r requirements/metrics.txt
Optional dependencies available: metrics, deepspeed. Install with: pip install -e . && pip install -r requirements/metrics.txt -r requirements/deepspeed.txt
Additional dependencies for specific features are available in examples/requirements/.
Install from Docker Image
docker run -it --rm --gpus=all --ipc=host hiyouga/llamafactory:latest
This image is built on Ubuntu 22.04 (x86_64), CUDA 12.4, Python 3.11, PyTorch 2.6.0, and Flash-attn 2.7.4.
Find the pre-built images: https://hub.docker.com/r/hiyouga/llamafactory/tags
Please refer to build docker to build the image yourself.
Setting up a virtual environment with uv
Create an isolated Python environment with uv:
uv run llamafactory-cli webui
For Windows users
Install PyTorch
You need to manually install the GPU version of PyTorch on the Windows platform. Please refer to the official website and the following command to install PyTorch with CUDA support:
pip uninstall torch torchvision torchaudio
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126
python -c "import torch; print(torch.cuda.is_available())"
If you see True then you have successfully installed PyTorch with CUDA support.
Try dataloader_num_workers: 0 if you encounter Can't pickle local object error.
Install BitsAndBytes
To enable Quantized LoRA (QLoRA) on Windows, you need to install bitsandbytes.
For most users, it is recommended to install the latest official release:
pip install bitsandbytes
If you are using uv to manage your virtual environment, it is recommended to install bitsandbytes after installing the GPU-enabled version of PyTorch:
uv pip install bitsandbytes --no-deps
[!IMPORTANT] Pay attention to the CUDA Toolkit version when installing bitsandbytes. Official bitsandbytes releases are built for specific CUDA Toolkit versions. On Windows x86-64, separate builds are currently provided for CUDA 11.8–12.6 and CUDA 12.8–12.9. Support for NVIDIA RTX 50 Series GPUs (e.g., RTX 5060 Ti, sm_120) requires the CUDA 12.8–12.9 builds.
If your environment uses an older CUDA version, or you need compatibility with older Windows / PyTorch combinations, you can install the third-party precompiled Windows wheel:
pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.2.post2-py3-none-win_amd64.whl
Install Flash Attention-2
To enable FlashAttention-2 on the Windows platform, please use the script from flash-attention-windows-wheel to compile and install it by yourself.
For Ascend NPU users
To install LlamaFactory on Ascend NPU devices, plea
For agents
This page has a .md twin and JSON over the API.