{"data":{"slug":"huggingface-trl","name":"trl","tagline":"Train transformer language models with reinforcement learning.","github_url":"https://github.com/huggingface/trl","owner":"huggingface","repo":"trl","owner_avatar_url":"https://avatars.githubusercontent.com/u/25720743?v=4","primary_language":"Python","stars":18786,"forks":2827,"topics":[],"archived":false,"github_pushed_at":"2026-07-07T19:47:32+00:00","url":"https://www.graphcanon.com/tools/huggingface-trl","markdown_url":"https://www.graphcanon.com/tools/huggingface-trl.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/huggingface-trl","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=huggingface-trl","description":"Train transformer language models with reinforcement learning.","homepage_url":"http://hf.co/docs/trl","license":"Apache-2.0","open_issues":393,"watchers":101,"ai_summary":"TRL is a comprehensive library for post-training foundation models using advanced techniques such as Supervised Fine-Tuning (SFT), Group Relative Policy Optimization (GRPO), and Direct Preference Optimization (DPO).","readme_excerpt":"# TRL - Transformers Reinforcement Learning\n\n
\n \n \n \n \n
\n\n \n\n
\n
A comprehensive library to post-train foundation models
\n\n\n
\n \n \n \n \n
\n\n## 🎉 What's New\n\n**TRL v1:** We released TRL v1 — a major milestone that marks a real shift in what TRL is. Read the [blog post](https://huggingface.co/blog/trl-v1) to learn more.\n\n**🚢 Harbor:** We now support [Harbor](https://huggingface.co/docs/trl/harbor) — train agents against sandboxed task suites (instruction + sandbox image + in-sandbox verifier) via [`GRPOTrainer`](https://huggingface.co/docs/trl/grpo_trainer)'s `environment_factory`.\n\n## Overview\n\nTRL is a cutting-edge library designed for post-training foundation models using advanced techniques like Supervised Fine-Tuning (SFT), Group Relative Policy Optimization (GRPO), and Direct Preference Optimization (DPO). Built on top of the [🤗 Transformers](https://github.com/huggingface/transformers) ecosystem, TRL supports a variety of model architectures and modalities, and can be scaled-up across various hardware setups.\n\n## Highlights\n\n- **Trainers**: Various fine-tuning methods are easily accessible via trainers like [`SFTTrainer`](https://huggingface.co/docs/trl/sft_trainer), [`GRPOTrainer`](https://huggingface.co/docs/trl/grpo_trainer), [`DPOTrainer`](https://huggingface.co/docs/trl/dpo_trainer), [`RewardTrainer`](https://huggingface.co/docs/trl/reward_trainer) and more.\n\n- **Efficient and scalable**:\n - Leverages [🤗 Accelerate](https://github.com/huggingface/accelerate) to scale from single GPU to multi-node clusters using methods like [DDP](https://pytorch.org/tutorials/intermediate/ddp_tutorial.html) and [DeepSpeed](https://github.com/deepspeedai/DeepSpeed).\n - Full integration with [🤗 PEFT](https://github.com/huggingface/peft) enables training on large models with modest hardware via quantization and LoRA/QLoRA.\n - Integrates [🦥 Unsloth](https://github.com/unslothai/unsloth) for accelerating training using optimized kernels.\n\n- **Command Line Interface (CLI)**: A simple interface lets you fine-tune with models without needing to write code.\n\n## Installation\n\n### Python Package\n\nInstall the library using `pip`:\n\n```bash\npip install trl\n```\n\n### From source\n\nIf you want to use the latest features before an official release, you can install TRL from source:\n\n```bash\npip install git+https://github.com/huggingface/trl.git\n```\n\n### Repository\n\nIf you want to use the examples you can clone the repository with the following command:\n\n```bash\ngit clone https://github.com/huggingface/trl.git\n```\n\n## Quick Start\n\nFor more flexibility and control over training, TRL provides dedicated trainer classes to post-train language models or PEFT adapters on a custom dataset. Each trainer in TRL is a light wrapper around the 🤗 Transformers trainer and natively supports distributed training methods like DDP, DeepSpeed ZeRO, and FSDP.\n\n### `SFTTrai","github_created_at":"2020-03-27T10:54:55+00:00","created_at":"2026-07-07T22:37:41.074462+00:00","updated_at":"2026-07-07T22:37:44.717729+00:00","categories":[{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"}],"tags":[{"slug":"reinforcement-learning","name":"reinforcement learning"},{"slug":"dpo","name":"dpo"},{"slug":"foundation-models","name":"foundation models"},{"slug":"post-training","name":"post-training"},{"slug":"transformers","name":"transformers"},{"slug":"grpo","name":"grpo"},{"slug":"sft","name":"sft"}]}}
![\"TRL](\"https://huggingface.co/datasets/trl-lib/documentation-images/resolve/main/trl_banner_light.png\")
\n ![\"License\"](\"https://img.shields.io/github/license/huggingface/trl.svg?color=blue\")
\n ![\"Documentation\"](\"https://img.shields.io/website?label=documentation&url=https%3A%2F%2Fhuggingface.co%2Fdocs%2Ftrl%2Findex&down_color=red&down_message=offline&up_color=blue&up_message=online\")
![\"GitHub](\"https://img.shields.io/github/release/huggingface/trl.svg\")
\n ![\"Hugging](\"https://img.shields.io/badge/🤗%20Hub-trl--lib-yellow\"){kind=icon}