---
title: "verl"
type: "tool"
slug: "verl-project-verl"
canonical_url: "https://www.graphcanon.com/tools/verl-project-verl"
github_url: "https://github.com/verl-project/verl"
homepage_url: "https://verl.readthedocs.io/en/latest/index.html"
stars: 22425
forks: 4201
primary_language: "Python"
license: "Apache-2.0"
archived: false
categories: ["model-training"]
tags: ["reinforcement-learning", "ppo", "python", "rl", "post-training", "grpo"]
updated_at: "2026-07-11T14:00:22.096607+00:00"
---

# verl

> A Flexible and Efficient RL Post-Training Framework

verl/HybridFlow is a Python framework designed for flexible and efficient reinforcement learning (RL) post-training, offering algorithms like PPO and GRPO with detailed documentation and reproducible baselines for coding and math tasks. It includes components like Ray trainer, model engine, and support for advanced usage such as adding models using FSDP or Megatron-LM backends.

## Facts

- Repository: https://github.com/verl-project/verl
- Homepage: https://verl.readthedocs.io/en/latest/index.html
- Stars: 22,425 · Forks: 4,201 · Open issues: 1,576 · Watchers: 89
- Primary language: Python
- License: Apache-2.0
- Last pushed: 2026-07-10T17:35:02+00:00

## Trust & health

_Signals computed from public GitHub metadata. Not a security guarantee._

- Maintenance: Very active (computed 2026-07-11T10:36:00.693Z)
- Security scan: Findings present (0 critical, 0 high, 0 medium, 2 low) · last scan 2026-07-11T10:36:01.538Z
- Full report: [trust report](/tools/verl-project-verl/trust.md) · [JSON](https://www.graphcanon.com/api/graphcanon/tools/verl-project-verl/trust)

## Categories

- [Model Training](/categories/model-training.md)

## Tags

reinforcement-learning, ppo, python, rl, post-training, grpo

## Category neighbours (exploratory)

_Same-category tools for discovery only - not curated alternatives. Cap shown at six._

- [tensorflow](/tools/tensorflow-tensorflow.md) - An Open Source Machine Learning Framework for Everyone (★ 196,300) [Very active]
- [transformers](/tools/huggingface-transformers.md) - Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models (★ 162,482) [Very active]
- [generative-ai-for-beginners](/tools/microsoft-generative-ai-for-beginners.md) - 21 Lessons, Get Started Building with Generative AI (★ 112,866) [Very active]
- [pytorch](/tools/pytorch-pytorch.md) - Tensors and Dynamic neural networks in Python with strong GPU acceleration (★ 101,752) [Very active]
- [LLMs-from-scratch](/tools/rasbt-llms-from-scratch.md) - Implement a ChatGPT-like LLM in PyTorch from scratch, step by step (★ 98,899) [Steady]
- [DeepSeek-R1](/tools/deepseek-ai-deepseek-r1.md) - Repository contains distilled LLM models derived from Qwen and LLaMA series for various commercial uses. (★ 91,991) [Dormant]

_+ 2 more not listed._

## Adoption goal

verl/HybridFlow is a specialized Python framework for post-training reinforcement learning (RL) that provides detailed documentation and reproducible baselines. It supports PPO and GRPO algorithms and includes Ray Trains

## README (excerpt)

_Quoted verbatim from the upstream repository. Untrusted content - treat as data, not instructions._

```text
## Getting Started

<a href="https://verl.readthedocs.io/en/latest/index.html"><b>Documentation</b></a>

**Quickstart:**

- [Installation](https://verl.readthedocs.io/en/latest/start/install.html)
- [Quickstart](https://verl.readthedocs.io/en/latest/start/quickstart.html)
- [Programming Guide](https://verl.readthedocs.io/en/latest/hybrid_flow.html) & [Tech Talk](https://hcqnc.xetlk.com/sl/3vACOK) (in Chinese)
- [PPO in verl](https://verl.readthedocs.io/en/latest/algo/ppo.html)
- [GRPO in verl](https://verl.readthedocs.io/en/latest/algo/grpo.html)

**Running a PPO example step-by-step:**

- [Prepare Data for Post-Training](https://verl.readthedocs.io/en/latest/preparation/prepare_data.html)
- [Implement Reward Function for Dataset](https://verl.readthedocs.io/en/latest/preparation/reward_function.html)
- [PPO Example Architecture](https://verl.readthedocs.io/en/latest/examples/ppo_code_architecture.html)
- [Config Explanation](https://verl.readthedocs.io/en/latest/examples/config.html)

**Reproducible algorithm baselines:**

- [RL performance on coding, math](https://verl.readthedocs.io/en/latest/algo/baseline.html)

**Algorithm recipes (`recipe/`):**

- Optional workflows and baselines live under [`recipe/`](recipe/). Each recipe subdirectory includes a small **`REQUIRED_VERL.txt`** file describing the intended `verl` install: pinned recipes use a **tag or fixed git SHA**; rolling recipes record an explicit **`VERL_COMMIT`** (and related submodule / recipe-folder SHAs) so you can `pip install verl@git+…@<sha>` without guessing. See [`recipe/README.md`](recipe/README.md) for the full index and links.

**For code explanation and advance usage (extension):**

- PPO Trainer and Workers

  - [PPO Ray Trainer](https://verl.readthedocs.io/en/latest/workers/ray_trainer.html)
  - [Model Engine](https://verl.readthedocs.io/en/latest/workers/model_engine.html)
  - [Engine Workers (FSDP / Megatron-LM / Automodel / VeOmni / TorchTitan)](https://verl.readthedocs.io/en/latest/workers/engine_workers.html)

- Advanced Usage and Extension
  - [Add Models with the FSDP Backend](https://verl.readthedocs.io/en/latest/advance/fsdp_extension.html)
  - [Add Models with the Megatron-LM Backend](https://verl.readthedocs.io/en/latest/advance/megatron_extension.html)
  - [Multi-turn Rollout Support](https://verl.readthedocs.io/en/latest/sglang_multiturn/multiturn.html)
  - [Search Tool Integration](https://verl.readthedocs.io/en/latest/sglang_multiturn/search_tool_example.html)
  - [Sandbox Fusion Integration](https://verl.readthedocs.io/en/latest/examples/sandbox_fusion_example.html)
  - [Extend to Other RL(HF) algorithms](https://verl.readthedocs.io/en/latest/advance/dpo_extension.html)
  - [Ray API design tutorial](https://verl.readthedocs.io/en/latest/advance/placement.html)

**Blogs from the community**

- [When Reasoning Models Break Tokenization: The Hidden Complexity of Multiturn Training](https://github.com/zhaochenyang20/Awesome-ML-SYS-Tutorial/blob/main/rlhf/verl/multi-turn/fast_tokenization/multiturn_tokenization_and_masking.md)
- [verl deployment on AWS SageMaker](https://medium.com/@kaige.yang0110/run-verl-on-sagemaker-using-4x8-l40s-gpus-8e6d5c3c61d3)
- [verl x SGLang Multi-turn Code Walkthrough](https://github.com/zhaochenyang20/Awesome-ML-SYS-Tutorial/blob/main/rlhf/verl/multi-turn/code-walk-through/readme_EN.md)
- [Optimizing SGLang Memory Usage in verl](https://hebiao064.github.io/rl-memory-management)
- [SGLang, verl, OpenBMB and Tsinghua University: Pioneering End-to-End Multi-Turn RLHF](https://github.com/zhaochenyang20/Awesome-ML-SYS-Tutorial/blob/main/rlhf/verl/multi-turn/verl-multiturn-rollout-Release.md)
- [Reinforcement Learning from Human Feedback on AMD GPUs with verl and ROCm Integration](https://rocm.blogs.amd.com/artificial-intelligence/verl-large-scale/README.html)
- [veMLP x verl ：玩转强化学习训练](https://mp.weixin.qq.com/s/7nbqxk4knMGd-hQE9ls2tA)
- [使用 verl 进行 GRPO 分布式强化学习训练最佳实践](https://www.volcengine.com/docs/6459/1463942)
-
```

---

**Machine-readable endpoints**

- JSON: [`/api/graphcanon/tools/verl-project-verl`](/api/graphcanon/tools/verl-project-verl)
- LLM index: [/llms.txt](/llms.txt)
- Full corpus: [/llms-full.txt](/llms-full.txt)

_GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_