{"data":{"slug":"verl-project-verl","name":"verl","tagline":"A Flexible and Efficient RL Post-Training Framework","github_url":"https://github.com/verl-project/verl","owner":"verl-project","repo":"verl","owner_avatar_url":"https://avatars.githubusercontent.com/u/212961691?v=4","primary_language":"Python","stars":22425,"forks":4201,"topics":[],"archived":false,"github_pushed_at":"2026-07-10T17:35:02+00:00","maintenance_label":"Very active","url":"https://www.graphcanon.com/tools/verl-project-verl","markdown_url":"https://www.graphcanon.com/tools/verl-project-verl.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/verl-project-verl","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=verl-project-verl","description":"verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework ","homepage_url":"https://verl.readthedocs.io/en/latest/index.html","license":"Apache-2.0","open_issues":1576,"watchers":89,"ai_summary":"verl/HybridFlow is a Python framework designed for flexible and efficient reinforcement learning (RL) post-training, offering algorithms like PPO and GRPO with detailed documentation and reproducible baselines for coding and math tasks. It includes components like Ray trainer, model engine, and support for advanced usage such as adding models using FSDP or Megatron-LM backends.","readme_excerpt":"## Getting Started\n\n<a href=\"https://verl.readthedocs.io/en/latest/index.html\"><b>Documentation</b></a>\n\n**Quickstart:**\n\n- [Installation](https://verl.readthedocs.io/en/latest/start/install.html)\n- [Quickstart](https://verl.readthedocs.io/en/latest/start/quickstart.html)\n- [Programming Guide](https://verl.readthedocs.io/en/latest/hybrid_flow.html) & [Tech Talk](https://hcqnc.xetlk.com/sl/3vACOK) (in Chinese)\n- [PPO in verl](https://verl.readthedocs.io/en/latest/algo/ppo.html)\n- [GRPO in verl](https://verl.readthedocs.io/en/latest/algo/grpo.html)\n\n**Running a PPO example step-by-step:**\n\n- [Prepare Data for Post-Training](https://verl.readthedocs.io/en/latest/preparation/prepare_data.html)\n- [Implement Reward Function for Dataset](https://verl.readthedocs.io/en/latest/preparation/reward_function.html)\n- [PPO Example Architecture](https://verl.readthedocs.io/en/latest/examples/ppo_code_architecture.html)\n- [Config Explanation](https://verl.readthedocs.io/en/latest/examples/config.html)\n\n**Reproducible algorithm baselines:**\n\n- [RL performance on coding, math](https://verl.readthedocs.io/en/latest/algo/baseline.html)\n\n**Algorithm recipes (`recipe/`):**\n\n- Optional workflows and baselines live under [`recipe/`](recipe/). Each recipe subdirectory includes a small **`REQUIRED_VERL.txt`** file describing the intended `verl` install: pinned recipes use a **tag or fixed git SHA**; rolling recipes record an explicit **`VERL_COMMIT`** (and related submodule / recipe-folder SHAs) so you can `pip install verl@git+…@<sha>` without guessing. See [`recipe/README.md`](recipe/README.md) for the full index and links.\n\n**For code explanation and advance usage (extension):**\n\n- PPO Trainer and Workers\n\n  - [PPO Ray Trainer](https://verl.readthedocs.io/en/latest/workers/ray_trainer.html)\n  - [Model Engine](https://verl.readthedocs.io/en/latest/workers/model_engine.html)\n  - [Engine Workers (FSDP / Megatron-LM / Automodel / VeOmni / TorchTitan)](https://verl.readthedocs.io/en/latest/workers/engine_workers.html)\n\n- Advanced Usage and Extension\n  - [Add Models with the FSDP Backend](https://verl.readthedocs.io/en/latest/advance/fsdp_extension.html)\n  - [Add Models with the Megatron-LM Backend](https://verl.readthedocs.io/en/latest/advance/megatron_extension.html)\n  - [Multi-turn Rollout Support](https://verl.readthedocs.io/en/latest/sglang_multiturn/multiturn.html)\n  - [Search Tool Integration](https://verl.readthedocs.io/en/latest/sglang_multiturn/search_tool_example.html)\n  - [Sandbox Fusion Integration](https://verl.readthedocs.io/en/latest/examples/sandbox_fusion_example.html)\n  - [Extend to Other RL(HF) algorithms](https://verl.readthedocs.io/en/latest/advance/dpo_extension.html)\n  - [Ray API design tutorial](https://verl.readthedocs.io/en/latest/advance/placement.html)\n\n**Blogs from the community**\n\n- [When Reasoning Models Break Tokenization: The Hidden Complexity of Multiturn Training](https://github.com/zhaochenyang20/Awesome-ML-SYS-Tutorial/blob/main/rlhf/verl/multi-turn/fast_tokenization/multiturn_tokenization_and_masking.md)\n- [verl deployment on AWS SageMaker](https://medium.com/@kaige.yang0110/run-verl-on-sagemaker-using-4x8-l40s-gpus-8e6d5c3c61d3)\n- [verl x SGLang Multi-turn Code Walkthrough](https://github.com/zhaochenyang20/Awesome-ML-SYS-Tutorial/blob/main/rlhf/verl/multi-turn/code-walk-through/readme_EN.md)\n- [Optimizing SGLang Memory Usage in verl](https://hebiao064.github.io/rl-memory-management)\n- [SGLang, verl, OpenBMB and Tsinghua University: Pioneering End-to-End Multi-Turn RLHF](https://github.com/zhaochenyang20/Awesome-ML-SYS-Tutorial/blob/main/rlhf/verl/multi-turn/verl-multiturn-rollout-Release.md)\n- [Reinforcement Learning from Human Feedback on AMD GPUs with verl and ROCm Integration](https://rocm.blogs.amd.com/artificial-intelligence/verl-large-scale/README.html)\n- [veMLP x verl ：玩转强化学习训练](https://mp.weixin.qq.com/s/7nbqxk4knMGd-hQE9ls2tA)\n- [使用 verl 进行 GRPO 分布式强化学习训练最佳实践](https://www.volcengine.com/docs/6459/1463942)\n-","github_created_at":"2024-10-31T06:11:15+00:00","created_at":"2026-07-11T10:36:00.019269+00:00","updated_at":"2026-07-11T14:00:22.096607+00:00","categories":[{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"reinforcement-learning","name":"reinforcement-learning"},{"slug":"ppo","name":"ppo"},{"slug":"python","name":"python"},{"slug":"rl","name":"rl"},{"slug":"post-training","name":"post-training"},{"slug":"grpo","name":"grpo"}],"trust":{"provenance":{"is_fork":false,"github_id":881221486,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T10:36:00.693Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":1,"days_since_push":0,"last_release_at":"2026-06-01T11:24:51Z"},"security_summary":{"status":"findings","scanner":"osv@v1","low_count":2,"high_count":0,"last_scan_at":"2026-07-11T10:36:01.538Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T13:59:39.679Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-07-11T13:59:39.679Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-07-11T13:59:39.679Z"}},"decision_facts":{"hosting":null,"pricing":{"model":"freemium","summary":"verl operates under the Apache-2.0 license and is free and open-source. However, you might incur costs associated with cloud services like AWS SageMaker if you plan to deploy large-scale projects on a"},"requirements":{"notes":["Ensure your development environment supports Python and the backend systems you intend to use (FSDP or Megatron-LM)."],"min_ram_gb":8,"requires_docker":false},"constraints":{"min_ram_gb":8,"pricing_model":"freemium","requires_docker":false},"when_to_use":["Opt for verl if your project requires flexibility in integrating advanced backend systems like FSDP or Megatron-LM to extend RL model capabilities.","Utilize when you need a framework that is rigorously documented, supporting reproducible baselines for both coding and math tasks, ensuring consistency across experiments."],"when_not_to_use":["Avoid verl if your project does not require advanced backend integration with systems like FSDP or Megatron-LM; it might be overkill and introduce unnecessary complexity.","Do not use if detailed documentation is less important to your workflow. While verl excels in this area, simpler frameworks may suffice for lighter requirements."],"source":"enrich:decision_facts","observed_at":"2026-07-11T14:00:21.739Z"},"constraint_facets":{"min_ram_gb":8,"pricing_model":"freemium","requires_docker":false},"decision_summary":[{"label":"Pricing","value":"freemium - verl operates under the Apache-2.0 license and is free and open-source. However, you might incur costs associated with cloud services like AWS SageMaker if you plan to deploy large-scale projects on a"},{"label":"Requirements","value":"Min 8 GB RAM; Ensure your development environment supports Python and the backend systems you intend to use (FSDP or Megatron-LM)."},{"label":"Adopt for","value":"verl/HybridFlow is a specialized Python framework for post-training reinforcement learning (RL) that provides detailed documentation and reproducible baselines. It supports PPO and GRPO algorithms and includes Ray Trains"}]}}