OpenRLHF
OpenRLHF/OpenRLHF
An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray
Overview
OpenRLHF is a production-ready reinforcement learning framework focused on human feedback (RLHF) that combines the Ray distributed computing platform with vLLM architecture. It provides scalable reinforcement learning solutions following a unified agent-based design paradigm.
Categories
Tags
Similar tools
ECC
affaan-m/ECC
The agent harness performance optimization system
hermes-agent
NousResearch/hermes-agent
The self-improving AI agent built by Nous Research
AutoGPT
Significant-Gravitas/AutoGPT
AutoGPT: Build, Deploy, and Run AI Agents
ollama
ollama/ollama
Get up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
transformers
huggingface/transformers
π€ Transformers: the model-definition framework for state-of-the-art machine learning models
langflow
langflow-ai/langflow
Langflow is a powerful platform for building and deploying AI-powered agents and workflows.
Install
pip install OpenRLHFREADME
Open-source / Comprehensive / Lightweight / Easy-to-use
[ English | δΈζ | ζ₯ζ¬θͺ ]
OpenRLHF is the first high-performance, production-ready open-source RLHF framework that combines Ray + vLLM distributed architecture with a unified agent-based design paradigm for scalable and extensible reinforcement learning from human feedback.
π Learn More: Documentation | Slides | Technical Report | Video
π Table of Contents
- ποΈ News
- ποΈ Architecture Foundation - Ray + vLLM + DeepSpeed distributed infrastructure
- π― Design Paradigm - Unified agent-based execution pipeline
- π RL Algorithms - PPO, REINFORCE++, GRPO, RLOO
- π Features Overview - Complete RLHF pipeline capabilities
- π¬ Quick Start - Installation and typical workflow
- π Training Guide - SFT, Reward Model, RL Training
- π― Single-Turn Agent - Custom reward functions
- π€ Multi-Turn Agent - Complex environments
- π§ Advanced Topics - LoRA, performance tuning
News
Show News
- [2026/4] OpenRLHF 0.10 adds Multi-Turn VLM RL β multi-step interactions with images in both prompts and environment feedback (e.g. screenshots). Example: vlm_multiturn_agent.py
- [2026/4] OpenRLHF 0.10 adds VLM (Vision-Language Model) RLHF support β train VLMs like Qwen3.5 with image inputs end-to-end. Training script: train_vlm_math_hybrid_engine.sh
- [2026/2] ProRL V2 uses REINFORCE++-baseline to train a state-of-the-art 1.5B reasoning model with prolonged RL training. Training script: train_prorlv2_math_hybrid_engine.sh
- [2025/10] ScaleRL validates the effectiveness of REINFORCE++-baseline in large-scale training scenarios. Releases [REINFORCE++ slides](https://docs.google.com/present