Home/LLM Frameworks/LLM-RLHF-Tuning

LLM-RLHF-Tuning

Joyce94/LLM-RLHF-Tuning

LLM Tuning with PEFT (SFT+RM+PPO+DPO with LoRA)

GitHub

GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026

52views this month

452 stars24 forksLast push Oct 11, 2023 Python

Decision brief

Framework for tuning large language models with PEFT & LoRA techniques like SFT, RM, PPO, DPO.

Good fit when

When you need to fine-tune LLMS using PEFT methods such as SFT+RM+PPO+DPO alongside LoRA.
If looking for a flexible framework supporting various tuning approaches with large models.

Avoid when

Avoid if your project only requires basic finetuning without the need for advanced techniques like PEFT or LoRA.
Not suitable if you require a tool that supports other specific fine-tuning methods not covered by this framework.

Observed Jul 12, 2026 · Source: enrich:decision_facts

LLM-RLHF-Tuning alternatives Trust report

Verify the decision

Maintenance and security

Full trust report

Maintenance: Dormant (1074d since push); As of Sep 19, 2026
Provenance: Not a fork · Personal account; As of Sep 19, 2026
Security (OSV): No lockfile; As of Jul 11, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

pip install LLM-RLHF-Tuning

PyPI

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

peft

State-of-the-art Parameter-Efficient Fine-Tuning

22k★

trl

Train transformer language models with reinforcement learning

19k★

alpaca-lora

Instruct-tune LLaMA on consumer hardware

19k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

Provides a framework for tuning large language models using Partially Frozen Efficient Fine-Tuning techniques including SFT, RM, PPO, DPO alongside LoRA.

Capability facts

Languages: python
Source: github.language · Sep 19, 2026

README

LLM RLHF Tuning 本项目从零实现了RLHF三阶段训练，并在文档中详细写了实现细节，欢迎大家交流讨论WeChat 主要内容：支持指令微调Alpaca模型支持训练Reward模型支持PPO算法训练RL模型支持基于两个基模型，两个lora的适配器，同时加载RM、SFT、Actor、Critic四个模型，支持accelerate分布式训练（PPO算法实现细节）支持基于一个基模型，两个lora适配器，同时加载RM、SFT、Actor、Critic四个模型，支持accelerate、deepspeed训练支持基于一个基模型，一个lora适...

For agents

This page has a .md twin and JSON over the API.

/tools/joyce94-llm-rlhf-tuning.md GET /api/graphcanon/tools/joyce94-llm-rlhf-tuning GET /api/graphcanon/graph?tool=joyce94-llm-rlhf-tuning