---
title: "align-anything"
type: "tool"
slug: "pku-alignment-align-anything"
canonical_url: "https://www.graphcanon.com/tools/pku-alignment-align-anything"
github_url: "https://github.com/PKU-Alignment/align-anything"
homepage_url: null
stars: 4663
forks: 505
primary_language: "Python"
license: "Apache-2.0"
categories: ["llm-frameworks", "model-training"]
tags: ["rlhf", "vision-language-model", "large-language-models", "chameleon", "dpo", "multimodal"]
updated_at: "2026-07-07T18:40:43.465215+00:00"
---
# align-anything
> Align Anything: Training All-modality Model with Feedback
Align-Anything is a highly modular framework designed for aligning large multi-modal models with human intentions and values through various alignment methods and support for diverse data modalities.
## Facts
- Repository: https://github.com/PKU-Alignment/align-anything
- Stars: 4,663 · Forks: 505 · Open issues: 32 · Watchers: 266
- Primary language: Python
- License: Apache-2.0
- Last pushed: 2025-11-27T06:50:36+00:00
## Categories
- [LLM Frameworks](/categories/llm-frameworks.md)
- [Model Training](/categories/model-training.md)
## Tags
rlhf, vision-language-model, large language models, chameleon, dpo, multimodal
## Related tools
- [ollama](/tools/ollama-ollama.md) - Get up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. (★ 175,659)
- [prompts.chat](/tools/f-prompts-chat.md) - The world's largest open-source prompt library for AI (★ 165,019)
- [transformers](/tools/huggingface-transformers.md) - 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models (★ 162,347)
- [open-webui](/tools/open-webui-open-webui.md) - User-friendly AI Interface (Supports Ollama, OpenAI API, ...) (★ 144,575)
- [awesome-llm-apps](/tools/shubhamsaboo-awesome-llm-apps.md) - 100+ AI Agent & RAG apps you can actually run — clone, customize, ship. (★ 116,702)
- [LLMs-from-scratch](/tools/rasbt-llms-from-scratch.md) - Implement a ChatGPT-like LLM in PyTorch from scratch (★ 98,711)
- [TradingAgents](/tools/tauricresearch-tradingagents.md) - TradingAgents: Multi-Agents LLM Financial Trading Framework (★ 91,610)
- [caveman](/tools/juliusbrussee-caveman.md) - Cuts 65% of tokens in AI coding agent responses. (★ 86,150)
## README (excerpt)
```text
[📘Documentation](https://align-anything.readthedocs.io/) |
[🛠️Quick Start](#quick-start) |
[🚀Algorithms](#algorithms) |
[👀Evaluation](./projects/eval-anything) |
[🤔Reporting Issues](#report-issues)
[Our All-Modality Alignment Datasets](https://huggingface.co/datasets/PKU-Alignment/align-anything)
Align-Anything aims to align any modality large models (any-to-any models) with human intentions and values.
- **Highly Modular Framework** allowing users to easily modify and customize the code for different tasks (see [framework design](https://align-anything.readthedocs.io/)).
- **Various Modality Model Fine-Tuning** for diverse multi-modal (image/video/audio) models (see [scripts](./scripts)).
- **Different Alignment Methods.** Different alignment algorithms, including SFT, DPO, PPO, and others.
- **Multi-Modal CLI.** Multi-modal CLI for image, audio, and video modalities (see [multi-modal CLI](#multi-modal-cli)).
- **O1-like Training.** O1-like training based on [DollyTails](https://huggingface.co/datasets/PKU-Alignment/DollyTails-12K) (see [scripts/llama_sft_o1.sh](./scripts)).
- **Rule-based RL.** Rule-based RL encouraged by [Deepseek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1).
**Note:** We provide a [quick start guide](https://align-anything.readthedocs.io/) for users to quickly get the code structure and development details.
## 📣 News
### Roadmap
We are actively working on the following features:
- ⚡️ **More Models:** Integrating cutting-edge models like the Qwen3-VL series.
- 🚀 **More Inference Engines:** Adding support for high-performance engines like SGLang.
- 🤖 **Advanced VLA Algorithms:** Implementing more VLA algorithms, including Safe-VLA.
- 🧠 **Agent RL:** Expanding capabilities to support Agent-based Reinforcement Learning.
- 🛠️ **Enhanced RLHF Features:** Upgrading our RL training framework with features like asynchronous rollout, vLLM sleep mode, and checkpoint-engine.
Stay tuned for more updates!
- **[2025.11.11]** 🎉🎉🎉 We now support the alignment fine-tuning of Qwen3 and Qwen3-MoE models!
- **[2025.11.11]** 🎉🎉🎉 We integrate the **InterMT** project (NeurIPS 2025 Spotlight) into the main repository, featuring the first multi-turn interleaved preference alignment dataset with human feedback and InterMT-Bench for evaluating multi-turn multimodal interaction capabilities. Check out [InterMT](./projects/InterMT) for more details.
- **[2025.11.11]** 🛠️🛠️🛠️ We integrate the **eval-anything** evaluation framework into the main repository as a dedicated project for large-scale evaluation of any-to-any models. Check out [eval-anything](./projects/eval-anything) for more details.
- **[2025.04.14]** 📜📜📜 We release the tutorial on SFT training for `text-image-to-text` models. Check out the [cookbook_en](./cookbooks/en/text_image_to_text_sft.ipynb) (for English) and [cookbook_zh](./cookbooks/zh/text_image_to_text_sft.ipynb) (for Chinese).
- **[2025.04.07]** 🥳🥳🥳 Align-Anything now serves as the homework platform for the PKU course [Large Language Models Basics and Alignment](https://pku-llm.ai/), supporting on both Nvidia GPU and Huawei Ascend NPU. The corresponding tutorial will be released soon!
> Align-Anything目前已成为北京大学本硕博课程《大模型基础与对齐》的课程作业平台，支持在Nvidia GPU和华为昇腾NPU上进行训练与评估。对应教程将持续发布！
- **[2025.03.31]** ✅✅✅ We enhance th
```
---
**Machine-readable endpoints**
- JSON: [`/api/graphcanon/tools/pku-alignment-align-anything`](/api/graphcanon/tools/pku-alignment-align-anything)
- LLM index: [/llms.txt](/llms.txt)
- Full corpus: [/llms-full.txt](/llms-full.txt)
_GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_