align-anything

Align-Anything aims to align any modality large models (any-to-any models) with human intentions and values.

• Highly Modular Framework allowing users to easily modify and customize the code for different tasks (see framework design).
• Various Modality Model Fine-Tuning for diverse multi-modal (image/video/audio) models (see scripts).
• Different Alignment Methods. Different alignment algorithms, including SFT, DPO, PPO, and others.
• Multi-Modal CLI. Multi-modal CLI for image, audio, and video modalities (see multi-modal CLI).
• O1-like Training. O1-like training based on DollyTails (see scripts/llama_sft_o1.sh).
• Rule-based RL. Rule-based RL encouraged by Deepseek-R1.

Note: We provide a quick start guide for users to quickly get the code structure and development details.

📣 News

Roadmap

We are actively working on the following features:

• ⚡️ More Models: Integrating cutting-edge models like the Qwen3-VL series.

• 🚀 More Inference Engines: Adding support for high-performance engines like SGLang.

• 🤖 Advanced VLA Algorithms: Implementing more VLA algorithms, including Safe-VLA.

• 🧠 Agent RL: Expanding capabilities to support Agent-based Reinforcement Learning.

• 🛠️ Enhanced RLHF Features: Upgrading our RL training framework with features like asynchronous rollout, vLLM sleep mode, and checkpoint-engine.

Stay tuned for more updates!

• [2025.11.11] 🎉🎉🎉 We now support the alignment fine-tuning of Qwen3 and Qwen3-MoE models!

• [2025.11.11] 🎉🎉🎉 We integrate the InterMT project (NeurIPS 2025 Spotlight) into the main repository, featuring the first multi-turn interleaved preference alignment dataset with human feedback and InterMT-Bench for evaluating multi-turn multimodal interaction capabilities. Check out InterMT for more details.

• [2025.11.11] 🛠️🛠️🛠️ We integrate the eval-anything evaluation framework into the main repository as a dedicated project for large-scale evaluation of any-to-any models. Check out eval-anything for more details.

• [2025.04.14] 📜📜📜 We release the tutorial on SFT training for text-image-to-text models. Check out the cookbook_en (for English) and cookbook_zh (for Chinese).

• [2025.04.07] 🥳🥳🥳 Align-Anything now serves as the homework platform for the PKU course Large Language Models Basics and Alignment, supporting on both Nvidia GPU and Huawei Ascend NPU. The corresponding tutorial will be released soon!

Align-Anything目前已成为北京大学本硕博课程《大模型基础与对齐》的课程作业平台，支持在Nvidia GPU和华为昇腾NPU上进行训练与评估。对应教程将持续发布！

• [2025.03.31] ✅✅✅ We enhance th