---
title: "Awesome-Multimodal-Large-Language-Models"
type: "tool"
slug: "bradyfu-awesome-multimodal-large-language-models"
canonical_url: "https://www.graphcanon.com/tools/bradyfu-awesome-multimodal-large-language-models"
github_url: "https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models"
homepage_url: null
stars: 17926
forks: 1129
primary_language: null
license: null
categories: ["llm-frameworks", "developer-tools"]
tags: ["chain-of-thought", "instruction-tuning", "multi-modality", "multimodal-chain-of-thought", "large-language-models", "large-vision-language-models", "in-context-learning", "instruction-following"]
updated_at: "2026-07-07T18:28:11.936915+00:00"
---
# Awesome-Multimodal-Large-Language-Models
> :sparkles::sparkles:Latest Advances on Multimodal Large Language Models
Repository curating the latest research and advancements in multimodal large language models (MLLMs), including surveys, interactive MLLMs like VITA series, and comprehensive evaluation benchmarks. It provides detailed papers and projects, aiming to enhance understanding and development of these models.
## Facts
- Repository: https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models
- Stars: 17,926 · Forks: 1,129 · Open issues: 104 · Watchers: 291
- Last pushed: 2026-07-02T14:38:53+00:00
## Categories
- [LLM Frameworks](/categories/llm-frameworks.md)
- [Developer Tools](/categories/developer-tools.md)
## Tags
chain-of-thought, instruction-tuning, multi-modality, multimodal-chain-of-thought, large-language-models, large-vision-language-models, in-context-learning, instruction-following
## Related tools
- [ECC](/tools/affaan-m-ecc.md) - The agent harness performance optimization system (★ 226,962)
- [AutoGPT](/tools/significant-gravitas-autogpt.md) - AutoGPT: Build, Deploy, and Run AI Agents (★ 185,417)
- [ollama](/tools/ollama-ollama.md) - Get up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. (★ 175,659)
- [prompts.chat](/tools/f-prompts-chat.md) - The world's largest open-source prompt library for AI (★ 165,019)
- [transformers](/tools/huggingface-transformers.md) - 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models (★ 162,347)
- [JavaGuide](/tools/snailclimb-javaguide.md) - Snailclimb/JavaGuide: 面试 & 后端通用面试指南，覆盖计算机基础、数据库、分布式、高并发、系统设计与 AI 应用开发 (★ 156,863)
- [open-webui](/tools/open-webui-open-webui.md) - User-friendly AI Interface (Supports Ollama, OpenAI API, ...) (★ 144,575)
- [awesome-llm-apps](/tools/shubhamsaboo-awesome-llm-apps.md) - 100+ AI Agent & RAG apps you can actually run — clone, customize, ship. (★ 116,702)
## README (excerpt)
```text
# Awesome-Multimodal-Large-Language-Models
## ✨ Highlights of NJU-MiG
> 🔥🔥 **Surveys of MLLMs** | **[💬 WeChat (MLLM微信交流群)](./images/wechat-group.png)**
- 🌟 **MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs**
arXiv 2025, [Paper](https://arxiv.org/pdf/2411.15296.pdf), [Project](https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Benchmarks)
- 🌟 **A Survey of Unified Multimodal Understanding and Generation: Advances and Challenges**
arXiv 2025, [Paper](https://www.techrxiv.org/doi/pdf/10.36227/techrxiv.176289261.16802577), [Project](https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Unified)
- **A Survey on Multimodal Large Language Models**
NSR 2024, [Paper](https://arxiv.org/pdf/2306.13549.pdf), [Project](https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models)
---
> 🔥🔥 **VITA Series Omni MLLMs** | **[💬 WeChat (VITA微信交流群)](https://github.com/VITA-MLLM/VITA/blob/main/asset/wechat-group.jpg)**
- **VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction**
NeurIPS 2025 Highlight, [Paper](https://arxiv.org/pdf/2501.01957.pdf), [Project](https://github.com/VITA-MLLM/VITA)
- **VITA: Towards Open-Source Interactive Omni Multimodal LLM**
arXiv 2024, [Paper](https://arxiv.org/pdf/2408.05211.pdf), [Project](https://vita-home.github.io/)
- **VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient Large Speech-Language Model**
NeurIPS 2025, [Paper](https://arxiv.org/pdf/2505.03739.pdf), [Project](https://github.com/VITA-MLLM/VITA-Audio)
---
> 🔥🔥 **MME Series MLLM Benchmarks**
- 🔥 **Video-MME-v2: Towards the Next Stage in Video Understanding Evaluation**
[[🍎 Project Page](https://video-mme-v2.netlify.app/)] [[📖 Paper](https://arxiv.org/pdf/2604.05015)] [[🤗 Dataset](https://huggingface.co/datasets/MME-Benchmarks/Video-MME-v2)] [[🏆 Leaderboard](https://video-mme-v2.netlify.app/#leaderboard)]
- 🌟 **MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs**
arXiv 2025, [Paper](https://arxiv.org/pdf/2411.15296.pdf), [Project](https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/tree/Benchmarks)
- **MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models**
NeurIPS 2025 DB Highlight, [Paper](https://arxiv.org/pdf/2306.13394.pdf), [Dataset](https://huggingface.co/datasets/lmms-lab/MME), [Eval Tool](https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models/blob/Evaluation/tools/eval_tool.zip), [✒️ Citation](./images/bib_mme.txt)
- **Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis**
CVPR 2025, [Paper](https://arxiv.org/pdf/2405.21075.pdf), [Project](https://video-mme.github.io/), [Dataset](https://github.com/BradyFU/Video-MME?tab=readme-ov-file#-dataset)
---
Table of Contents
- [Awesome Papers](#awesome-papers)
- [Multimodal Instruction Tuning (& Latest Works)](#multimodal-instruction-tuning--latest-works)
- [Multimodal Hallucination](#multimodal-hallucination)
- [Multimodal In-Context Learning](#multimodal-in-context-learning)
- [Multimodal Chain-of-Thought](#multimodal-chain-of-thought)
- [LLM-Aided Visual Reasoning](#llm-aided-visual-reasoning)
- [Foundation Models](#foundation-models)
- [Evaluation](#evaluation)
- [Multimodal RLHF](#multimodal-rlhf)
- [Others](#others)
- [Awesome Datasets](#awesome-datasets)
- [Datasets of Pre-Training for Alignment](#datasets-of-pre-training-for-alignment)
- [Datasets of Multimodal Instruction Tuning](#datasets-of-multimodal-instruction-tuning)
- [Datasets of In-Context Learning](#datasets-of-in-context-learning)
- [Datasets of Multimod
```
---
**Machine-readable endpoints**
- JSON: [`/api/graphcanon/tools/bradyfu-awesome-multimodal-large-language-models`](/api/graphcanon/tools/bradyfu-awesome-multimodal-large-language-models)
- LLM index: [/llms.txt](/llms.txt)
- Full corpus: [/llms-full.txt](/llms-full.txt)
_GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_