---
title: "Video-LLaMA"
type: "tool"
slug: "damo-nlp-sg-video-llama"
canonical_url: "https://www.graphcanon.com/tools/damo-nlp-sg-video-llama"
github_url: "https://github.com/DAMO-NLP-SG/Video-LLaMA"
homepage_url: null
stars: 3142
forks: 286
primary_language: "Python"
license: "BSD-3-Clause"
categories: ["data-retrieval", "llm-frameworks", "inference-serving"]
tags: ["video-language-pretraining", "llama", "blip2", "multi-modal-chatgpt", "cross-modal-pretraining", "large-language-models", "minigpt4", "vision-language-pretraining"]
updated_at: "2026-07-07T18:43:29.338027+00:00"
---

# Video-LLaMA

> Video-LLaMA: Instruction-tuned Audio-Visual Language Model

An instruction-tuned model for understanding videos with audio and visual components, based on LLaMA. This repository hosts Video-LLaMA, a project aimed at enabling large language models to understand video and audio content.

## Facts

- Repository: https://github.com/DAMO-NLP-SG/Video-LLaMA
- Stars: 3,142 · Forks: 286 · Open issues: 69 · Watchers: 31
- Primary language: Python
- License: BSD-3-Clause
- Last pushed: 2024-06-04T07:06:41+00:00

## Categories

- [Data & Retrieval](/categories/data-retrieval.md)
- [LLM Frameworks](/categories/llm-frameworks.md)
- [Inference & Serving](/categories/inference-serving.md)

## Tags

video-language-pretraining, llama, blip2, multi-modal-chatgpt, cross-modal-pretraining, large-language-models, minigpt4, vision-language-pretraining

## Related tools

- [ollama](/tools/ollama-ollama.md) - Get up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. (★ 175,659)
- [prompts.chat](/tools/f-prompts-chat.md) - The world's largest open-source prompt library for AI (★ 165,019)
- [transformers](/tools/huggingface-transformers.md) - 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models (★ 162,347)
- [langflow](/tools/langflow-ai-langflow.md) - Langflow is a powerful platform for building and deploying AI-powered agents and workflows. (★ 151,298)
- [firecrawl](/tools/firecrawl-firecrawl.md) - The API to search, scrape, and interact with the web at scale. (★ 147,117)
- [open-webui](/tools/open-webui-open-webui.md) - User-friendly AI Interface (Supports Ollama, OpenAI API, ...) (★ 144,575)
- [awesome-llm-apps](/tools/shubhamsaboo-awesome-llm-apps.md) - 100+ AI Agent & RAG apps you can actually run — clone, customize, ship. (★ 116,702)
- [LLMs-from-scratch](/tools/rasbt-llms-from-scratch.md) - Implement a ChatGPT-like LLM in PyTorch from scratch (★ 98,711)

## README (excerpt)

```text
<p align="center" width="100%">
<a target="_blank"><img src="figs/video_llama_logo.jpg" alt="Video-LLaMA" style="width: 50%; min-width: 200px; display: block; margin: auto;"></a>
</p>



# Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding


This is the repo for the Video-LLaMA project, which is working on empowering large language models with video and audio understanding capabilities. 

<div style='display:flex; gap: 0.25rem; '>
<a href='https://modelscope.cn/studios/damo/video-llama/summary'><img src='https://img.shields.io/badge/ModelScope-Demo-blueviolet'></a>
<a href='https://www.modelscope.cn/models/damo/videollama_7b_llama2_finetuned/summary'><img src='https://img.shields.io/badge/ModelScope-Checkpoint-blueviolet'></a>
<a href='https://huggingface.co/spaces/DAMO-NLP-SG/Video-LLaMA'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Demo-blue'></a>
<a href='https://huggingface.co/DAMO-NLP-SG/Video-LLaMA-2-7B-Finetuned'><img src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Checkpoint-blue'></a> 
<a href='https://arxiv.org/abs/2306.02858'><img src='https://img.shields.io/badge/Paper-PDF-red'></a>
</div>

## News
- <h3> [2024.06.03] 🚀🚀 We officially launch <a href='https://github.com/DAMO-NLP-SG/VideoLLaMA2'>VideoLLaMA2</a> with stronger performances and easier-to-use codebase, come to try it out!</h3>
- [11.14] ⭐️ The current README file is for **Video-LLaMA-2** (LLaMA-2-Chat as language decoder) only, instructions for using the previous version of Video-LLaMA (Vicuna as language decoder) can be found at [here](https://github.com/DAMO-NLP-SG/Video-LLaMA/blob/main/README_Vicuna.md).
- [08.03] 🚀🚀 Release **Video-LLaMA-2** with [Llama-2-7B/13B-Chat](https://huggingface.co/meta-llama) as language decoder
    - **NO** delta weights and separate Q-former weights anymore, full weights to run Video-LLaMA are all here :point_right: [[7B](https://huggingface.co/DAMO-NLP-SG/Video-LLaMA-2-7B-Finetuned)][[13B](https://huggingface.co/DAMO-NLP-SG/Video-LLaMA-2-13B-Finetuned)] 
    - Allow further customization starting from our pre-trained checkpoints [[7B-Pretrained](https://huggingface.co/DAMO-NLP-SG/Video-LLaMA-2-7B-Pretrained)] [[13B-Pretrained](https://huggingface.co/DAMO-NLP-SG/Video-LLaMA-2-13B-Pretrained)]
- [06.14]  **NOTE**: The current online interactive demo is primarily for English chatting and it may **NOT** be a good option to ask Chinese questions since Vicuna/LLaMA does not represent Chinese texts very well. 
- [06.13]  **NOTE**: The audio support is **ONLY** for Vicuna-7B by now although we have several VL checkpoints available for other decoders.
- [06.10]  **NOTE**: We have NOT updated the HF demo yet because the whole framework (with the audio branch) cannot run normally on A10-24G. The current running demo is still the previous version of Video-LLaMA. We will fix this issue soon.
- [06.08] 🚀🚀 Release the checkpoints of the audio-supported Video-LLaMA. Documentation and example outputs are also updated.    
- [05.22] 🚀🚀 Interactive demo online, try our Video-LLaMA (with **Vicuna-7B** as language decoder) at [Hugging Face](https://huggingface.co/spaces/DAMO-NLP-SG/Video-LLaMA) and [ModelScope](https://pre.modelscope.cn/studios/damo/video-llama/summary)!!
- [05.22] ⭐️ Release **Video-LLaMA v2** built with Vicuna-7B
- [05.18] 🚀🚀 Support video-grounded chat in Chinese 
    - [**Video-LLaMA-BiLLA**](https://huggingface.co/DAMO-NLP-SG/Video-LLaMA-Series/resolve/main/finetune-billa7b-zh.pth): we introduce [BiLLa-7B-SFT](https://huggingface.co/Neutralzz/BiLLa-7B-SFT) as language decoder and fine-tune the video-language aligned model (i.e., stage 1 model) with machine-translated [VideoChat](https://github.com/OpenGVLab/InternVideo/tree/main/Data/instruction_data) instructions.   
    - [**Video-LLaMA-Ziya**](https://huggingface.co/DAMO-NLP-SG/Video-LLaMA-Series/resolve/main/finetune-ziya13b-zh.pth): same with Video-LLaMA-BiLLA but the language deco
```

---

**Machine-readable endpoints**

- JSON: [`/api/graphcanon/tools/damo-nlp-sg-video-llama`](/api/graphcanon/tools/damo-nlp-sg-video-llama)
- LLM index: [/llms.txt](/llms.txt)
- Full corpus: [/llms-full.txt](/llms-full.txt)

_GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_