Video-LLaMA
Instruction-tuned Audio-Visual Language Model for Video Understanding
GraphCanon updated 3d · GitHub synced 3d
Decision brief
Video-LLaMA is an audio-visual language model that enhances video and audio understanding capabilities for language models.
Good fit when
- When you need to process video content with instruction-tuned multimodal capabilities, especially when working with videos that require both visual and auditory analysis.
- For applications requiring a deep understanding of video content in Chinese, as Video-LLaMA includes support for video-grounded chat in this language.
Avoid when
- Do not use when the primary focus is on languages other than English and Chinese, as the model's representation capabilities outside these languages might be limited.
- Avoid using Video-LLaMA if you require real-time audio processing in a deployment environment that does not support Vicuna-7B audio branch currently running on A10-24G GPUs.
- Requirements:
- Ensure access to compatible hardware for video and audio processing tasks.; Consider the availability of Chinese text representation as a potential advantage or limitation based on your project needs.
Observed Jul 14, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (804d since push)
- As of 3d
- Provenance
- Not a fork · Organization account
- As of 3d
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install Video-LLaMAHow it fits your stack(8)
Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.
Alternative
Integrates
Relationship graph
Optional deeper exploration of typed edges and category neighbours.
llama.cpp123k★
LlamaFactory74k★
Qwen22k★
Chinese-LLaMA-Alpaca19k★
ollama178k★
suna20k★
transformers164k★
tensorflow197k★
vllm88k★llama.cpp123k★LlamaFactory74k★Qwen22k★Chinese-LLaMA-Alpaca19k★ollama178k★suna20k★transformers164k★tensorflow197k★vllm88k★Similar tools
Same-category neighbours not already linked as typed edges.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
This repository focuses on enhancing large language models with capabilities to understand video and audio content.
Capability facts
- Languages
- python
Source: github.language · Aug 18, 2026
Categories
Graph entities
Tags
README
Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
This is the repo for the Video-LLaMA project, which is working on empowering large language models with video and audio understanding capabilities.
News
-
[2024.06.03] 🚀🚀 We officially launch VideoLLaMA2 with stronger performances and easier-to-use codebase, come to try it out!
- [11.14] ⭐️ The current README file is for Video-LLaMA-2 (LLaMA-2-Chat as language decoder) only, instructions for using the previous version of Video-LLaMA (Vicuna as language decoder) can be found at here.
- [08.03] 🚀🚀 Release Video-LLaMA-2 with Llama-2-7B/13B-Chat as language decoder
- NO delta weights and separate Q-former weights anymore, full weights to run Video-LLaMA are all here :point_right: [7B][13B]
- Allow further customization starting from our pre-trained checkpoints [7B-Pretrained] [13B-Pretrained]
- [06.14] NOTE: The current online interactive demo is primarily for English chatting and it may NOT be a good option to ask Chinese questions since Vicuna/LLaMA does not represent Chinese texts very well.
- [06.13] NOTE: The audio support is ONLY for Vicuna-7B by now although we have several VL checkpoints available for other decoders.
- [06.10] NOTE: We have NOT updated the HF demo yet because the whole framework (with the audio branch) cannot run normally on A10-24G. The current running demo is still the previous version of Video-LLaMA. We will fix this issue soon.
- [06.08] 🚀🚀 Release the checkpoints of the audio-supported Video-LLaMA. Documentation and example outputs are also updated.
- [05.22] 🚀🚀 Interactive demo online, try our Video-LLaMA (with Vicuna-7B as language decoder) at Hugging Face and ModelScope!!
- [05.22] ⭐️ Release Video-LLaMA v2 built with Vicuna-7B
- [05.18] 🚀🚀 Support video-grounded chat in Chinese
- Video-LLaMA-BiLLA: we introduce BiLLa-7B-SFT as language decoder and fine-tune the video-language aligned model (i.e., stage 1 model) with machine-translated VideoChat instructions.
- Video-LLaMA-Ziya: same with Video-LLaMA-BiLLA but the language deco
For agents
This page has a .md twin and JSON over the API.