GraphCanon updated 2w · GitHub synced 2w
Decision brief
LLaMA-Omni is a specialized multimodal large language model tailored for enhancing speech interaction capabilities through advanced integration of speech-to-speech and speech-to-text functionalities.
Good fit when
- - When targeting low-latency, high-quality end-to-end speech interactions that need to be performed in an academic research environment.
- - If you are working on projects that require integrating both speech and text modalities for processing or generating responses, especially focusing on tasks where reducing latency is critical.
Avoid when
- - Avoid LLaMA-Omni if your project requires commercial deployment since its usage rights are strictly non-commercial.
- - If real-time interaction constraints are less critical than achieving high-quality speech output, another tool with more flexibility regarding latency and deployment options may be preferable.
Observed Jul 10, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (437d since push)
- As of 2w
- Provenance
- Not a fork · Organization account
- As of 2w
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install LLaMA-OmniHow it fits your stack(4)
Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.
Related
Relationship graph
Optional deeper exploration of typed edges and category neighbours.
Similar tools
Same-category neighbours not already linked as typed edges.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
LLaMA-Omni is designed for high-quality and low-latency speech interaction capabilities, aiming to achieve GPT-4o levelspeech functionalities.
Capability facts
- Languages
- python
Source: github.language+pyproject.toml · Jul 30, 2026
Categories
Graph entities
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Jul 30, 2026)
conda create -n llama-omni python=3.10Source link
Tags
README
Install
- Clone this repository.
git clone https://github.com/ictnlp/LLaMA-Omni
cd LLaMA-Omni
- Install packages.
conda create -n llama-omni python=3.10
conda activate llama-omni
pip install pip==24.0
pip install -e .
- Install
fairseq.
git clone https://github.com/pytorch/fairseq
cd fairseq
pip install -e . --no-build-isolation
- Install
flash-attention.
pip install flash-attn --no-build-isolation
Quick Start
-
Download the
Llama-3.1-8B-Omnimodel from 🤗Huggingface. -
Download the
Whisper-large-v3model.
import whisper
model = whisper.load_model("large-v3", download_root="models/speech_encoder/")
- Download the unit-based HiFi-GAN vocoder.
wget https://dl.fbaipublicfiles.com/fairseq/speech_to_speech/vocoder/code_hifigan/mhubert_vp_en_es_fr_it3_400k_layer11_km1000_lj/g_00500000 -P vocoder/
wget https://dl.fbaipublicfiles.com/fairseq/speech_to_speech/vocoder/code_hifigan/mhubert_vp_en_es_fr_it3_400k_layer11_km1000_lj/config.json -P vocoder/
LICENSE
Our code is released under the Apache-2.0 License. Our model is intended for academic research purposes only and may NOT be used for commercial purposes.
You are free to use, modify, and distribute this model in academic settings, provided that the following conditions are met:
- Non-commercial use: The model may not be used for any commercial purposes.
- Citation: If you use this model in your research, please cite the original work.
For agents
This page has a .md twin and JSON over the API.