AudioGPT
AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head
GraphCanon updated 6d · GitHub synced 6d
Decision brief
AudioGPT is a Python-based tool for generating and understanding various audio forms including speech, music, sound effects, and talking head animations using pre-trained models.
Good fit when
- - Utilize AudioGPT when you need to generate speech or music with specific style transfer capabilities using GenerSpeech.
- - Prefer this tool if project requires the latest advancements in text-to-speech generation through FastSpeech, SyntaSpeech, and VITS.
Avoid when
- - Avoid AudioGPT if your audio processing toolkit needs to be exclusively self-contained; some model references are external links requiring separate access.
- - Do not use for projects that absolutely need completed features for all tasks as certain capabilities (speech translation) are still work-in-progress.
Observed Jul 11, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (769d since push)
- As of 6d
- Provenance
- Not a fork · Organization account
- As of 6d
- Security (OSV)
- 137 low (137 low)
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install AudioGPTSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
A repository for generating speech, music, sound, and talking head animations using various pretrained models.
Capability facts
- Languages
- python
Source: github.language · Aug 15, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Aug 15, 2026)
[LangChain](https://github.com/hwchase17/langchain)  Source link
Source: README excerpt (regex_v1, Aug 15, 2026)
[Visual ChatGPT](https://github.com/microsoft/visual-chatgpt)  Source link
Tags
README
AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head
We provide our implementation and pretrained models as open source in this repository.
Get Started
Please refer to run.md
Capabilities
Here we list the capability of AudioGPT at this time. More supported models and tasks are coming soon. For prompt examples, refer to asset.
Currently not every model has repository.
Speech
| Task | Supported Foundation Models | Status |
|---|---|---|
| Text-to-Speech | FastSpeech, SyntaSpeech, VITS | Yes (WIP) |
| Style Transfer | GenerSpeech | Yes |
| Speech Recognition | whisper, Conformer | Yes |
| Speech Enhancement | ConvTasNet | Yes (WIP) |
| Speech Separation | TF-GridNet | Yes (WIP) |
| Speech Translation | Multi-decoder | WIP |
| Mono-to-Binaural | NeuralWarp | Yes |
Sing
| Task | Supported Foundation Models | Status |
|---|---|---|
| Text-to-Sing | DiffSinger, VISinger | Yes (WIP) |
Audio
| Task | Supported Foundation Models | Status |
|---|---|---|
| Text-to-Audio | Make-An-Audio | Yes |
| Audio Inpainting | Make-An-Audio | Yes |
| Image-to-Audio | Make-An-Audio | Yes |
| Sound Detection | Audio-transformer | Yes |
| Target Sound Detection | TSDNet | Yes |
| Sound Extraction | LASSNet | Yes |
Talking Head
| Task | Supported Foundation Models | Status |
|---|---|---|
| Talking Head Synthesis | GeneFace | Yes (WIP) |
Acknowledgement
We appreciate the open source of the following projects:
ESPNet NATSpeech Visual ChatGPT Hugging Face LangChain Stable Diffusion
For agents
This page has a .md twin and JSON over the API.