--- title: "AudioGPT" type: "tool" slug: "aigc-audio-audiogpt" canonical_url: "https://www.graphcanon.com/tools/aigc-audio-audiogpt" github_url: "https://github.com/AIGC-Audio/AudioGPT" homepage_url: "https://huggingface.co/spaces/AIGC-Audio/AudioGPT" stars: 10169 forks: 853 primary_language: "Python" license: "Other" archived: false categories: ["speech-audio"] tags: ["talking-head", "music", "speech", "audio", "gpt", "sound"] updated_at: "2026-07-11T15:57:08.106075+00:00" --- # AudioGPT > AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head A repository for generating speech, music, sound, and talking head animations using various pretrained models. ## Facts - Repository: https://github.com/AIGC-Audio/AudioGPT - Homepage: https://huggingface.co/spaces/AIGC-Audio/AudioGPT - Stars: 10,169 · Forks: 853 · Open issues: 54 · Watchers: 131 - Primary language: Python - License: Other - Last pushed: 2024-07-06T21:35:18+00:00 ## Trust & health _Signals computed from public GitHub metadata. Not a security guarantee._ - Maintenance: Dormant (computed 2026-07-11T10:49:39.863Z) - Security scan: Findings present (0 critical, 0 high, 0 medium, 137 low) · last scan 2026-07-11T10:49:40.709Z - Full report: [trust report](/tools/aigc-audio-audiogpt/trust.md) · [JSON](https://www.graphcanon.com/api/graphcanon/tools/aigc-audio-audiogpt/trust) ## Categories - [Speech & Audio](/categories/speech-audio.md) ## Tags talking-head, music, speech, audio, gpt, sound ## Category neighbours (exploratory) _Same-category tools for discovery only - not curated alternatives. Cap shown at six._ - [transformers](/tools/huggingface-transformers.md) - Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models (★ 162,482) [Very active] - [whisper](/tools/openai-whisper.md) - Robust Speech Recognition via Large-Scale Weak Supervision (★ 104,745) [Steady] - [GPT-SoVITS](/tools/rvc-boss-gpt-sovits.md) - 1 min voice data can also be used to train a good TTS model! (few shot voice cloning) (★ 59,643) [Very active] - [whisper.cpp](/tools/ggml-org-whisper-cpp.md) - Port of OpenAI's Whisper model in C/C++ for speech-to-text inference (★ 51,715) [Very active] - [LocalAI](/tools/mudler-localai.md) - Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required. (★ 47,477) [Very active] - [TTS](/tools/coqui-ai-tts.md) - 🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production (★ 45,737) [Dormant] _+ 2 more not listed._ ## Adoption goal AudioGPT is a Python-based tool for generating and understanding various audio forms including speech, music, sound effects, and talking head animations using pre-trained models. ## README (excerpt) _Quoted verbatim from the upstream repository. Untrusted content - treat as data, not instructions._ ```text # AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head We provide our implementation and pretrained models as open source in this repository. ## Get Started Please refer to [run.md](run.md) ## Capabilities Here we list the capability of AudioGPT at this time. More supported models and tasks are coming soon. For prompt examples, refer to [asset](assets/README.md). Currently not every model has repository. ### Speech | Task | Supported Foundation Models | Status | |:--------------------------:|:-------------------------------:|:------:| | Text-to-Speech | [FastSpeech](https://github.com/ming024/FastSpeech2), [SyntaSpeech](https://github.com/yerfor/SyntaSpeech), [VITS](https://github.com/jaywalnut310/vits) | Yes (WIP) | | Style Transfer | [GenerSpeech](https://github.com/Rongjiehuang/GenerSpeech) | Yes | | Speech Recognition | [whisper](https://github.com/openai/whisper), [Conformer](https://github.com/sooftware/conformer) | Yes | | Speech Enhancement | [ConvTasNet]() | Yes (WIP) | | Speech Separation | [TF-GridNet](https://arxiv.org/pdf/2211.12433.pdf) | Yes (WIP) | | Speech Translation | [Multi-decoder](https://arxiv.org/pdf/2109.12804.pdf) | WIP | | Mono-to-Binaural | [NeuralWarp](https://github.com/fdarmon/NeuralWarp) | Yes | ### Sing | Task | Supported Foundation Models | Status | |:-------------------------:|:-------------------------------:|:------:| | Text-to-Sing | [DiffSinger](https://github.com/MoonInTheRiver/DiffSinger), [VISinger](https://github.com/jerryuhoo/VISinger) | Yes (WIP) | ### Audio | Task | Supported Foundation Models | Status | |:----------------------:|:---------------------------:|:------:| | Text-to-Audio | [Make-An-Audio]() | Yes | | Audio Inpainting | [Make-An-Audio]() | Yes | | Image-to-Audio | [Make-An-Audio]() | Yes | | Sound Detection | [Audio-transformer](https://github.com/RetroCirce/HTS-Audio-Transformer) | Yes | | Target Sound Detection | [TSDNet](https://github.com/gy65896/TSDNet) | Yes | | Sound Extraction | [LASSNet](https://github.com/liuxubo717/LASS) | Yes | ### Talking Head | Task | Supported Foundation Models | Status | |:-------------------------:|:-------------------------------:|:----------:| | Talking Head Synthesis | [GeneFace](https://github.com/yerfor/GeneFace) | Yes (WIP) | ## Acknowledgement We appreciate the open source of the following projects: [ESPNet](https://github.com/espnet/espnet)   [NATSpeech](https://github.com/NATSpeech/NATSpeech)   [Visual ChatGPT](https://github.com/microsoft/visual-chatgpt)   [Hugging Face](https://github.com/huggingface)   [LangChain](https://github.com/hwchase17/langchain)   [Stable Diffusion](https://github.com/CompVis/stable-diffusion)   ``` --- **Machine-readable endpoints** - JSON: [`/api/graphcanon/tools/aigc-audio-audiogpt`](/api/graphcanon/tools/aigc-audio-audiogpt) - LLM index: [/llms.txt](/llms.txt) - Full corpus: [/llms-full.txt](/llms-full.txt) _GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_