{"data":{"slug":"aigc-audio-audiogpt","name":"AudioGPT","tagline":"AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head","github_url":"https://github.com/AIGC-Audio/AudioGPT","owner":"AIGC-Audio","repo":"AudioGPT","owner_avatar_url":"https://avatars.githubusercontent.com/u/128504843?v=4","primary_language":"Python","stars":10169,"forks":853,"topics":["audio","gpt","music","sound","speech","talking-head"],"archived":false,"github_pushed_at":"2024-07-06T21:35:18+00:00","maintenance_label":"Dormant","url":"https://www.graphcanon.com/tools/aigc-audio-audiogpt","markdown_url":"https://www.graphcanon.com/tools/aigc-audio-audiogpt.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/aigc-audio-audiogpt","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=aigc-audio-audiogpt","description":"AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head","homepage_url":"https://huggingface.co/spaces/AIGC-Audio/AudioGPT","license":"Other","open_issues":54,"watchers":131,"ai_summary":"A repository for generating speech, music, sound, and talking head animations using various pretrained models.","readme_excerpt":"# AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head\n\n\n\n\n\n\n\nWe provide our implementation and pretrained models as open source in this repository.\n\n\n## Get Started\n\nPlease refer to [run.md](run.md)\n\n\n## Capabilities\n\nHere we list the capability of AudioGPT at this time. More supported models and tasks are coming soon. For prompt examples, refer to [asset](assets/README.md).\n\nCurrently not every model has repository.\n### Speech\n|            Task            |   Supported Foundation Models   | Status |\n|:--------------------------:|:-------------------------------:|:------:|\n|       Text-to-Speech       | [FastSpeech](https://github.com/ming024/FastSpeech2), [SyntaSpeech](https://github.com/yerfor/SyntaSpeech), [VITS](https://github.com/jaywalnut310/vits) |  Yes (WIP)   |\n|       Style Transfer       |         [GenerSpeech](https://github.com/Rongjiehuang/GenerSpeech)         |  Yes   |\n|     Speech Recognition     |           [whisper](https://github.com/openai/whisper), [Conformer](https://github.com/sooftware/conformer)           |  Yes   |\n|     Speech Enhancement     |          [ConvTasNet]()         |  Yes (WIP)   |\n|     Speech Separation      |          [TF-GridNet](https://arxiv.org/pdf/2211.12433.pdf)         |  Yes (WIP)   |\n|     Speech Translation     |          [Multi-decoder](https://arxiv.org/pdf/2109.12804.pdf)      |  WIP   |\n|      Mono-to-Binaural      |          [NeuralWarp](https://github.com/fdarmon/NeuralWarp)         |  Yes   |\n\n### Sing\n\n|           Task            |   Supported Foundation Models   | Status |\n|:-------------------------:|:-------------------------------:|:------:|\n|       Text-to-Sing        |         [DiffSinger](https://github.com/MoonInTheRiver/DiffSinger), [VISinger](https://github.com/jerryuhoo/VISinger)          |  Yes (WIP)   |\n\n### Audio\n|          Task          | Supported Foundation Models | Status |\n|:----------------------:|:---------------------------:|:------:|\n|     Text-to-Audio      |      [Make-An-Audio]()      |  Yes   |\n|    Audio Inpainting    |      [Make-An-Audio]()      |  Yes   |\n|     Image-to-Audio     |      [Make-An-Audio]()      |  Yes   |\n|    Sound Detection     |    [Audio-transformer](https://github.com/RetroCirce/HTS-Audio-Transformer)    | Yes    |\n| Target Sound Detection |    [TSDNet](https://github.com/gy65896/TSDNet)    |  Yes   |\n|    Sound Extraction    |    [LASSNet](https://github.com/liuxubo717/LASS)    |  Yes   |\n\n\n### Talking Head\n\n|           Task            |   Supported Foundation Models   |   Status   |\n|:-------------------------:|:-------------------------------:|:----------:|\n|  Talking Head Synthesis   |          [GeneFace](https://github.com/yerfor/GeneFace)           | Yes (WIP)  |\n\n\n## Acknowledgement\nWe appreciate the open source of the following projects:\n\n[ESPNet](https://github.com/espnet/espnet)  \n[NATSpeech](https://github.com/NATSpeech/NATSpeech)  \n[Visual ChatGPT](https://github.com/microsoft/visual-chatgpt)  \n[Hugging Face](https://github.com/huggingface)  \n[LangChain](https://github.com/hwchase17/langchain)  \n[Stable Diffusion](https://github.com/CompVis/stable-diffusion)  ","github_created_at":"2023-03-16T07:12:18+00:00","created_at":"2026-07-11T10:49:39.199096+00:00","updated_at":"2026-07-11T15:57:08.106075+00:00","categories":[{"slug":"speech-audio","name":"Speech & Audio","url":"https://www.graphcanon.com/categories/speech-audio","markdown_url":"https://www.graphcanon.com/categories/speech-audio.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/speech-audio"}],"tags":[{"slug":"talking-head","name":"talking-head"},{"slug":"music","name":"music"},{"slug":"speech","name":"speech"},{"slug":"audio","name":"audio"},{"slug":"gpt","name":"gpt"},{"slug":"sound","name":"sound"}],"trust":{"provenance":{"is_fork":false,"github_id":614719201,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T10:49:39.863Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":734,"last_release_at":null},"security_summary":{"status":"findings","scanner":"osv@v1","low_count":137,"high_count":0,"last_scan_at":"2026-07-11T10:49:40.709Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T15:56:44.449Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-07-11T15:56:44.449Z"},"license_spdx":{"value":"Other","source":"github.license","observed_at":"2026-07-11T15:56:44.449Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["- Utilize AudioGPT when you need to generate speech or music with specific style transfer capabilities using GenerSpeech.","- Prefer this tool if project requires the latest advancements in text-to-speech generation through FastSpeech, SyntaSpeech, and VITS."],"when_not_to_use":["- Avoid AudioGPT if your audio processing toolkit needs to be exclusively self-contained; some model references are external links requiring separate access.","- Do not use for projects that absolutely need completed features for all tasks as certain capabilities (speech translation) are still work-in-progress."],"source":"enrich:decision_facts","observed_at":"2026-07-11T15:57:07.821Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"AudioGPT is a Python-based tool for generating and understanding various audio forms including speech, music, sound effects, and talking head animations using pre-trained models."}]}}