{"data":{"slug":"jaywalnut310-vits","name":"vits","tagline":"VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech","github_url":"https://github.com/jaywalnut310/vits","owner":"jaywalnut310","repo":"vits","owner_avatar_url":"https://avatars.githubusercontent.com/u/20279210?v=4","primary_language":"Python","stars":7875,"forks":1388,"topics":["deep-learning","pytorch","speech-synthesis","text-to-speech","tts"],"archived":false,"github_pushed_at":"2023-12-06T01:29:50+00:00","maintenance_label":"Dormant","url":"https://www.graphcanon.com/tools/jaywalnut310-vits","markdown_url":"https://www.graphcanon.com/tools/jaywalnut310-vits.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/jaywalnut310-vits","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=jaywalnut310-vits","description":"VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech","homepage_url":"https://jaywalnut310.github.io/vits-demo/index.html","license":"MIT","open_issues":165,"watchers":53,"ai_summary":null,"readme_excerpt":"# VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech\n\n### Jaehyeon Kim, Jungil Kong, and Juhee Son\n\nIn our recent [paper](https://arxiv.org/abs/2106.06103), we propose VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech.\n\nSeveral recent end-to-end text-to-speech (TTS) models enabling single-stage training and parallel sampling have been proposed, but their sample quality does not match that of two-stage TTS systems. In this work, we present a parallel end-to-end TTS method that generates more natural sounding audio than current two-stage models. Our method adopts variational inference augmented with normalizing flows and an adversarial training process, which improves the expressive power of generative modeling. We also propose a stochastic duration predictor to synthesize speech with diverse rhythms from input text. With the uncertainty modeling over latent variables and the stochastic duration predictor, our method expresses the natural one-to-many relationship in which a text input can be spoken in multiple ways with different pitches and rhythms. A subjective human evaluation (mean opinion score, or MOS) on the LJ Speech, a single speaker dataset, shows that our method outperforms the best publicly available TTS systems and achieves a MOS comparable to ground truth.\n\nVisit our [demo](https://jaywalnut310.github.io/vits-demo/index.html) for audio samples.\n\nWe also provide the [pretrained models](https://drive.google.com/drive/folders/1ksarh-cJf3F5eKJjLVWY0X1j1qsQqiS2?usp=sharing).\n\n** Update note: Thanks to [Rishikesh (ऋषिकेश)](https://github.com/jaywalnut310/vits/issues/1), our interactive TTS demo is now available on [Colab Notebook](https://colab.research.google.com/drive/1CO61pZizDj7en71NQG_aqqKdGaA_SaBf?usp=sharing).\n\n<table style=\"width:100%\">\n  <tr>\n    <th>VITS at training</th>\n    <th>VITS at inference</th>\n  </tr>\n  <tr>\n    <td><img src=\"resources/fig_1a.png\" alt=\"VITS at training\" height=\"400\"></td>\n    <td><img src=\"resources/fig_1b.png\" alt=\"VITS at inference\" height=\"400\"></td>\n  </tr>\n</table>\n\n\n## Pre-requisites\n0. Python >= 3.6\n0. Clone this repository\n0. Install python requirements. Please refer [requirements.txt](requirements.txt)\n    1. You may need to install espeak first: `apt-get install espeak`\n0. Download datasets\n    1. Download and extract the LJ Speech dataset, then rename or create a link to the dataset folder: `ln -s /path/to/LJSpeech-1.1/wavs DUMMY1`\n    1. For mult-speaker setting, download and extract the VCTK dataset, and downsample wav files to 22050 Hz. Then rename or create a link to the dataset folder: `ln -s /path/to/VCTK-Corpus/downsampled_wavs DUMMY2`\n0. Build Monotonic Alignment Search and run preprocessing if you use your own datasets.\n```sh\n# Cython-version Monotonoic Alignment Search\ncd monotonic_align\npython setup.py build_ext --inplace\n\n# Preprocessing (g2p) for your own datasets. Preprocessed phonemes for LJ Speech and VCTK have been already provided.\n# python preprocess.py --text_index 1 --filelists filelists/ljs_audio_text_train_filelist.txt filelists/ljs_audio_text_val_filelist.txt filelists/ljs_audio_text_test_filelist.txt \n# python preprocess.py --text_index 2 --filelists filelists/vctk_audio_sid_text_train_filelist.txt filelists/vctk_audio_sid_text_val_filelist.txt filelists/vctk_audio_sid_text_test_filelist.txt\n```\n\n\n## Training Exmaple\n```sh\n# LJ Speech\npython train.py -c configs/ljs_base.json -m ljs_base\n\n# VCTK\npython train_ms.py -c configs/vctk_base.json -m vctk_base\n```\n\n\n## Inference Example\nSee [inference.ipynb](inference.ipynb)","github_created_at":"2021-05-26T23:38:12+00:00","created_at":"2026-07-11T12:05:47.177548+00:00","updated_at":"2026-07-11T12:06:06.015223+00:00","categories":[{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"},{"slug":"speech-audio","name":"Speech & Audio","url":"https://www.graphcanon.com/categories/speech-audio","markdown_url":"https://www.graphcanon.com/categories/speech-audio.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/speech-audio"},{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"}],"tags":[{"slug":"deep-learning","name":"deep-learning"},{"slug":"text-to-speech","name":"text-to-speech"},{"slug":"python","name":"python"},{"slug":"tts","name":"tts"},{"slug":"pytorch","name":"pytorch"},{"slug":"speech-synthesis","name":"speech-synthesis"}],"trust":{"provenance":{"is_fork":false,"github_id":371194369,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T12:05:47.858Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":948,"last_release_at":null},"security_summary":{"status":"findings","scanner":"osv@v1","low_count":37,"high_count":0,"last_scan_at":"2026-07-11T12:05:51.952Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T12:05:51.594Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-07-11T12:05:51.594Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-07-11T12:05:51.594Z"}}}}