ToucanTTSa massively multilingual model covering over 7,000 languages
AI Voice CloneTry out ToucanTTS with the following examples.
ToucanTTS is a toolkit developed by the Institute for Natural Language Processing (IMS) at the University of Stuttgart, Germany, for teaching, training, and using state-of-the-art speech synthesis models. It is built entirely in Python and PyTorch, aiming to be simple, beginner-friendly, yet powerful.
Supports multilingual speech synthesis covering over 7,000 languages through a massively multilingual pretrained model. Enables multi-speaker speech synthesis and cloning of prosody (rhythm, stress, intonation) across speakers
Allows human-in-the-loop editing of synthesized speech, e.g., for poetry reading and literary studies
Provides interactive demos for massively multilingual speech synthesis, style cloning across speakers, voice design, and human-edited poetry reading
Primarily based on the FastSpeech 2 architecture with modifications like a normalizing flow-based PostNet inspired by PortaSpeech. Includes a self-contained aligner trained with Connectionist Temporal Classification (CTC) and spectrogram reconstruction for various applications.Offers pretrained models for the multilingual model, aligner, embedding function, vocoder, and embedding GAN
Built entirely in Python and PyTorch, aiming to be simple and beginner-friendly while still powerful
The IMS Toucan system incorporates articulatory representations of phonemes as input, allowing multilingual data to benefit low-resource languages
Let's get started with ToucanTTS in just a few simple steps.
Clone the IMS-Toucan repository from GitHub:
git clone https://github.com/DigitalPhonetics/IMS-ToucanWrite a function mapping audio paths to transcripts, create a custom training pipeline script, and ensure text frontend language support.
Download pretrained models like the multilingual model, aligner, and vocoder using the provided script.
Apply any necessary patches or fixes to the codebase.
Run your custom training pipeline script to fine-tune the models on your datas
Use provided interactive demos or scripts to generate speech from text using the trained models.
Have a question? Check out some of the common queries below.
ToucanTTS is primarily based on the FastSpeech 2 architecture with modifications like a normalizing flow-based PostNet inspired by PortaSpeech.
ToucanTTS incorporates articulatory representations of phonemes as input, allowing multilingual data to benefit low-resource languages.
Yes, ToucanTTS enables multi-speaker speech synthesis and cloning of prosody (rhythm, stress, intonation) across speakers.
ToucanTTS provides interactive demos for massively multilingual speech synthesis, style cloning across speakers, voice design, and human-edited poetry reading.
The massively multilingual pretrained model in ToucanTTS covers over 7,000 languages.
Yes, ToucanTTS is built entirely in Python and PyTorch, aiming to be simple and beginner-friendly while still powerful.