microsoft/SpeechT5

Implements an encoder-decoder transformer architecture with unified masked prediction across speech and text modalities, leveraging unpaired data to improve representation learning. Supports diverse downstream tasks including ASR, speech synthesis, speech translation, and multilingual processing through variants like Speech2C, SpeechLM, and VioLA. Integrates with HuggingFace and provides pre-trained checkpoints scaled from base to large models trained on LibriSpeech and Libri-Light corpora.

1,435 stars. No commits in the last 6 months.