lucasnewman/best-rq-pytorch

Combines mel-spectrogram feature extraction with a Conformer encoder and random projection quantizer to learn discrete semantic tokens from unlabeled audio. The pretraining pipeline includes masking-based self-supervision (~60% mask probability) and supports downstream applications like TTS and vocoding. Integrates with RVQ (Residual Vector Quantization) codebooks and outputs can feed into models like Spear-TTS or SoundStorm for speech synthesis.

133 stars. No commits in the last 6 months. Available on PyPI.