zhenye234/xcodec

Combines acoustic encoding with learned semantic representations (HuBERT, WavLM) through dual feature extraction and joint quantization, enabling unified codec tokens that preserve both content and acoustic properties. Integrates seamlessly with Hugging Face Transformers with pre-trained checkpoints optimized for speech (Librispeech, MLS) and general audio domains, supporting straightforward adaptation by injecting semantic modules into existing codecs via projection layers and MSE-based semantic preservation loss.

294 stars.