declare-lab/jamify

Implements rectified flow diffusion with word and phoneme-level timing control via a compact 530M-parameter DiT backbone, enabling precise vocal prosody specification in lyrics-to-song generation. Achieves 3× lower phoneme/word error rates through phoneme boundary attention and incorporates Direct Preference Optimization using synthetic preference datasets for aesthetic alignment without manual annotation. Distributes inference across multi-GPU setups via Hugging Face Accelerate and supports both reference audio style extraction and text prompts for controllable generation up to 3m50s duration.

154 stars. No commits in the last 6 months.