HorizonWind2004/reconstruction-alignment

Reconstruction Alignment (RecA) applies self-supervised cross-modal reconstruction as an auxiliary task to align visual and semantic features in unified multimodal models, enabling zero-shot improvements without additional labeled data. The approach is architecture-agnostic—validated across Show-o (VQGAN/CLIP), BAGEL, Harmon, OpenUni, and MMaDA—requiring minimal computational overhead (6×A100s, 4.5 hours) while boosting generation quality and image editing capabilities. Models trained with RecA achieve competitive results on GenEval and DPGBench benchmarks, with 1.5B-parameter variants outperforming larger 7B-24B parameter baselines on zero-shot tasks.

378 stars.