JIA-Lab-research/MGM-Omni

Leverages modality-specific encoders (audio, video, image, text) feeding into a unified MLLM, with speech output generated via a SpeechLM using Chunk-Based Parallel Decoding and Flow Matching vocoding for streaming synthesis. Handles hour-long speech inputs and generates 10+ minutes of coherent audio, supporting zero-shot voice cloning from ~10-second reference clips in Chinese and English. Built on MiniGemini/Lyra architecture with open-source model weights on Hugging Face and includes Long-TTS-Eval benchmark for long-form speech synthesis evaluation.

265 stars.