jdh-algo/JoyHallo
JoyHallo: Digital human model for Mandarin
Implements audio-driven video synthesis with a semi-decoupled architecture that decouples lip, expression, and pose features to improve efficiency and cross-lingual capability. Uses Chinese wav2vec2 for Mandarin audio embedding and integrates Stable Diffusion with motion modules for frame generation, achieving 14.3% faster inference than the base Hallo model. Supports both Mandarin and English video generation while maintaining strong cross-language performance on the proprietary 29-hour jdh-Hallo dataset.
522 stars. No commits in the last 6 months.
Stars
522
Forks
51
Language
Python
License
MIT
Category
Last pushed
Sep 21, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/generative-ai/jdh-algo/JoyHallo"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
OpenVGLab/OmniLottie
[CVPR 2026🔥] 🧑🎨 OmniLottie, an open-sourced multi-modal instructed vector animation generator...
Mrkomiljon/awesome-generative-ai
Multimodal generative AI resources : talking heads, STT, TTS, image & video generation, and more.
NVIDIA/Maya-ACE
Maya-ACE: A Reference Client Implementation for NVIDIA ACE Audio2Face Service
michaelzhang-ai/Speech2Video
ACCV 2020 "Speech2Video Synthesis with 3D Skeleton Regularization and Expressive Body Poses"
Boese0601/Dyadic-Interaction-Modeling
[ECCV 2024] Dyadic Interaction Modeling for Social Behavior Generation