saiteja-talluri/Speech2Face

Uses audio spectrograms and FaceNet embeddings to learn cross-modal associations between speech and facial features, trained on the AVSpeech dataset with face detection and frame extraction pipelines. The model predicts facial embeddings from audio input, evaluated via face retrieval metrics to assess whether the predicted features can identify the correct speaker's face. Supports multi-GPU training with configurable batch processing and includes preprocessing utilities for downloading and preparing audio-video data from YouTube.

178 stars. No commits in the last 6 months.