HiBorn4/TensorFusion_Network_for_Multimodal_sentiment_analysis
This repository implements the Tensor Fusion Network (TFN) for multimodal sentiment analysis using the CMU-MOSI dataset. TFN integrates language, visual, and acoustic modalities to predict sentiment intensity, enhancing sentiment prediction accuracy by modeling unimodal, bimodal, and trimodal interactions.
The implementation uses LSTM-based modality embedding subnetworks for language (GloVe vectors), visual (FACET/OpenFace facial features), and acoustic (COVAREP) streams, which feed into a tensor fusion layer that explicitly computes three-fold Cartesian products to capture unimodal, bimodal, and trimodal feature interactions. The architecture culminates in a fully connected inference network supporting binary/five-class classification and regression tasks on the 2,199-utterance CMU-MOSI dataset, with ablation studies demonstrating that trimodal dynamics modeling is critical for performance gains.
No commits in the last 6 months.
Stars
13
Forks
5
Language
Jupyter Notebook
License
—
Category
Last pushed
May 21, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/HiBorn4/TensorFusion_Network_for_Multimodal_sentiment_analysis"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
dermatologist/crisp-t
CRISP-T: AI assisted Qualitative Research with vibe analytics!
GeorgiosIoannouCoder/tessa
Text Emotion System Sentiment Analysis (TESSA) is an open-source project focused on sentiment analysis. 💬
nainiayoub/emotion-classifier-web-app
Logistic regression, text emotion classifier web application (with Streamlit), from data...
jhabaa/Scikit-Learn-Sandbox
Vibe coded ML model
Mohshaikh23/Text-Emotions-Classification
Text emotions classification by natural language processing and text classification