Multimodal Deep Learning Edit Task
Natural Language Processing
About
Benchmarks
| TREND | DATASET | BEST METHOD | PAPER TITLE | PAPER | CODE | COMPARE |
|---|---|---|---|---|---|---|
|
Subtasks
Greatest papers with code
ShapeWorld - A new test methodology for multimodal language understanding
We introduce a novel framework for evaluating multimodal deep learning models with respect to their language understanding and generalization abilities.
Jointly Fine-Tuning “BERT-like” Self Supervised Models to Improve Multimodal Speech Emotion Recognition
Interspeech 2020
• shamanez/BERT-like-is-All-You-Need
• 
Multimodal emotion recognition from speech is an important area in affective computing.
MULTIMODAL DEEP LEARNING MULTIMODAL EMOTION RECOGNITION MULTIMODAL SENTIMENT ANALYSIS SELF-SUPERVISED LEARNING SPEECH EMOTION RECOGNITION TRANSFER LEARNING
Learn to Combine Modalities in Multimodal Deep Learning
• skywaLKer518/MultiplicativeMultimodal
• 
Combining complementary information from multiple modalities is intuitively appealing for improving the performance of learning-based approaches.
More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification
• danfenghong/IEEE_TGRS_MDL-RS
•
In particular, we also investigate a special case of multi-modality learning (MML) -- cross-modality learning (CML) that exists widely in RS image classification applications.
Audio-Conditioned U-Net for Position Estimation in Full Sheet Images
• CPJKU/audio_conditioned_unet
•
The goal of score following is to track a musical performance, usually in the form of audio, in a corresponding score representation.
XFlow: Cross-modal Deep Neural Networks for Audiovisual Classification
• catalina17/XFlow
•
Our work improves on existing multimodal deep learning algorithms in two essential ways: (1) it presents a novel method for performing cross-modality (before features are learned from individual modalities) and (2) extends the previously proposed cross-connections which only transfer information between streams that process compatible data.
Are These Birds Similar: Learning Branched Networks for Fine-grained Representations
• nicolalandro/ntsnet-cub200
•
In recent years, natural language descriptions are used to obtain information on discriminative parts of the object.
Ranked #1 on
Multimodal Deep Learning
on CUB-200-2011
DOCUMENT TEXT CLASSIFICATION FINE-GRAINED IMAGE CLASSIFICATION MULTIMODAL TEXT AND IMAGE CLASSIFICATION
Detecting Hate Speech in Memes Using Multimodal Deep Learning Approaches: Prize-winning solution to Hateful Memes Challenge
Memes on the Internet are often harmless and sometimes amusing.
Ranked #1 on
Meme Classification
on Hateful Memes
(using extra training data)
Multimodal Emotion Recognition with Transformer-Based Self Supervised Feature Fusion
• shamanez/Self-Supervised-Embedding-Fusion-Transformer
•
Emotion Recognition is a challenging research area given its complex nature, and humans express emotional cues across various modalities such as language, facial expressions, and speech.
MULTIMODAL DEEP LEARNING MULTIMODAL EMOTION RECOGNITION MULTIMODAL SENTIMENT ANALYSIS REPRESENTATION LEARNING SELF-SUPERVISED LEARNING
Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response
Multimedia content in social media platforms provides significant information during disaster events.
Ranked #1 on
Disaster Response
on CrisisMMD
DISASTER RESPONSE MULTIMODAL DEEP LEARNING SMALL DATA IMAGE CLASSIFICATION