Search Results for author:
Found 8 papers, 1 papers with code
Video Language Co-Attention with Multimodal Fast-Learning Feature Fusion for VideoQA
We propose the Video Language Co-Attention Network (VLCN) – a novel memory-enhanced model for Video Question Answering (VideoQA).
Gaze-enhanced Crossmodal Embeddings for Emotion Recognition
Emotional expressions are inherently multimodal -- integrating facial behavior, speech, and gaze -- but their automatic recognition is often limited to a single modality, e. g. speech during a phone call.
Multimodal Integration of Human-Like Attention in Visual Question Answering
We present the Multimodal Human-like Attention Network (MULAN) - the first method for multimodal integration of human-like attention on image and text during training of VQA models.
VQA-MHUG: A Gaze Dataset to Study Multimodal Neural Attention in Visual Question Answering
We present VQA-MHUG - a novel 49-participant dataset of multimodal human gaze on both images and questions during visual question answering (VQA) collected using a high-speed eye tracker.
Neural Photofit: Gaze-based Mental Image Reconstruction
The encoder extracts image features and predicts a neural activation map for each face looked at by a human observer.
Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention
A lack of corpora has so far limited advances in integrating human gaze data as a supervisory signal in neural attention mechanisms for natural language processing(NLP).
Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehension
We compare state of the art networks based on long short-term memory (LSTM), convolutional neural models (CNN) and XLNet Transformer architectures.
Comparing Attention-based Convolutional and Recurrent Neural Networks: Success and Limitations in Machine Reading Comprehension
We propose a machine reading comprehension model based on the compare-aggregate framework with two-staged attention that achieves state-of-the-art results on the MovieQA question answering dataset.
