Search Results for author:
Found 21 papers, 11 papers with code
NDH-Full: Learning and Evaluating Navigational Agents on Full-Length Dialogue
In this paper, we explore the Navigation from Dialogue History (NDH) task, which is based on the Cooperative Vision-and-Dialogue Navigation (CVDN) dataset, and present a state-of-the-art model which is built upon Vision-Language transformers.
SELMA: Learning and Merging Skill-Specific Text-to-Image Experts with Auto-Generated Data
In this paper, we introduce SELMA: Skill-Specific Expert Learning and Merging with Auto-Generated Data, a novel paradigm to improve the faithfulness of T2I models by fine-tuning models on automatically generated, multi-skill image-text datasets, with skill-specific expert learning and merging.
Analysis of Self-Supervised Speech Models on Children's Speech and Infant Vocalizations
To understand why self-supervised learning (SSL) models have empirically achieved strong performances on several speech-processing downstream tasks, numerous studies have focused on analyzing the encoded information of the SSL layer representations in adult speech.
VLN-Video: Utilizing Driving Videos for Outdoor Vision-and-Language Navigation
Outdoor Vision-and-Language Navigation (VLN) requires an agent to navigate through realistic 3D outdoor environments based on natural language instructions.
Every Node is Different: Dynamically Fusing Self-Supervised Tasks for Attributed Graph Clustering
In this paper, we propose to dynamically learn the weights of SSL tasks for different nodes and fuse the embeddings learned from different SSL tasks to boost performance.
DPATD: Dual-Phase Audio Transformer for Denoising
Recent high-performance transformer-based speech enhancement models demonstrate that time domain methods could achieve similar performance as time-frequency domain methods.
Complex Image Generation SwinTransformer Network for Audio Denoising
Achieving high-performance audio denoising is still a challenging task in real-world applications.
Multimodal Large Language Model for Visual Navigation
Recent efforts to enable visual navigation using large language models have mainly focused on developing complex prompt systems.
Enhancing Child Vocalization Classification in Multi-Channel Child-Adult Conversations Through Wav2vec2 Children ASR Features
In this study, we leverage the self-supervised learning model, Wav2Vec 2. 0 (W2V2), pretrained on 4300h of home recordings of children under 5 years old, to build a unified system that performs both speaker diarization (SD) and vocalization classification (VC) tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Scaling Data Generation in Vision-and-Language Navigation
Recent research in language-guided visual navigation has demonstrated a significant demand for the diversity of traversable environments and the quantity of supervision for training generalizable agents.
Towards Robust Family-Infant Audio Analysis Based on Unsupervised Pretraining of Wav2vec 2.0 on Large-Scale Unlabeled Family Audio
To perform automatic family audio analysis, past studies have collected recordings using phone, video, or audio-only recording devices like LENA, investigated supervised learning methods, and used or fine-tuned general-purpose embeddings learned from large pretrained models.
Improving Vision-and-Language Navigation by Generating Future-View Image Semantics
We then fine-tune the agent on the VLN task with an auxiliary loss that minimizes the difference between the view semantics generated by the agent and the ground truth view semantics of the next step.
BirdSoundsDenoising: Deep Visual Audio Denoising for Bird Sounds
Audio denoising has been explored for decades using both traditional and deep learning-based methods.
CLEAR: Improving Vision-Language Navigation with Cross-Lingual, Environment-Agnostic Representations
Empirically, on the Room-Across-Room dataset, we show that our multilingual agent gets large improvements in all metrics over the strong baseline model when generalizing to unseen environments with the cross-lingual language representation and the environment-agnostic visual representation.
EnvEdit: Environment Editing for Vision-and-Language Navigation
Training on these edit-augmented environments prevents the agent from overfitting to existing environments and helps generalize better to new, unseen environments.
Ranked #2 on
Vision and Language Navigation
on RxR
(using extra training data)
Visualizations of Complex Sequences of Family-Infant Vocalizations Using Bag-of-Audio-Words Approach Based on Wav2vec 2.0 Features
We demonstrate that our high-quality visualizations capture major types of family vocalization interactions, in categories indicative of mental, behavioral, and developmental health, for both labeled and unlabeled LB audio.
Accent-Robust Automatic Speech Recognition Using Supervised and Unsupervised Wav2vec Embeddings
Domain-adversarial training (DAT) and multi-task learning (MTL) are two common approaches for building accent-robust ASR models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Human Attention during Goal-directed Reading Comprehension Relies on Task Optimization
Furthermore, when readers scan a passage without a question in mind, their reading time is predicted by DNNs optimized for a word prediction task.
Exploring the Role of Argument Structure in Online Debate Persuasion
Online debate forums provide users a platform to express their opinions on controversial topics while being exposed to opinions from diverse set of viewpoints.
Bayesian estimation of a semiparametric recurrent event model with applications to the penetrance estimation of multiple primary cancers in Li-Fraumeni Syndrome
We employed a family-wise likelihood that facilitates using genetic information inherited through the family pedigree and properly adjusted for the ascertainment bias that was inevitable in studies of rare diseases by using an inverse probability weighting scheme.
Applications
