Search Results for author:
Found 104 papers, 27 papers with code
Improving Embedding-based Large-scale Retrieval via Label Enhancement
Current embedding-based large-scale retrieval models are trained with 0-1 hard label that indicates whether a query is relevant to a document, ignoring rich information of the relevance degree.
WANDR: Intention-guided Human Motion Generation
To address this, we introduce WANDR, a data-driven model that takes an avatar's initial pose and a goal's 3D position and generates natural human motions that place the end effector (wrist) on the goal location.
FilterPrompt: Guiding Image Transfer in Diffusion Models
Achieving this requires the effective decoupling of key attributes within the input image data, aiming to get representations accurately.
Analysis of Classifier-Free Guidance Weight Schedulers
Classifier-Free Guidance (CFG) enhances the quality and condition adherence of text-to-image diffusion models.
Gaze-Guided Graph Neural Network for Action Anticipation Conditioned on Intention
We introduce the Gaze-guided Action Anticipation algorithm, which establishes a visual-semantic graph from the video input.
A Transformer-Based Model for the Prediction of Human Gaze Behavior on Videos
Eye-tracking applications that utilize the human gaze in video understanding tasks have become increasingly important.
A Directional Diffusion Graph Transformer for Recommendation
To account for and model possible noise in the users' interactions in graph neural recommenders, we propose a novel Diffusion Graph Transformer (DiffGT) model for top-k recommendation.
I-Design: Personalized LLM Interior Designer
Interior design allows us to be who we are and live how we want - each design is as unique as our distinct personality.
What Do You See in Vehicle? Comprehensive Vision Solution for In-Vehicle Gaze Estimation
GazeDPTR shows state-of-the-art performance on the IVGaze dataset.
Characteristic AI Agents via Large Language Models
In response to this research gap, we create a benchmark for the characteristic AI agents task, including dataset, techniques, and evaluation metrics.
Model Will Tell: Training Membership Inference for Diffusion Models
In this paper, we explore a novel perspective for the TMI task by leveraging the intrinsic generative priors within the diffusion model.
Noise Level Adaptive Diffusion Model for Robust Reconstruction of Accelerated MRI
In general, diffusion model-based MRI reconstruction methods incrementally remove artificially added noise while imposing data consistency to reconstruct the underlying images.
Batch size invariant Adam
We propose a batch size invariant version of Adam, for use in large-scale, distributed settings, in which the mini-batch is divided into micro-batches which are distributed among worker nodes.
Joint Beamforming Design and Stream Allocation for Non-Coherent Joint Transmission in Cell-Free MIMO Networks
We then propose a joint beamforming and linear stream allocation algorithm, termed as RWMMSE-LSA, which yields closed-form updates with linear stream allocation complexity and is guaranteed to converge to the stationary points of the original joint optimization problem.
OpenSUN3D: 1st Workshop Challenge on Open-Vocabulary 3D Scene Understanding
This report provides an overview of the challenge hosted at the OpenSUN3D Workshop on Open-Vocabulary 3D Scene Understanding held in conjunction with ICCV 2023.
Multi-scale Spatio-temporal Transformer-based Imbalanced Longitudinal Learning for Glaucoma Forecasting from Irregular Time Series Images
Extensive experiments on the Sequential fundus Images for Glaucoma Forecast (SIGF) dataset demonstrate the superiority of the proposed MST-former method, achieving an AUC of 98. 6% for glaucoma forecasting.
BlackJAX: Composable Bayesian inference in JAX
BlackJAX is a library implementing sampling and variational inference algorithms commonly used in Bayesian computation.
Natural Language User Profiles for Transparent and Scrutable Recommendations
This lack of transparency not only limits user understanding of why certain items are suggested but also reduces the user's ability to easily scrutinize and edit their preferences.
Clarifying the Path to User Satisfaction: An Investigation into Clarification Usefulness
Clarifying questions are an integral component of modern information retrieval systems, directly impacting user satisfaction and overall system performance.
SWEA: Updating Factual Knowledge in Large Language Models via Subject Word Embedding Altering
In particular, local editing methods, which directly update model parameters, are more suitable for updating a small amount of knowledge.
An Empirical Investigation of Domain Adaptation Ability for Chinese Spelling Check Models
Chinese Spelling Check (CSC) is a meaningful task in the area of Natural Language Processing (NLP) which aims at detecting spelling errors in Chinese texts and then correcting these errors.
Cross-modality Guidance-aided Multi-modal Learning with Dual Attention for MRI Brain Tumor Grading
Brain tumor represents one of the most fatal cancers around the world, and is very common in children and the elderly.
RHOBIN Challenge: Reconstruction of Human Object Interaction
Modeling the interaction between humans and objects has been an emerging research direction in recent years.
Medical Report Generation based on Segment-Enhanced Contrastive Representation Learning
Automated radiology report generation has the potential to improve radiology reporting and alleviate the workload of radiologists.
StyleSpeech: Self-supervised Style Enhancing with VQ-VAE-based Pre-training for Expressive Audiobook Speech Synthesis
Both objective and subjective evaluations demonstrate that our proposed method can effectively improve the naturalness and expressiveness of the synthesized speech in audiobook synthesis especially for the role and out-of-domain scenarios.
G-MEMP: Gaze-Enhanced Multimodal Ego-Motion Prediction in Driving
In contrast, we focus on inferring the ego trajectory of a driver's vehicle using their gaze data.
Decoupling Degradation and Content Processing for Adverse Weather Image Restoration
The former facilitates channel-dependent degradation removal operation, allowing the network to tailor responses to various adverse weather types; the latter, by integrating Fourier's global properties into channel-independent content features, enhances network capacity for consistent global content reconstruction.
LALM: Long-Term Action Anticipation with Language Models
Understanding human activity is a crucial yet intricate task in egocentric vision, a field that focuses on capturing visual perspectives from the camera wearer's viewpoint.
Optimizing and Fine-tuning Large Language Model for Urban Renewal
This study demonstrates the effectiveness and superiority of the joint fine-tuning method using Prefix and LoRA for ChatGLM in the urban renewal knowledge QA tasks.
A Social-aware Gaussian Pre-trained Model for Effective Cold-start Recommendation
We propose a novel recommendation model, the Social-aware Gaussian Pre-trained model (SGP), which encodes the user social relations and interaction data at the pre-training stage in a Graph Neural Network (GNN).
Model-aware 3D Eye Gaze from Weak and Few-shot Supervisions
In this work, we propose to predict 3D eye gaze from weak supervision of eye semantic segmentation masks and direct supervision of a few 3D gaze vectors.
Improving Conversational Recommendation Systems via Bias Analysis and Language-Model-Enhanced Data Augmentation
Conversational Recommendation System (CRS) is a rapidly growing research area that has gained significant attention alongside advancements in language modelling techniques.
LoRA ensembles for large language model fine-tuning
To address these issues, we propose an ensemble approach using Low-Rank Adapters (LoRA), a parameter-efficient fine-tuning technique.
OSM-Net: One-to-Many One-shot Talking Head Generation with Spontaneous Head Motions
Other works construct one-to-one mapping between audio signal and head motion sequences, introducing ambiguity correspondences into the mapping since people can behave differently in head motions when speaking the same content.
Extragradient Type Methods for Riemannian Variational Inequality Problems
In the context of Euclidean space, it is established that the last-iterates of both the extragradient (EG) and past extragradient (PEG) methods converge to the solution of monotone variational inequality problems at a rate of $O\left(\frac{1}{\sqrt{T}}\right)$ (Cai et al., 2022).
Selecting which Dense Retriever to use for Zero-Shot Search
We propose the new problem of choosing which dense retrieval model to use when searching on a new collection for which no labels are available, i. e. in a zero-shot setting.
Large-Scale Automatic Audiobook Creation
In this work, we present a system that can automatically generate high-quality audiobooks from online e-books.
BluNF: Blueprint Neural Field
BluNF provides a robust and user-friendly 2D blueprint, enabling intuitive scene editing.
MuLanTTS: The Microsoft Speech Synthesis System for Blizzard Challenge 2023
In this paper, we present MuLanTTS, the Microsoft end-to-end neural text-to-speech (TTS) system designed for the Blizzard Challenge 2023.
MFR-Net: Multi-faceted Responsive Listening Head Generation via Denoising Diffusion Model
Responsive listening head generation is an important task that aims to model face-to-face communication scenarios by generating a listener head video given a speaker video and a listener head image.
Bayesian Low-rank Adaptation for Large Language Models
Low-rank adaptation (LoRA) has emerged as a new paradigm for cost-efficient fine-tuning of large language models (LLMs).
Natural Language Instructions for Intuitive Human Interaction with Robotic Assistants in Field Construction Work
When introducing HRC in construction, it is critical to recognize the importance of teamwork and supervision in field construction and establish a natural and intuitive communication system for the human workers and robotic assistants.
ContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph Reading
Experiments show that ContextSpeech significantly improves the voice quality and prosody expressiveness in paragraph reading with competitive model efficiency.
Palm: Predicting Actions through Language Models @ Ego4D Long-Term Action Anticipation Challenge 2023
We present Palm, a solution to the Long-Term Action Anticipation (LTA) task utilizing vision-language and large language models.
Online Distillation for Pseudo-Relevance Feedback
Model distillation has emerged as a prominent technique to improve neural search models.
Self Contrastive Learning for Session-based Recommendation
Session-based recommendation, which aims to predict the next item of users' interest as per an existing sequence interaction of items, has attracted growing applications of Contrastive Learning (CL) with improved user and item representations.
A Survey on Asking Clarification Questions Datasets in Conversational Systems
The ability to understand a user's underlying needs is critical for conversational systems, especially with limited input from users in a conversation.
EFE: End-to-end Frame-to-Gaze Estimation
In this paper, we propose a frame-to-gaze network that directly predicts both 3D gaze origin and 3D gaze direction from the raw frame out of the camera without any face or eye cropping.
When and What to Ask Through World States and Text Instructions: IGLU NLP Challenge Solution
We move towards this target with two sub-tasks, a classification task and a ranking task.
Deep Learning in Breast Cancer Imaging: A Decade of Progress and Future Directions
Breast cancer has reached the highest incidence rate worldwide among all malignancies since 2020.
FONT: Flow-guided One-shot Talking Head Generation with Natural Head Motions
Specifically, the head pose prediction module is designed to generate head pose sequences from the source face and driving audio.
JAWS: Just A Wild Shot for Cinematic Transfer in Neural Radiance Fields
This paper presents JAWS, an optimization-driven approach that achieves the robust transfer of visual cinematic features from a reference in-the-wild video clip to a newly generated clip.
SpatialFormer: Semantic and Target Aware Attentions for Few-Shot Learning
Then we derive two specific attention modules, named SpatialFormer Semantic Attention (SFSA) and SpatialFormer Target Attention (SFTA), to enhance the target object regions while reduce the background distraction.
INO at Factify 2: Structure Coherence based Multi-Modal Fact Verification
Therefore, we proposed a structure coherence-based multi-modal fact verification scheme to classify fake news.
OPT: One-shot Pose-Controllable Talking Head Generation
To solve the identity mismatch problem and achieve high-quality free pose control, we present One-shot Pose-controllable Talking head generation network (OPT).
Human Fall Detection- Multimodality Approach
Falls have become more frequent in recent years, which has been harmful for senior citizens. Therefore detecting falls have become important and several data sets and machine learning model have been introduced related to fall detection.
Task2KB: A Public Task-Oriented Knowledge Base
Search engines and conversational assistants are commonly used to help users complete their every day tasks such as booking travel, cooking, etc.
Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipation
This task requires an understanding of the spatio-temporal context formed by past actions on objects, coined action context.
Structure-guided Image Outpainting
we propose a deep learning method based on Generative Adversarial Network (GAN) and condition edges as structural prior in order to assist the generation.
GazeNeRF: 3D-Aware Gaze Redirection with Neural Radiance Fields
We propose GazeNeRF, a 3D-aware method for the task of gaze redirection.
Particle-based Variational Inference with Preconditioned Functional Gradient Flow
With the popularity of Stein variational gradient descent (SVGD), the focus of particle-based VI algorithms has been on the properties of functions in Reproducing Kernel Hilbert Space (RKHS) to approximate the gradient flow.
Global Meets Local: Effective Multi-Label Image Classification via Category-Aware Weak Supervision
This paper builds a unified framework to perform effective noisy-proposal suppression and to interact between global and local features for robust feature learning.
A survey on the development status and application prospects of knowledge graph in smart grids
With the advent of the electric power big data era, semantic interoperability and interconnection of power data have received extensive attention.
Rethinking the Metric in Few-shot Learning: From an Adaptive Multi-Distance Perspective
In this paper, for the first time, we investigate the contributions of different distance metrics, and propose an adaptive fusion scheme, bringing significant improvements in few-shot classification.
MetaASSIST: Robust Dialogue State Tracking with Meta Learning
Experimental results demonstrate that all three schemes can achieve competitive performance.
Joint control variate for faster black-box variational inference
Black-box variational inference performance is sometimes hindered by the use of gradient estimators with high variance.
Causal Intervention for Fairness in Multi-behavior Recommendation
Specifically, we find that: 1) item popularity is a confounder between the exposed items and users' post-click interactions, leading to the first unfairness; and 2) some hidden confounders (e. g., the reputation of item producers) affect both item popularity and quality, resulting in the second unfairness.
Reconstructing Action-Conditioned Human-Object Interactions Using Commonsense Knowledge Priors
We further qualitatively evaluate the effectiveness of our method on real images and demonstrate its generalizability towards interaction types and object categories.
Decoupling Classifier for Boosting Few-shot Object Detection and Instance Segmentation
This paper focus on few-shot object detection~(FSOD) and instance segmentation~(FSIS), which requires a model to quickly adapt to novel classes with a few labeled instances.
Ranked #3 on
Few-Shot Object Detection
on MS-COCO (1-shot)
Large-scale Knowledge Distillation with Elastic Heterogeneous Computing Resources
Although more layers and more parameters generally improve the accuracy of the models, such big models generally have high computational complexity and require big memory, which exceed the capacity of small devices for inference and incurs long training time.
Rethinking Persistent Homology for Visual Recognition
This paper performs a detailed analysis of the effectiveness of topological properties for image classification in various training scenarios, defined by: the number of training samples, the complexity of the training data and the complexity of the backbone network.
Self-supervised Context-aware Style Representation for Expressive Speech Synthesis
In this paper, we propose a novel framework for learning style representation from abundant plain text in a self-supervised manner.
Robustness to corruption in pre-trained Bayesian neural networks
We develop ShiftMatch, a new training-data-dependent likelihood for robustness to corruption in Bayesian neural networks (BNNs).
Deepfake Caricatures: Amplifying attention to artifacts increases deepfake detection by humans and machines
Second, they allow us to generate novel "Deepfake Caricatures": transformations of the deepfake that exacerbate artifacts to improve human detection.
PrEF: Percolation-based Evolutionary Framework for the diffusion-source-localization problem in large networks
We assume that the state of a number of nodes in a network could be investigated if necessary, and study what configuration of those nodes could facilitate a better solution for the diffusion-source-localization (DSL) problem.
NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality
In this paper, we answer these questions by first defining the human-level quality based on the statistical significance of subjective measure and introducing appropriate guidelines to judge it, and then developing a TTS system called NaturalSpeech that achieves human-level quality on a benchmark dataset.
Ranked #1 on
Text-To-Speech Synthesis
on LJSpeech
(using extra training data)
Sparse Local Patch Transformer for Robust Face Alignment and Landmarks Inherent Relation Learning
The SLPT generates the representation of each single landmark from a local patch and aggregates them by an adaptive inherent relation based on the attention mechanism.
Ranked #2 on
Face Alignment
on COFW-68
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
A unified framework named ERNIE 3. 0 was recently proposed for pre-training large-scale knowledge enhanced models and trained a model with 10 billion parameters.
EPNet++: Cascade Bi-directional Fusion for Multi-Modal 3D Object Detection
Recently, fusing the LiDAR point cloud and camera image to improve the performance and robustness of 3D object detection has received more and more attention, as these two modalities naturally possess strong complementarity.
No-regret Online Learning over Riemannian Manifolds
We consider online optimization over Riemannian manifolds, where a learner attempts to minimize a sequence of time-varying loss functions defined on Riemannian manifolds.
Advancing COVID-19 Diagnosis with Privacy-Preserving Collaboration in Artificial Intelligence
Artificial intelligence (AI) provides a promising substitution for streamlining COVID-19 diagnoses.
Monolithic Integrated Multiband Acoustic Devices on Heterogeneous Substrate for Sub-6 GHz RF-FEMs
Monolithic integration of multiband (1. 4~ 6. 0 GHz) RF acoustic devices were successfully demonstrated within the same process flow by using the lithium niobate (LN) thin film on silicon carbide (LNOSiC) substrate.
LATFormer: Locality-Aware Point-View Fusion Transformer for 3D Shape Recognition
To investigate this, we propose a novel Locality-Aware Point-View Fusion Transformer (LATFormer) for 3D shape retrieval and classification.
PeCLR: Self-Supervised 3D Hand Pose Estimation from monocular RGB via Equivariant Contrastive Learning
Encouraged by the success of contrastive learning on image classification tasks, we propose a new self-supervised method for the structured regression task of 3D hand pose estimation.
Speech BERT Embedding For Improving Prosody in Neural TTS
Experimental results obtained by the Transformer TTS show that the proposed BERT can extract fine-grained, segment-level prosody, which is complementary to utterance-level prosody to improve the final prosody of the TTS speech.
QuadrupletBERT: An Efficient Model For Embedding-Based Large-Scale Retrieval
The embedding-based large-scale query-document retrieval problem is a hot topic in the information retrieval (IR) field.
Turnover-Adjusted Information Ratio
In this paper, we study the behavior of information ratio (IR) as determined by the fundamental law of active investment management.
Rethinking Annotation Granularity for Overcoming Shortcuts in Deep Learning-based Radiograph Diagnosis: A Multicenter Study
The models were also compared to radiologists on a subset of the internal testing set (n=496).
Bayesian OOD detection with aleatoric uncertainty and outlier exposure
In particular, aleatoric uncertainty signals a specific type of OOD point: one without a well-defined class-label, and our model of data curation gives a likelihood for these points, giving us a mechanism for conditioning on outlier points and thus performing principled Bayesian outlier exposure.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Leveraging Review Properties for Effective Recommendation
Furthermore, inspired by the users' information adoption framework, we integrate two loss functions and a negative sampling strategy into our proposed RPRM model, to ensure that the properties of reviews are correlated with the users' preferences.
Learning Dual Priors for JPEG Compression Artifacts Removal
Specifically, we design a variational model to formulate the image de-blocking problem and propose two prior terms for the image content and gradient, respectively.
CoFF: Cooperative Spatial Feature Fusion for 3D Object Detection on Autonomous Vehicles
To reduce the amount of transmitted data, feature map based fusion is recently proposed as a practical solution to cooperative 3D object detection by autonomous vehicles.
Toward Quantifying Ambiguities in Artistic Images
It has long been hypothesized that perceptual ambiguities play an important role in aesthetic experience: a work with some ambiguity engages a viewer more than one that does not.
Negative Confidence-Aware Weakly Supervised Binary Classification for Effective Review Helpfulness Classification
However, a classification model that learns to classify binary instances with incomplete positive labels while assuming all unlabelled data to be negative examples will often generate a biased classifier.
Deep Mining External Imperfect Data for Chest X-ray Disease Screening
Recent researches have demonstrated that performance bottleneck exists in joint training on different CXR datasets, and few made efforts to address the obstacle.
Unifying Structure Analysis and Surrogate-driven Function Regression for Glaucoma OCT Image Screening
Besides, the proposed multi-task learning network is capable of exploring the structure and function relationship from the OCT image and visual field measurement simultaneously, which contributes to classification performance boosting.
Forward-Backward Decoding for Regularizing End-to-End TTS
Experimental results show our proposed methods especially the second one (bidirectional decoder regularization), leads a significantly improvement on both robustness and overall naturalness, as outperforming baseline (the revised version of Tacotron2) with a MOS gap of 0. 14 in a challenging test, and achieving close to human quality (4. 42 vs. 4. 49 in MOS) on general test.
Deep Angular Embedding and Feature Correlation Attention for Breast MRI Cancer Analysis
In this paper, we propose to identify breast tumor in MRI by Cosine Margin Sigmoid Loss (CMSL) with deep learning (DL) and localize possible cancer lesion by COrrelation Attention Map (COAM) based on the learned features.
Center of circle after perspective transformation
The boundary of the pupil is fitted with an ellipse and the euclidean center of the ellipse in the image is taken as the center of the pupil.
Large-Scale Domain Adaptation via Teacher-Student Learning
High accuracy speech recognition requires a large amount of transcribed data for supervised training.
Aggregating Frame-level Features for Large-Scale Video Classification
This paper introduces the system we developed for the Google Cloud & YouTube-8M Video Understanding Challenge, which can be considered as a multi-label classification problem defined on top of the large scale YouTube-8M Dataset.
Fusing Multi-Stream Deep Networks for Video Classification
A multi-stream framework is proposed to fully utilize the rich multimodal information in videos.
Evaluating Two-Stream CNN for Video Classification
In this paper, we conduct an in-depth study to investigate important implementation options that may affect the performance of deep nets on video classification.
Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification
In this paper, we propose a hybrid deep learning framework for video classification, which is able to model static spatial information, short-term motion, as well as long-term temporal clues in the videos.
