Search Results for author:
Found 518 papers, 160 papers with code
A Coarse-to-Fine Labeling Framework for Joint Word Segmentation, POS Tagging, and Constituent Parsing
In the coarse labeling stage, the joint model outputs a bracketed tree, in which each node corresponds to one of four labels (i. e., phrase, subphrase, word, subword).
Joint Goal Segmentation and Goal Success Prediction on Multi-Domain Conversations
To evaluate the performance of a multi-domain goal-oriented Dialogue System (DS), it is important to understand what the users’ goals are for the conversations and whether those goals are successfully achieved.
\textrm{DuReader}_{\textrm{vis}}: A Chinese Dataset for Open-domain Document Visual Question Answering
Open-domain question answering has been used in a wide range of applications, such as web search and enterprise search, which usually takes clean texts extracted from various formats of documents (e. g., web pages, PDFs, or Word documents) as the information source.
Learn to Cross-lingual Transfer with Meta Graph Learning Across Heterogeneous Languages
Recent emergence of multilingual pre-training language model (mPLM) has enabled breakthroughs on various downstream cross-lingual transfer (CLT) tasks.
Learning to See in the Dark with Events
A detail enhancing branch is proposed to reconstruct day light-specific features from the domain-invariant representations in a residual manner, regularized by a ranking loss.
All Information is Valuable: Question Matching over Full Information Transmission Network
In this process, there are two kinds of critical information that are commonly employed: the representation information of original questions and the interactive information between pairs of questions.
Learning Visuotactile Skills with Two Multifingered Hands
Two significant challenges exist: the lack of an affordable and accessible teleoperation system suitable for a dual-arm setup with multifingered hands, and the scarcity of multifingered hand hardware equipped with touch sensing.
NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results
This paper reviews the NTIRE 2024 Challenge on Shortform UGC Video Quality Assessment (S-UGC VQA), where various excellent solutions are submitted and evaluated on the collected dataset KVQ from popular short-form video platform, i. e., Kuaishou/Kwai Platform.
Scenario-Adaptive Fine-Grained Personalization Network: Tailoring User Behavior Representation to the Scenario Context
To enhance the model's capacity to capture user interests from historical behavior sequences in each scenario, we develop a ranking framework named the Scenario-Adaptive Fine-Grained Personalization Network (SFPNet), which designs a kind of fine-grained method for multi-scenario personalized recommendations.
Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs
Then, we propose a simple and effective framework called Graph Chain-of-thought (Graph-CoT) to augment LLMs with graphs by encouraging LLMs to reason on the graph iteratively.
CORP: A Multi-Modal Dataset for Campus-Oriented Roadside Perception Tasks
Numerous roadside perception datasets have been introduced to propel advancements in autonomous driving and intelligent transportation systems research and development.
Addressing Heterogeneity in Federated Load Forecasting with Personalization Layers
The advent of smart meters has enabled pervasive collection of energy consumption data for training short-term load forecasting models.
UAlign: Pushing the Limit of Template-free Retrosynthesis Prediction with Unsupervised SMILES Alignment
Single-step retrosynthesis prediction, a crucial step in the planning process, has witnessed a surge in interest in recent years due to advancements in AI for science.
A Survey on Consumer IoT Traffic: Security and Privacy
From the security and privacy perspective, this survey seeks out the new characteristics in CIoT traffic analysis, the state-of-the-art progress in CIoT traffic analysis, and the challenges yet to be solved.
Task-Aware Low-Rank Adaptation of Segment Anything Model
Furthermore, we introduce modified SAM (mSAM) for multi-task learning where we remove the prompt encoder of SAM and use task-specific no mask embeddings and mask decoder for each task.
Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation
Multimodal large language models (MLLMs) have shown impressive reasoning abilities, which, however, are also more vulnerable to jailbreak attacks than their LLM predecessors.
Conditional Score-Based Diffusion Model for Cortical Thickness Trajectory Prediction
Alzheimer's Disease (AD) is a neurodegenerative condition characterized by diverse progression rates among individuals, with changes in cortical thickness (CTh) closely linked to its progression.
SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting
We present SplattingAvatar, a hybrid 3D representation of photorealistic human avatars with Gaussian Splatting embedded on a triangle mesh, which renders over 300 FPS on a modern GPU and 30 FPS on a mobile device.
ERASOR++: Height Coding Plus Egocentric Ratio Based Dynamic Object Removal for Static Point Cloud Mapping
Mapping plays a crucial role in location and navigation within automatic systems.
Real-Time Adaptive Safety-Critical Control with Gaussian Processes in High-Order Uncertain Models
Subsequently, the hyperparameters of the Gaussian model are trained with a specially compound kernel, and the Gaussian model's online inferential capability and computational efficiency are strengthened by updating a solitary inducing point derived from new samples, in conjunction with the learned hyperparameters.
Online Efficient Safety-Critical Control for Mobile Robots in Unknown Dynamic Multi-Obstacle Environments
This paper proposes a LiDAR-based goal-seeking and exploration framework, addressing the efficiency of online obstacle avoidance in unstructured environments populated with static and moving obstacles.
CFRet-DVQA: Coarse-to-Fine Retrieval and Efficient Tuning for Document Visual Question Answering
Document Visual Question Answering (DVQA) is a task that involves responding to queries based on the content of images.
A Unified Taxonomy-Guided Instruction Tuning Framework for Entity Set Expansion and Taxonomy Expansion
Entity Set Expansion, Taxonomy Expansion, and Seed-Guided Taxonomy Construction are three representative tasks that can be used to automatically populate an existing taxonomy with new entities.
SoLA: Solver-Layer Adaption of LLM for Better Logic Reasoning
In this paper, we propose a novel solver-layer adaptation (SoLA) method, where we introduce a solver as a new layer of the LLM to differentially guide solutions towards satisfiability.
Evaluating Large Language Models in Analysing Classroom Dialogue
Recognizing the knowledge-intensive and labor-intensive nature of traditional qualitative methods in educational research, this study investigates the potential of LLM to streamline and enhance the analysis process.
KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completion
Knowledge Graph Completion (KGC) is crucial for addressing knowledge graph incompleteness and supporting downstream applications.
Rendering Graphs for Graph Reasoning in Multimodal Large Language Models
In this paper, we take the first step in incorporating visual information into graph reasoning tasks and propose a new benchmark GITQA, where each sample is a tuple (graph, image, textual description).
iMove: Exploring Bio-impedance Sensing for Fitness Activity Recognition
While IMUs are currently the prominent fitness tracking modality, through iMove, we show bio-impedence can help improve IMU-based fitness tracking through sensor fusion and contrastive learning. To evaluate our methods, we conducted an experiment including six upper body fitness activities performed by ten subjects over five days to collect synchronized data from bio-impedance across two wrists and IMU on the left wrist. The contrastive learning framework uses the two modalities to train a better IMU-only classification model, where bio-impedance is only required at the training phase, by which the average Macro F1 score with the input of a single IMU was improved by 3. 22 \% reaching 84. 71 \% compared to the 81. 49 \% of the IMU baseline model.
Distribution-consistency Structural Causal Models
In the field of causal modeling, potential outcomes (PO) and structural causal models (SCMs) stand as the predominant frameworks.
Seed-Guided Fine-Grained Entity Typing in Science and Engineering Domains
In this paper, we study the task of seed-guided fine-grained entity typing in science and engineering domains, which takes the name and a few seed entities for each entity type as the only supervision and aims to classify new entity mentions into both seen and unseen types (i. e., those without seed entities).
Multilingual and Fully Non-Autoregressive ASR with Large Language Model Fusion: A Comprehensive Study
In the era of large models, the autoregressive nature of decoding often results in latency serving as a significant bottleneck.
SMUTF: Schema Matching Using Generative Tags and Hybrid Features
Recognizing the lack of extensive, publicly available datasets for SM, we have created and open-sourced the HDXSM dataset from the public humanitarian data.
HG3-NeRF: Hierarchical Geometric, Semantic, and Photometric Guided Neural Radiance Fields for Sparse View Inputs
We propose Hierarchical Geometric Guidance (HGG) to incorporate the attachment of Structure from Motion (SfM), namely sparse depth prior, into the scene representations.
A First-Order Multi-Gradient Algorithm for Multi-Objective Bi-Level Optimization
In this paper, we study the Multi-Objective Bi-Level Optimization (MOBLO) problem, where the upper-level subproblem is a multi-objective optimization problem and the lower-level subproblem is for scalar optimization.
E^2-LLM: Efficient and Extreme Length Extension of Large Language Models
Typically, training LLMs with long context sizes is computationally expensive, requiring extensive training hours and GPU resources.
Scaling Laws And Statistical Properties of The Transaction Flows And Holding Times of Bitcoin
Defining age-dependent transaction flows as the fraction of bitcoins that are traded at a given time and that were born (last traded) at some specific earlier time, we document that the time-averaged transaction flow fraction has a power law dependence as a function of age, with an exponent close to $-1. 5$, a value compatible with priority queuing theory.
Online Test-Time Adaptation of Spatial-Temporal Traffic Flow Forecasting
To make the model trained on historical data better adapt to future data in a fully online manner, this paper conducts the first study of the online test-time adaptation techniques for spatial-temporal traffic flow forecasting problems.
Ranked #4 on
Traffic Prediction
on PeMS07
VLLaVO: Mitigating Visual Gap through LLMs
Recent advances achieved by deep learning models rely on the independent and identically distributed assumption, hindering their applications in real-world scenarios with domain shifts.
Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuning
Instruction tuning of Large Vision-language Models (LVLMs) has revolutionized the development of versatile models with zero-shot generalization across a wide range of downstream vision-language tasks.
StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesis
Moreover, existing SVS methods encounter a decline in the quality of synthesized singing voices in OOD scenarios, as they rest upon the assumption that the target vocal attributes are discernible during the training phase.
Memory-Efficient Reversible Spiking Neural Networks
In this paper, we propose the reversible spiking neural network to reduce the memory cost of intermediate activations and membrane potentials during training.
A Unified Framework for Unsupervised Domain Adaptation based on Instance Weighting
Specifically, the proposed LIWUDA method constructs a weight network to assign weights to each instance based on its probability of belonging to common classes, and designs Weighted Optimal Transport (WOT) for domain alignment by leveraging instance weights.
Lite-Mind: Towards Efficient and Robust Brain Representation Network
The limited data availability and the low signal-to-noise ratio of fMRI signals lead to the challenging task of fMRI-to-image retrieval.
Dynamic Inertial Poser (DynaIP): Part-Based Motion Dynamics Learning for Enhanced Human Pose Estimation with Sparse Inertial Sensors
This paper introduces a novel human pose estimation approach using sparse inertial sensors, addressing the shortcomings of previous methods reliant on synthetic data.
IG Captioner: Information Gain Captioners are Strong Zero-shot Classifiers
To tackle this problem, we redesign the scoring objective for the captioner to alleviate the distributional bias and focus on measuring the gain of information brought by the visual inputs.
HiPose: Hierarchical Binary Surface Encoding and Correspondence Pruning for RGB-D 6DoF Object Pose Estimation
In this work, we present a novel dense-correspondence method for 6DoF object pose estimation from a single RGB-D image.
Privacy-Preserving Load Forecasting via Personalized Model Obfuscation
The widespread adoption of smart meters provides access to detailed and localized load consumption data, suitable for training building-level load forecasting models.
Spatio-Temporal Similarity Measure based Multi-Task Learning for Predicting Alzheimer's Disease Progression using MRI Data
The experimental results show that compared with directly ROI based learning, our proposed method is more effective in predicting disease progression.
Signal Processing Meets SGD: From Momentum to Filter
To address this contradiction, we propose a novel optimization method that aims to accelerate the convergence rate of SGD without loss of generalization.
E3 TTS: Easy End-to-End Diffusion-based Text to Speech
Instead, E3 TTS models the temporal structure of the waveform through the diffusion process.
"Why Should I Review This Paper?" Unifying Semantic, Topic, and Citation Factors for Paper-Reviewer Matching
As many academic conferences are overwhelmed by a rapidly increasing number of paper submissions, automatically finding appropriate reviewers for each submission becomes a more urgent need than ever.
Machine Learning Methods for Background Potential Estimation in 2DEGs
In the realm of quantum-effect devices and materials, two-dimensional electron gases (2DEGs) stand as fundamental structures that promise transformative technologies.
Ontology Enrichment for Effective Fine-grained Entity Typing
In this study, we propose OnEFET, where we (1) enrich each node in the ontology structure with two types of extra information: instance information for training sample augmentation and topic information to relate types to contexts, and (2) develop a coarse-to-fine typing algorithm that exploits the enriched information by training an entailment model with contrasting topics and instance-based augmented training samples.
Non-autoregressive Text Editing with Copy-aware Latent Alignments
In this work, we propose a novel non-autoregressive text editing method to circumvent the above issues, by modeling the edit process with latent CTC alignments.
Learning Multiplex Embeddings on Text-rich Networks with One Text Encoder
Mainstream text representation learning methods use pretrained language models (PLMs) to generate one embedding for each text unit, expecting that all types of relations between texts can be captured by these single-view embeddings.
CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model
It is specifically designed for code-related tasks with both English and Chinese prompts and supports over 40 programming languages.
BYOM: Building Your Own Multi-Task Model For Free
Recently, various merging methods have been proposed to build a multi-task model from task-specific finetuned models without retraining.
SLM: Bridge the thin gap between speech and text foundation models
We present a joint Speech and Language Model (SLM), a multitask, multilingual, and dual-modal model that takes advantage of pretrained foundational speech and language models.
IAIFNet: An Illumination-Aware Infrared and Visible Image Fusion Network
Infrared and visible image fusion (IVIF) is used to generate fusion images with comprehensive features of both images, which is beneficial for downstream vision tasks.
SSPFusion: A Semantic Structure-Preserving Approach for Infrared and Visible Image Fusion
Most existing learning-based infrared and visible image fusion (IVIF) methods exhibit massive redundant information in the fusion images, i. e., yielding edge-blurring effect or unrecognizable for object detectors.
Adversarial Attacks on Video Object Segmentation with Hard Region Discovery
Particularly, the gradients from the segmentation model are exploited to discover the easily confused region, in which it is difficult to identify the pixel-wise objects from the background in a frame.
Unify word-level and span-level tasks: NJUNLP's Participation for the WMT2023 Quality Estimation Shared Task
We generate pseudo MQM data using parallel data from the WMT translation task.
Domain-Guided Conditional Diffusion Model for Unsupervised Domain Adaptation
However, the performance of existing UDA methods is constrained by the large domain shift and limited target domain data.
Fully Transformer-Equipped Architecture for End-to-End Referring Video Object Segmentation
Referring Video Object Segmentation (RVOS) requires segmenting the object in video referred by a natural language query.
Health diagnosis and recuperation of aged Li-ion batteries with data analytics and equivalent circuit modeling
Battery health assessment and recuperation play a crucial role in the utilization of second-life Li-ion batteries.
Efficient Long-Short Temporal Attention Network for Unsupervised Video Object Segmentation
Unsupervised Video Object Segmentation (VOS) aims at identifying the contours of primary foreground objects in videos without any prior knowledge.
Semantic Segmentation
Unsupervised Video Object Segmentation
+1
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
Our MetaMath-7B model achieves 66. 4% on GSM8K and 19. 4% on MATH, exceeding the state-of-the-art models of the same size by 11. 5% and 8. 7%.
Ranked #53 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
Multimodal Modeling For Spoken Language Identification
Spoken language identification refers to the task of automatically predicting the spoken language in a given utterance.
USM-SCD: Multilingual Speaker Change Detection Based on Large Pretrained Foundation Models
We show that the USM-SCD model can achieve more than 75% average speaker change detection F1 score across a test set that consists of data from 96 languages.
Multi-view Self-supervised Disentanglement for General Image Denoising
It is understandable as the model is designed to learn paired mapping (e. g. from a noisy image to its clean version).
Ranked #1 on
Denoising
on CBSD68 sigm75
StereoFlowGAN: Co-training for Stereo and Flow with Unsupervised Domain Adaptation
We introduce a novel training strategy for stereo matching and optical flow estimation that utilizes image-to-image translation between synthetic and real image domains.
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
While large language models (LLMs) have demonstrated remarkable capabilities across a range of downstream tasks, a significant concern revolves around their propensity to exhibit hallucinations: LLMs occasionally generate content that diverges from the user input, contradicts previously generated context, or misaligns with established world knowledge.
Occlusion-Aware Detection and Re-ID Calibrated Network for Multi-Object Tracking
Multi-Object Tracking (MOT) is a crucial computer vision task that aims to predict the bounding boxes and identities of objects simultaneously.
Dual-Balancing for Multi-Task Learning
Multi-task learning (MTL), a learning paradigm to learn multiple related tasks simultaneously, has achieved great success in various fields.
High-Fidelity Lake Extraction via Two-Stage Prompt Enhancement: Establishing a Novel Baseline and Benchmark
Lake extraction from remote sensing imagery is a complex challenge due to the varied lake shapes and data noise.
Forward-Backward Reasoning in Large Language Models for Mathematical Verification
Instead of using forward or backward reasoning alone, we propose FOBAR to combine FOrward and BAckward Reasoning for verification.
DiffCR: A Fast Conditional Diffusion Framework for Cloud Removal from Optical Satellite Images
Optical satellite images are a critical data source; however, cloud cover often compromises their quality, hindering image applications and analysis.
LEFormer: A Hybrid CNN-Transformer Architecture for Accurate Lake Extraction from Remote Sensing Imagery
LEFormer contains three main modules: CNN encoder, Transformer encoder, and cross-encoder fusion.
Decomposing and Coupling Saliency Map for Lesion Segmentation in Ultrasound Images
Complex scenario of ultrasound image, in which adjacent tissues (i. e., background) share similar intensity with and even contain richer texture patterns than lesion region (i. e., foreground), brings a unique challenge for accurate lesion segmentation.
MATNilm: Multi-appliance-task Non-intrusive Load Monitoring with Limited Labeled Data
Non-intrusive load monitoring (NILM) identifies the status and power consumption of various household appliances by disaggregating the total power usage signal of an entire house.
A Dual-mode Local Search Algorithm for Solving the Minimum Dominating Set Problem
Given a graph, the minimum dominating set (MinDS) problem is to identify a smallest set $D$ of vertices such that every vertex not in $D$ is adjacent to at least one vertex in $D$.
"It Felt Like Having a Second Mind": Investigating Human-AI Co-creativity in Prewriting with Large Language Models
This collaborative process champions the human in a dominant role, in addition to mixed and shifting levels of initiative that exist between humans and LLMs.
Real-World Evaluation of Full-Duplex Millimeter Wave Communication Systems
Noteworthy strides continue to be made in the development of full-duplex millimeter wave (mmWave) communication systems, but most of this progress has been built on theoretical models and validated through simulation.
Weakly Supervised Multi-Label Classification of Full-Text Scientific Papers
Instead of relying on human-annotated training samples to build a classifier, weakly supervised scientific paper classification aims to classify papers only using category descriptions (e. g., category names, category-indicative keywords).
FlowFace++: Explicit Semantic Flow-supervised End-to-End Face Swapping
The discriminator is shape-aware and relies on a semantic flow-guided operation to explicitly calculate the shape discrepancies between the target and source faces, thus optimizing the face swapping network to generate highly realistic results.
AudioPaLM: A Large Language Model That Can Speak and Listen
AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2.
Learning Variable Impedance Skills from Demonstrations with Passivity Guarantee
The proposed approach involves the estimation of full stiffness matrices from human demonstrations, which are then combined with sensed forces and motion information to create a model using the non-parametric method.
PersonaPKT: Building Personalized Dialogue Agents via Parameter-efficient Knowledge Transfer
Personalized dialogue agents (DAs) powered by large pre-trained language models (PLMs) often rely on explicit persona descriptions to maintain personality consistency.
Efficient Adapters for Giant Speech Models
However, finetuning all parameters from the self-supervised learned model can be computationally expensive, and becomes infeasiable as the size of the model and the number of downstream tasks scales.
A Graph Transformer-Driven Approach for Network Robustness Learning
Learning and analysis of network robustness, including controllability robustness and connectivity robustness, is critical for various networked systems against attacks.
Gotta: Generative Few-shot Question Answering by Prompt-based Cloze Data Augmentation
Few-shot question answering (QA) aims at precisely discovering answers to a set of questions from context passages while only a few training samples are available.
COPR: Consistency-Oriented Pre-Ranking for Online Advertising
In this architecture, the pre-ranking model is expected to be a lightweight approximation of the ranking model, which handles more candidates with strict latency requirements.
Bi-LRFusion: Bi-Directional LiDAR-Radar Fusion for 3D Dynamic Object Detection
LiDAR and Radar are two complementary sensing approaches in that LiDAR specializes in capturing an object's 3D shape while Radar provides longer detection ranges as well as velocity hints.
How to Estimate Model Transferability of Pre-Trained Speech Models?
In this work, we introduce a "score-based assessment" framework for estimating the transferability of pre-trained speech models (PSMs) for fine-tuning target tasks.
Effective Structured Prompting by Meta-Learning and Representative Verbalizer
Combining meta-learning the prompt pool and RepVerb, we propose MetaPrompter for effective structured prompting.
Explanation Graph Generation via Generative Pre-training over Synthetic Graphs
The generation of explanation graphs is a significant task that aims to produce explanation graphs in response to user input, revealing the internal reasoning process.
LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus
The constituent samples of LibriTTS-R are identical to those of LibriTTS, with only the sound quality improved.
Mixture-of-Expert Conformer for Streaming Multilingual ASR
We evaluate the proposed model on a set of 12 languages, and achieve an average 11. 9% relative improvement in WER over the baseline.
PIEClass: Weakly-Supervised Text Classification with Prompting and Noise-Robust Iterative Ensemble Training
Weakly-supervised text classification trains a classifier using the label name of each target class as the only supervision, which largely reduces human annotation efforts.
Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding
Scientific literature understanding tasks have gained significant attention due to their potential to accelerate scientific discovery.
Capturing Conversion Rate Fluctuation during Sales Promotions: A Novel Historical Data Reuse Approach
However, we observe that a well-trained CVR prediction model often performs sub-optimally during sales promotions.
Patton: Language Model Pretraining on Text-Rich Networks
A real-world text corpus sometimes comprises not only text documents but also semantic links between them (e. g., academic papers in a bibliographic network are linked by citations and co-authorships).
Temporal Consistent Automatic Video Colorization via Semantic Correspondence
Extensive experiments demonstrate that our method outperforms other methods in maintaining temporal consistency both qualitatively and quantitatively.
A Self-Training Framework Based on Multi-Scale Attention Fusion for Weakly Supervised Semantic Segmentation
Weakly supervised semantic segmentation (WSSS) based on image-level labels is challenging since it is hard to obtain complete semantic regions.
A Unifying Framework of Attention-based Neural Load Forecasting
In this paper, we propose a unifying deep learning framework for load forecasting, which includes time-varying feature weighting, hierarchical temporal attention, and feature-reinforced error correction.
Chain-of-Skills: A Configurable Model for Open-domain Question Answering
Our approach outperforms recent self-supervised retrievers in zero-shot evaluations and achieves state-of-the-art fine-tuned retrieval performance on NQ, HotpotQA and OTT-QA.
Ranked #4 on
Question Answering
on HotpotQA
Transforming Visual Scene Graphs to Image Captions
In TSG, we apply multi-head attention (MHA) to design the Graph Neural Network (GNN) for embedding scene graphs.
An Adaptive Policy to Employ Sharpness-Aware Minimization
Sharpness-aware minimization (SAM), which searches for flat minima by min-max optimization, has been shown to be useful in improving model generalization.
Understanding Shared Speech-Text Representations
Recently, a number of approaches to train speech models by incorpo-rating text into end-to-end models have been developed, with Mae-stro advancing state-of-the-art automatic speech recognition (ASR)and Speech Translation (ST) performance.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Detection of Pavement Cracks by Deep Learning Models of Transformer and UNet
In this study, we investigated nine promising models to evaluate their performance in pavement surface crack detection by model accuracy, computational complexity, and model stability.
Mastering Asymmetrical Multiplayer Game with Multi-Agent Asymmetric-Evolution Reinforcement Learning
We propose asymmetric-evolution training (AET), a novel multi-agent reinforcement learning framework that can train multiple kinds of agents simultaneously in AMP game.
Handling Heavy Occlusion in Dense Crowd Tracking by Focusing on the Heads
Being able to identify and track all the pedestrians in the dense crowd scene with computer vision approaches is a typical challenge in this field, also known as the Multiple Object Tracking (MOT) challenge.
SPColor: Semantic Prior Guided Exemplar-based Image Colorization
Previous methods search for correspondence across the entire reference image, and this type of global matching is easy to get mismatch.
Simplifying Low-Light Image Enhancement Networks with Relative Loss Functions
In this paper, to make the learning easier in low-light image enhancement, we introduce FLW-Net (Fast and LightWeight Network) and two relative loss functions.
Safe Explicable Planning
The goal of SEP is to find behaviors that align with human expectations while adhering to the specified safety criterion.
Personalized Federated Learning with Local Attention
The key challenge of FL is the heterogeneity of local data in different clients, such as heterogeneous label distribution and feature shift, which could lead to significant performance degradation of the learned models.
Exemplar-based Video Colorization with Long-term Spatiotemporal Dependency
Exemplar-based video colorization is an essential technique for applications like old movie restoration.
$P^{3}O$: Transferring Visual Representations for Reinforcement Learning via Prompting
In particular, we specify a prompt-transformer for representation conversion and propose a two-step training process to train the prompt-transformer for the target environment, while the rest of the DRL pipeline remains unchanged.
Diffusion-based Target Sampler for Unsupervised Domain Adaptation
The generated samples can well simulate the data distribution of the target domain and help existing UDA methods transfer from the source domain to the target domain more easily, thus improving the transfer performance.
Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representations
Experiments show that Miipher (i) is robust against various audio degradation and (ii) enable us to train a high-quality text-to-speech (TTS) model from restored speech samples collected from the Web.
Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages
We introduce the Universal Speech Model (USM), a single large model that performs automatic speech recognition (ASR) across 100+ languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
PCR-CG: Point Cloud Registration via Deep Explicit Color and Geometry
Different from previous methods that only use geometry representation, our module is specifically designed to effectively correlate color into geometry for the point cloud registration task.
Edgeformers: Graph-Empowered Transformers for Representation Learning on Textual-Edge Networks
Edges in many real-world social/information networks are associated with rich text information (e. g., user-user communications or user-product reviews).
Massively Multilingual Shallow Fusion with Large Language Models
In this work, we propose to train a single multilingual language model (LM) for shallow fusion in multiple languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
JEIT: Joint End-to-End Model and Internal Language Model Training for Speech Recognition
We propose JEIT, a joint end-to-end (E2E) model and internal language model (ILM) training method to inject large-scale unpaired text into ILM during E2E training which improves rare-word speech recognition.
The Effect of Metadata on Scientific Literature Tagging: A Cross-Field Cross-Model Study
Due to the exponential growth of scientific publications on the Web, there is a pressing need to tag each paper with fine-grained topics so that researchers can track their interested fields of study rather than drowning in the whole literature.
TrajMatch: Towards Automatic Spatio-temporal Calibration for Roadside LiDARs through Trajectory Matching
In this paper, we propose TrajMatch -- the first system that can automatically calibrate for roadside LiDARs in both time and space.
Efficient Domain Adaptation for Speech Foundation Models
The FM encoder adapter and decoder are then finetuned to the target domain with a small amount of supervised in-domain data.
Physics-guided Residual Learning for Probabilistic Power Flow Analysis
Probabilistic power flow (PPF) analysis is critical to power system operation and planning.
From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognition
In this work, we propose a new parameter-efficient learning framework based on neural model reprogramming for cross-lingual speech recognition, which can \textbf{re-purpose} well-trained English automatic speech recognition (ASR) models to recognize the other languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Super-Resolution Harmonic Retrieval of Non-Circular Signals
This paper proposes a super-resolution harmonic retrieval method for uncorrelated strictly non-circular signals, whose covariance and pseudo-covariance present Toeplitz and Hankel structures, respectively.
OA-BEV: Bringing Object Awareness to Bird's-Eye-View Representation for Multi-Camera 3D Object Detection
To this end, we propose OA-BEV, a network that can be plugged into the BEV-based 3D object detection framework to bring out the objects by incorporating object-aware pseudo-3D features and depth features.
Learning Trajectory-Word Alignments for Video-Language Tasks
To amend this, we propose a novel TW-BERT to learn Trajectory-Word alignment by a newly designed trajectory-to-word (T2W) attention for solving video-language tasks.
Adaptively Clustering Neighbor Elements for Image Captioning
We design a novel global-local Transformer named \textbf{Ada-ClustFormer} (\textbf{ACF}) to generate captions.
PEAL: Prior-Embedded Explicit Attention Learning for Low-Overlap Point Cloud Registration
Recently, it has achieved huge success in incorporating Transformer into point cloud feature representation, which usually adopts a self-attention module to learn intra-point-cloud features first, then utilizes a cross-attention module to perform feature exchange between input point clouds.
Range-Nullspace Video Frame Interpolation With Focalized Motion Estimation
Continuous-time video frame interpolation is a fundamental technique in computer vision for its flexibility in synthesizing motion trajectories and novel video frames at arbitrary intermediate time steps.
E2NeRF: Event Enhanced Neural Radiance Fields from Blurry Images
To solve this problem, we propose a novel Event-Enhanced NeRF (E2NeRF) by utilizing the combination data of a bio-inspired event camera and a standard RGB camera.
Adaptive Positional Encoding for Bundle-Adjusting Neural Radiance Fields
Neural Radiance Fields have shown great potential to synthesize novel views with only a few discrete image observations of the world.
Dynamic Sparse Network for Time Series Classification: Learning What to "see''
The receptive field (RF), which determines the region of time series to be ``seen'' and used, is critical to improve the performance for time series classification (TSC).
Mu$^{2}$SLAM: Multitask, Multilingual Speech and Language Models
We present Mu$^{2}$SLAM, a multilingual sequence-to-sequence model pre-trained jointly on unlabeled speech, unlabeled text and supervised data spanning Automatic Speech Recognition (ASR), Automatic Speech Translation (AST) and Machine Translation (MT), in over 100 languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Effective Seed-Guided Topic Discovery by Integrating Multiple Types of Contexts
Instead of mining coherent topics from a given text corpus in a completely unsupervised manner, seed-guided topic discovery methods leverage user-provided seed words to extract distinctive and coherent topics so that the mined topics can better cater to the user's interest.
Unsupervised Deep Learning for AC Optimal Power Flow via Lagrangian Duality
Non-convex AC optimal power flow (AC-OPF) is a fundamental optimization problem in power system analysis.
Entity Set Co-Expansion in StackOverflow
Given a few seed entities of a certain type (e. g., Software or Programming Language), entity set expansion aims to discover an extensive set of entities that share the same type as the seeds.
Feature Aggregation and Propagation Network for Camouflaged Object Detection
In this paper, we propose a novel Feature Aggregation and Propagation Network (FAP-Net) for camouflaged object detection.
ExpNet: A unified network for Expert-Level Classification
In the paper, we call them expert-level classification.
TSGP: Two-Stage Generative Prompting for Unsupervised Commonsense Question Answering
In this paper, we aim to address the above limitation by leveraging the implicit knowledge stored in PrLMs and propose a two-stage prompt-based unsupervised commonsense question answering framework (TSGP).
Leveraging per Image-Token Consistency for Vision-Language Pre-training
(2) Under-utilization of the unmasked tokens: CMLM primarily focuses on the masked token but it cannot simultaneously leverage other tokens to learn vision-language associations.
Disentangling Task Relations for Few-shot Text Classification via Self-Supervised Hierarchical Task Clustering
However, most prior works assume that all the tasks are sampled from a single data source, which cannot adapt to real-world scenarios where tasks are heterogeneous and lie in different distributions.
Normative Modeling via Conditional Variational Autoencoder and Adversarial Learning to Identify Brain Dysfunction in Alzheimer's Disease
Normative modeling is an emerging and promising approach to effectively study disorder heterogeneity in individual participants.
TLP: A Deep Learning-based Cost Model for Tensor Program Tuning
Instead of extracting features from the tensor program itself, TLP extracts features from the schedule primitives.
Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning
In this work, we study few-shot learning with PLMs from a different perspective: We first tune an autoregressive PLM on the few-shot samples and then use it as a generator to synthesize a large amount of novel training samples which augment the original training set.
Max Markov Chain
In this paper, we introduce Max Markov Chain (MMC), a novel representation for a useful subset of High-order Markov Chains (HMCs) with sparse correlations among the states.
A Quantum Kernel Learning Approach to Acoustic Modeling for Spoken Command Recognition
We propose a quantum kernel learning (QKL) framework to address the inherent data sparsity issues often encountered in training large-scare acoustic models in low-resource scenarios.
MedSegDiff: Medical Image Segmentation with Diffusion Probabilistic Model
Inspired by the success of DPM, we propose the first DPM based model toward general medical image segmentation tasks, which we named MedSegDiff.
Modular Hybrid Autoregressive Transducer
In this work, we propose a modular hybrid autoregressive transducer (MHAT) that has structurally separated label and blank decoders to predict label and blank distributions, respectively, along with a shared acoustic encoder.
Accelerating RNN-T Training and Inference Using CTC guidance
We propose a novel method to accelerate training and inference process of recurrent neural network transducer (RNN-T) based on the guidance from a co-trained connectionist temporal classification (CTC) model.
Visually-Aware Audio Captioning With Adaptive Audio-Visual Attention
Audio captioning aims to generate text descriptions of audio clips.
Residual Adapters for Few-Shot Text-to-Speech Speaker Adaptation
Adapting a neural text-to-speech (TTS) model to a target speaker typically involves fine-tuning most if not all of the parameters of a pretrained multi-speaker backbone model.
Personalized Dialogue Generation with Persona-Adaptive Attention
Persona-based dialogue systems aim to generate consistent responses based on historical context and predefined persona.
Virtuoso: Massive Multilingual Speech-Text Joint Semi-Supervised Learning for Text-To-Speech
This paper proposes Virtuoso, a massively multilingual speech-text joint semi-supervised learning framework for text-to-speech synthesis (TTS) models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Maestro-U: Leveraging joint speech-text representation learning for zero supervised speech ASR
First, we show that by combining speech representations with byte-level text representations and use of language embeddings, we can dramatically reduce the Character Error Rate (CER) on languages with no supervised speech from 64. 8\% to 30. 8\%, a relative reduction of 53\%.
Improving generalizability of distilled self-supervised speech processing models under distorted settings
Self-supervised learned (SSL) speech pre-trained models perform well across various speech processing tasks.
JOIST: A Joint Speech and Text Streaming Model For ASR
In addition, we explore JOIST using a streaming E2E model with an order of magnitude more data, which are also novelties compared to previous works.
Comparison of Soft and Hard Target RNN-T Distillation for Large-scale ASR
Knowledge distillation is an effective machine learning technique to transfer knowledge from a teacher model to a smaller student model, especially with unlabeled data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
GIDN: A Lightweight Graph Inception Diffusion Network for High-efficient Link Prediction
In this paper, we propose a Graph Inception Diffusion Networks(GIDN) model.
Ranked #1 on
Link Property Prediction
on ogbl-ddi
On Clustering Trend in Language Evolution Based on Dynamical Behaviors of Multi-Agent Model
Computer model has been extensively adopted to overcome the time limitation of language evolution by transforming language theory into physical modeling mechanism, which helps to explore the general laws of the evolution.
Selective Partial Domain Adaptation
To solve this problem, we propose a Selective Partial Domain Adaptation (SPDA) method, which selects useful data for the adaptation to the target domain.
Ranked #1 on
Partial Domain Adaptation
on VisDA2017
A Novel Observer-Centric Approach for Detecting Faults in Islanded AC Microgrids with Uncertainties
This paper proposes a strategy based on observers and residuals for detecting internal faults in grid-forming inverters with power-sharing coordination.
Spatiotemporal Multi-scale Bilateral Motion Network for Gait Recognition
The critical goal of gait recognition is to acquire the inter-frame walking habit representation from the gait sequences.
Real-Time Power System Event Detection: A Novel Instance Selection Approach
Instance selection is a vital technique for energy big data analytics.
How Good Is Neural Combinatorial Optimization? A Systematic Evaluation on the Traveling Salesman Problem
Hopefully, this work would help with a better understanding of the strengths and weaknesses of NCO and provide a comprehensive evaluation protocol for further benchmarking NCO approaches in comparison to other approaches.
Discrete Linear Canonical Transform on Graphs
In this paper, we propose and design the definition of the discrete linear canonical transform on graphs (GLCT), which is an extension of the discrete linear canonical transform (DLCT), just as the graph Fourier transform (GFT) is an extension of the discrete Fourier transform (DFT).
Online Beam Learning with Interference Nulling for Millimeter Wave MIMO Systems
Furthermore, a hardware proof-of-concept prototype based on mmWave phased arrays is built and used to implement and evaluate the developed online beam learning solutions in realistic scenarios.
Exploiting Deep Reinforcement Learning for Edge Caching in Cell-Free Massive MIMO Systems
Cell-free massive multiple-input-multiple-output is promising to meet the stringent quality-of-experience (QoE) requirements of railway wireless communications by coordinating many successional access points (APs) to serve the onboard users coherently.
Local Low-Rank Approximation With Superpixel-Guided Locality Preserving Graph for Hyperspectral Image Classification
By taking advantage of the local manifold structure, a Laplacian graph is constructed from the superpixels to ensure that a typical pixel should be similar to its neighbors within the same superpixel.
An Adaptive Repeated-Intersection-Reduction Local Search for the Maximum Independent Set Problem
The maximum independent set (MIS) problem, a classical NP-hard problem with extensive applications in various areas, aims to find the largest set of vertices with no edge among them.
See What You See: Self-supervised Cross-modal Retrieval of Visual Stimuli from Brain Activity
Recent studies demonstrate the use of a two-stage supervised framework to generate images that depict human perception to visual stimuli from EEG, referring to EEG-visual reconstruction.
An Efficient Person Clustering Algorithm for Open Checkout-free Groceries
Then, to ensure that the method adapts to the dynamic and unseen person flow, we propose Graph Convolutional Network (GCN) with a simple Nearest Neighbor (NN) strategy to accurately cluster the instances of CSG.
CROLoss: Towards a Customizable Loss for Retrieval Models in Recommender Systems
In this paper, we proposed the Customizable Recall@N Optimization Loss (CROLoss), a loss function that can directly optimize the Recall@N metrics and is customizable for different choices of N. This proposed CROLoss formulation defines a more generalized loss function space, covering most of the conventional loss functions as special cases.
A Study of Modeling Rising Intonation in Cantonese Neural Speech Synthesis
In human speech, the attitude of a speaker cannot be fully expressed only by the textual content.
Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentation
A key to this challenging task is to fully utilize the information in the support images by exploiting fine-grained correlations between the query and support images.
Ranked #4 on
Few-Shot Semantic Segmentation
on COCO-20i (1-shot)
Masked Spatial-Spectral Autoencoders Are Excellent Hyperspectral Defenders
Deep learning methodology contributes a lot to the development of hyperspectral image (HSI) analysis community.
Deformer: Towards Displacement Field Learning for Unsupervised Medical Image Registration
In this study, we propose a novel Deformer module along with a multi-scale framework for the deformable image registration task.
Structured Light with Redundancy Codes
We propose three applications using our redundancy codes: (1) Self error-correction for SL imaging under strong ambient light, (2) Error detection for adaptive reconstruction under global illumination, and (3) Interference filtering with device-specific projection sequence encoding, especially for event camera-based SL and light curtain devices.
Soft Retargeting Network for Click Through Rate Prediction
In this paper, we focus on a new type of user interest, i. e., user retargeting interest.
FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech
We introduce FLEURS, the Few-shot Learning Evaluation of Universal Representations of Speech benchmark.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Co-optimization of Battery Routing and Load Restoration for Microgrids with Mobile Energy Storage Systems
Mobile energy storage systems (MESS) offer great operational flexibility to enhance the resiliency of distribution systems in an emergency condition.
Heterformer: Transformer-based Deep Node Representation Learning on Heterogeneous Text-Rich Networks
In heterogeneous text-rich networks, this task is more challenging due to (1) presence or absence of text: Some nodes are associated with rich textual information, while others are not; (2) diversity of types: Nodes and edges of multiple types form a heterogeneous network structure.
Transferable Physical Attack against Object Detection with Separable Attention
Transferable adversarial attack is always in the spotlight since deep learning models have been demonstrated to be vulnerable to adversarial samples.
Leveraging Pseudo-labeled Data to Improve Direct Speech-to-Speech Translation
Direct Speech-to-speech translation (S2ST) has drawn more and more attention recently.
Seed-Guided Topic Discovery with Out-of-Vocabulary Seeds
Discovering latent topics from text corpora has been studied for decades.
Side-aware Meta-Learning for Cross-Dataset Listener Diagnosis with Subjective Tinnitus
With the development of digital technology, machine learning has paved the way for the next generation of tinnitus diagnoses.
Variation-cognizant Probabilistic Power Flow Analysis via Multi-task Learning
With an increasing high penetration of solar photovoltaic generation in electric power grids, voltage phasors and branch power flows experience more severe fluctuations.
Differentially Private Load Restoration for Microgrids with Distributed Energy Storage
Distributed energy storage systems (ESSs) can be efficiently leveraged for load restoration (LR) for a microgrid (MG) in island mode.
Interpretable Graph Convolutional Network of Multi-Modality Brain Imaging for Alzheimer's Disease Diagnosis
Identification of brain regions related to the specific neurological disorders are of great importance for biomarker and diagnostic studies.
Adversarial Filtering Modeling on Long-term User Behavior Sequences for Click-Through Rate Prediction
Rich user behavior information is of great importance for capturing and understanding user interest in click-through rate (CTR) prediction.
MAESTRO: Matched Speech Text Representations through Modality Matching
Self-supervised learning from speech signals aims to learn the latent structure inherent in the signal, while self-supervised learning from text attempts to capture lexical information.
Unsupervised Data Selection via Discrete Speech Representation for ASR
Self-supervised learning of speech representations has achieved impressive results in improving automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Modern Views of Machine Learning for Precision Psychiatry
In this review, we provide a comprehensive review of the ML methodologies and applications by combining neuroimaging, neuromodulation, and advanced mobile technologies in psychiatry practice.
BIG-bench Machine Learning
Explainable Artificial Intelligence (XAI)
