Search Results for author:
Found 85 papers, 40 papers with code
3D-Speaker-Toolkit: An Open Source Toolkit for Multi-modal Speaker Verification and Diarization
This paper introduces 3D-Speaker-Toolkit, an open source toolkit for multi-modal speaker verification and diarization.
Decoupled Contrastive Learning for Long-Tailed Recognition
In the scenario of long-tailed recognition, where the number of samples in each class is imbalanced, treating two types of positive samples equally leads to the biased optimization for intra-category distance.
An Embarrassingly Simple Approach for LLM with Strong ASR Capacity
We found that delicate designs are not necessary, while an embarrassingly simple composition of off-the-shelf speech encoder, LLM, and the only trainable linear projector is competent for the ASR task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
A Bionic Data-driven Approach for Long-distance Underwater Navigation with Anomaly Resistance
We integrate the developed mechanism with the TA-LSTM, and calibrate the predicted heading angles to gain resistance against geomagnetic anomalies.
emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation
To the best of our knowledge, emotion2vec is the first universal representation model in various emotion-related tasks, filling a gap in the field.
Privacy-preserving transactive energy systems: Key topics and open research challenges
This manuscript aims to formalize and conclude the discussions initiated during the PriTEM workshop 22-23 March 2023.
Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models
Recently, instruction-following audio-language models have received broad attention for audio interaction with humans.
Ranked #1 on
Acoustic Scene Classification
on TUT Acoustic Scenes 2017
(using extra training data)
Loss Masking Is Not Needed in Decoder-only Transformer for Discrete-token-based ASR
We find that applying the conventional cross-entropy loss on input speech tokens does not consistently improve the ASR performance over the Loss Masking approach.
LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT
In this paper, we propose LauraGPT, a unified GPT model for audio recognition, understanding, and generation.
Pink: Unveiling the Power of Referential Comprehension for Multi-modal LLMs
Specifically, we present a new method for constructing the instruction tuning dataset at a low cost by leveraging annotations in existing datasets.
Exploring RWKV for Memory Efficient and Low Latency Streaming ASR
Recently, self-attention-based transformers and conformers have been introduced as alternatives to RNNs for ASR acoustic modeling.
Leveraging Speech PTM, Text LLM, and Emotional TTS for Speech Emotion Recognition
In this paper, we explored how to boost speech emotion recognition (SER) with the state-of-the-art speech pre-trained model (PTM), data2vec, text generation technique, GPT-4, and speech synthesis technique, Azure TTS.
Improving Speaker Diarization using Semantic Information: Joint Pairwise Constraints Propagation
Speaker diarization has gained considerable attention within speech processing research community.
Incorporating Class-based Language Model for Named Entity Recognition in Factorized Neural Transducer
In spite of the excellent strides made by end-to-end (E2E) models in speech recognition in recent years, named entity recognition is still challenging but critical for semantic understanding.
FunCodec: A Fundamental, Reproducible and Integrable Open-source Toolkit for Neural Speech Codec
We also demonstrate that the pre-trained models are suitable for downstream tasks, including automatic speech recognition and personalized text-to-speech synthesis.
MixBCT: Towards Self-Adapting Backward-Compatible Training
As a solution, backward-compatible training can be employed to avoid the necessity of updating old retrieval datasets.
Adaptive robust tracking control with active learning for linear systems with ellipsoidal bounded uncertainties
This paper is concerned with the robust tracking control of linear uncertain systems, whose unknown system parameters and disturbances are bounded within ellipsoidal sets.
SeACo-Paraformer: A Non-Autoregressive ASR System with Flexible and Effective Hotword Customization Ability
It possesses the advantages of AED-based model's accuracy, NAR model's efficiency, and explicit customization capacity of superior performance.
Self-Distillation Network with Ensemble Prototypes: Learning Robust Speaker Representations without Supervision
It assigns representation of augmented views of utterances to the same prototypes as the representation of the original view, thereby enabling effective knowledge transfer between the views.
BA-SOT: Boundary-Aware Serialized Output Training for Multi-Talker ASR
The recently proposed serialized output training (SOT) simplifies multi-talker automatic speech recognition (ASR) by generating speaker transcriptions separated by a special token.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
CASA-ASR: Context-Aware Speaker-Attributed ASR
In addition, a two-pass decoding strategy is further proposed to fully leverage the contextual modeling ability resulting in a better recognition performance.
Semantic VAD: Low-Latency Voice Activity Detection for Speech Interaction
For speech interaction, voice activity detection (VAD) is often used as a front-end.
BAT: Boundary aware transducer for memory-efficient and low-latency ASR
Recently, recurrent neural network transducer (RNN-T) gains increasing popularity due to its natural streaming capability as well as superior performance.
Ranked #9 on
Speech Recognition
on AISHELL-1
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
FunASR: A Fundamental End-to-End Speech Recognition Toolkit
FunASR offers models trained on large-scale industrial corpora and the ability to deploy them in applications.
Ranked #1 on
Speech Recognition
on WenetSpeech
(using extra training data)
Accurate and Reliable Confidence Estimation Based on Non-Autoregressive End-to-End Speech Recognition System
Estimating confidence scores for recognition results is a classic task in ASR field and of vital importance for kinds of downstream tasks and training strategies.
TOLD: A Novel Two-Stage Overlap-Aware Framework for Speaker Diarization
Recently, end-to-end neural diarization (EEND) is introduced and achieves promising results in speaker-overlapped scenarios.
Ranked #1 on
Speaker Diarization
on CALLHOME
Achieving Timestamp Prediction While Recognizing with Non-Autoregressive End-to-End ASR Model
Conventional ASR systems use frame-level phoneme posterior to conduct force-alignment~(FA) and provide timestamps, while end-to-end ASR systems especially AED based ones are short of such ability.
Evolved Part Masking for Self-Supervised Learning
The accuracy of partitioned parts is on par with the capability of the pre-trained model, leading to evolved mask patterns at different training stages.
3D Human Mesh Recovery with Sequentially Global Rotation Estimation
This pipeline needs to transform each relative rotation matrix into a global rotation matrix to articulate the canonical mesh, and suffers from accumulated errors along the kinematics chain.
MMSpeech: Multi-modal Multi-task Encoder-Decoder Pre-training for Speech Recognition
Therefore, we propose to introduce the phoneme modality into pre-training, which can help capture modality-invariant information between Mandarin speech and text.
Ranked #2 on
Speech Recognition
on AISHELL-1
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Deep Active Learning for Computer Vision: Past and Future
As an important data selection schema, active learning emerges as the essential component when iterating an Artificial Intelligence (AI) model.
ParCNetV2: Oversized Kernel with Enhanced Attention
Specifically, we propose a new convolutional neural network, ParCNetV2, that extends position-aware circular convolution (ParCNet) with oversized convolutions and bifurcate gate units to enhance attention.
A Comparative Study on Multichannel Speaker-Attributed Automatic Speech Recognition in Multi-party Meetings
Speaker-attributed automatic speech recognition (SA-ASR) in multi-party meeting scenarios is one of the most valuable and challenging ASR task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
ALBench: A Framework for Evaluating Active Learning in Object Detection
To facilitate the research in this field, this paper contributes an active learning benchmark framework named as ALBench for evaluating active learning in object detection.
Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition
However, due to an independence assumption within the output tokens, performance of single-step NAR is inferior to that of AR models, especially with a large-scale corpus.
A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings
Therefore, we propose the second approach, WD-SOT, to address alignment errors by introducing a word-level diarization model, which can get rid of such timestamp alignment dependency.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Speaker Embedding-aware Neural Diarization: an Efficient Framework for Overlapping Speech Diarization in Meeting Scenarios
Through this formulation, we propose the speaker embedding-aware neural diarization (SEND) framework, where a speech encoder, a speaker encoder, two similarity scorers, and a post-processing network are jointly optimized to predict the encoded labels according to the similarities between speech features and speaker embeddings.
Ranked #1 on
Speaker Diarization
on AliMeeting
Extended vehicle energy dataset (eVED): an enhanced large-scale dataset for deep learning on vehicle trip energy consumption
Compared with its original version, the extended VED (eVED) dataset is enhanced with accurate vehicle trip GPS coordinates, serving as a basis to associate the VED trip records with external information, e. g., road speed limit and intersections, from accessible map services to accumulate attributes that is essential in analyzing vehicle energy consumption.
ProsoSpeech: Enhancing Prosody With Quantized Vector Pre-training in Text-to-Speech
Specifically, we first introduce a word-level prosody encoder, which quantizes the low-frequency band of the speech and compresses prosody attributes in the latent prosody vector (LPV).
Contextualize differential privacy in image database: a lightweight image differential privacy approach based on principle component analysis inverse
To this end, we design a lightweight approach dedicating to privatizing image database as a whole and preserving the statistical semantics of the image database to an adjustable level, while making individual images' contribution to such statistics indistinguishable.
Contextual Instance Decoupling for Robust Multi-Person Pose Estimation
Instead of relying on person bounding boxes to spatially differentiate persons, CID decouples persons in an image into multiple instance-aware feature maps.
Robust Pose Estimation in Crowded Scenes with Direct Pose-Level Inference
Instead of inferring individual keypoints, the Pose-level Inference Network (PINet) directly infers the complete pose cues for a person from his/her visible body parts.
Speaker Embedding-aware Neural Diarization for Flexible Number of Speakers with Textual Information
In this paper, we reformulate this task as a single-label prediction problem by encoding the multi-speaker labels with power set.
An Energy Consumption Model for Electrical Vehicle Networks via Extended Federated-learning
The two components collaborate to enhance learning robustness against data heterogeneities in networks.
MDERank: A Masked Document Embedding Rank Approach for Unsupervised Keyphrase Extraction
In this work, we propose a novel unsupervised embedding-based KPE approach, Masked Document Embedding Rank (MDERank), to address this problem by leveraging a mask strategy and ranking candidates by the similarity between embeddings of the source document and the masked document.
BeamTransformer: Microphone Array-based Overlapping Speech Detection
We propose BeamTransformer, an efficient architecture to leverage beamformer's edge in spatial filtering and transformer's capability in context sequence modeling.
Enhancing Social Relation Inference with Concise Interaction Graph and Discriminative Scene Representation
There has been a recent surge of research interest in attacking the problem of social relation inference based on images.
MFGNet: Dynamic Modality-Aware Filter Generation for RGB-T Tracking
The visible and thermal filters will be used to conduct a dynamic convolutional operation on their corresponding input feature maps respectively.
Ranked #21 on
Rgb-T Tracking
on RGBT234
Large-Scale Spatio-Temporal Person Re-identification: Algorithms and Benchmark
We also verified that models pre-trained on LaST can generalize well on existing datasets with short-term and cloth-changing scenarios.
Graph Consistency Based Mean-Teaching for Unsupervised Domain Adaptive Person Re-Identification
Specifically, given unlabeled training images, we apply teacher networks to extract corresponding features and further construct a teacher graph for each teacher network to describe the similarity relationships among training images.
Contrastive Learning
Domain Adaptive Person Re-Identification
+2
AAformer: Auto-Aligned Transformer for Person Re-Identification
In this paper, we introduce an alignment scheme in Transformer architecture for the first time and propose the Auto-Aligned Transformer (AAformer) to automatically locate both the human parts and non-human ones at patch-level.
Intra-Inter Camera Similarity for Unsupervised Person Re-Identification
The second stage considers the classification scores of each sample on different cameras as a new feature vector.
Ranked #1 on
Person Re-Identification
on SYSU-30k
(using extra training data)
Viewpoint and Scale Consistency Reinforcement for UAV Vehicle Re-Identification
Moreover, our method also achieves competitive performance compared with recent works on existing vehicle ReID datasets including VehicleID, VeRi-776 and VERI-Wild.
Domain Adaptive Person Re-Identification via Coupling Optimization
Extensive experiments on three large-scale datasets, i. e., Market-1501, DukeMTMC-reID, and MSMT17, show that our coupling optimization outperforms state-of-the-art methods by a large margin.
Domain Adaptive Person Re-Identification
Transfer Learning
+1
Joint Visual and Temporal Consistency for Unsupervised Domain Adaptive Person Re-Identification
This paper tackles this challenge through jointly enforcing visual and temporal consistency in the combination of a local one-hot classification and a global multi-class classification.
Distillation Guided Residual Learning for Binary Convolutional Neural Networks
We observe that, this performance gap leads to substantial residuals between intermediate feature maps of BCNN and FCNN.
Streaming Chunk-Aware Multihead Attention for Online End-to-End Speech Recognition
Recently, streaming end-to-end automatic speech recognition (E2E-ASR) has gained more and more attention.
Sound Audio and Speech Processing
Simplified Self-Attention for Transformer-based End-to-End Speech Recognition
Transformer models have been introduced into end-to-end speech recognition with state-of-the-art performance on various tasks owing to their superiority in modeling long-term dependencies.
Unsupervised Person Re-identification via Multi-label Classification
Our label prediction and MMCL work iteratively and substantially boost the ReID performance.
Ranked #6 on
Unsupervised Domain Adaptation
on Duke to MSMT
Robust Partial Matching for Person Search in the Wild
This paper also contributes a Large-Scale dataset for Person Search in the wild (LSPS), which is by far the largest and the most challenging dataset for person search.
Neural Zero-Inflated Quality Estimation Model For Automatic Speech Recognition System
The performances of automatic speech recognition (ASR) systems are usually evaluated by the metric word error rate (WER) when the manually transcribed data are provided, which are, however, expensively available in the real scenario.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Global-Local Temporal Representations For Video Person Re-Identification
The long-term relations are captured by a temporal self-attention model to alleviate the occlusions and noises in video sequences.
Resolution-invariant Person Re-Identification
RIFE adopts two feature extraction streams weighted by a dual-attention block to learn features for low and high resolution images, respectively.
Automatic Spelling Correction with Transformer for CTC-based End-to-End Speech Recognition
Results in a 20, 000 hours Mandarin speech recognition task show that the proposed spelling correction model can achieve a CER of 3. 41%, which results in 22. 9% and 53. 2% relative improvement compared to the baseline CTC-based systems decoded with and without language model respectively.
Bi-Directional Cascade Network for Perceptual Edge Detection
Exploiting multi-scale representations is critical to improve edge detection for objects at different scales.
Ranked #2 on
Edge Detection
on BRIND
Multi-scale 3D Convolution Network for Video Based Person Re-Identification
A temporal stream in this network is constructed by inserting several Multi-scale 3D (M3D) convolution layers into a 2D CNN network.
RAM: A Region-Aware Deep Model for Vehicle Re-Identification
Specifically, in addition to extracting global features, RAM also extracts features from a series of local regions.
Deep-FSMN for Large Vocabulary Continuous Speech Recognition
In a 20000 hours Mandarin recognition task, the LFR trained DFSMN can achieve more than 20% relative improvement compared to the LFR trained BLSTM.
Deep Feed-forward Sequential Memory Networks for Speech Synthesis
The Bidirectional LSTM (BLSTM) RNN based speech synthesis system is among the best parametric Text-to-Speech (TTS) systems in terms of the naturalness of generated speech, especially the naturalness in prosody.
LVreID: Person Re-Identification with Long Sequence Videos
This paper mainly establishes a large-scale Long sequence Video database for person re-IDentification (LVreID).
Person Transfer GAN to Bridge Domain Gap for Person Re-Identification
Although the performance of person Re-Identification (ReID) has been significantly boosted, many challenging issues in real scenarios have not been fully investigated, e. g., the complex scenes and lighting variations, viewpoint and pose changes, and the large number of identities in a camera network.
Ranked #11 on
Unsupervised Person Re-Identification
on DukeMTMC-reID
(Rank-10 metric)
Pose-driven Deep Convolutional Model for Person Re-identification
Our deep architecture explicitly leverages the human part cues to alleviate the pose variations and learn robust feature representations from both the global image and different local parts.
Ranked #105 on
Person Re-Identification
on Market-1501
E$^2$BoWs: An End-to-End Bag-of-Words Model via Deep Convolutional Neural Network
To conquer these issues, we propose an End-to-End BoWs (E$^2$BoWs) model based on Deep Convolutional Neural Network (DCNN).
GLAD: Global-Local-Alignment Descriptor for Pedestrian Retrieval
Targeting to solve these problems, this work proposes a Global-Local-Alignment Descriptor (GLAD) and an efficient indexing and retrieval framework, respectively.
Ranked #93 on
Person Re-Identification
on Market-1501
One-Shot Fine-Grained Instance Retrieval
Aiming to conquer this issue, we propose a retrieval task named One-Shot Fine-Grained Instance Retrieval (OSFGIR).
Deep Representation Learning with Part Loss for Person Re-Identification
The representation learning risk is evaluated by the proposed part loss, which automatically generates several parts for an image, and computes the person classification loss on each part separately.
Ranked #97 on
Person Re-Identification
on Market-1501
DR2-Net: Deep Residual Reconstruction Network for Image Compressive Sensing
Accordingly, DR$^{2}$-Net consists of two components, \emph{i. e.,} linear mapping network and residual network, respectively.
Neural Networks Models for Entity Discovery and Linking
This paper describes the USTC_NELSLIP systems submitted to the Trilingual Entity Detection and Linking (EDL) track in 2016 TAC Knowledge Base Population (KBP) contests.
Deep Attributes Driven Multi-Camera Person Re-identification
And we propose a semi-supervised attribute learning framework which progressively boosts the accuracy of attributes only using a limited number of labeled data.
Feedforward Sequential Memory Networks: A New Structure to Learn Long-term Dependency
In this paper, we propose a novel neural network structure, namely \emph{feedforward sequential memory networks (FSMN)}, to model long-term dependency in time series without using recurrent feedback.
Multi-Task Learning With Low Rank Attribute Embedding for Person Re-Identification
Since attributes are generally correlated, we introduce a low rank attribute embedding into the MTL formulation to embed original binary attributes to a continuous attribute space, where incorrect and incomplete attributes are rectified and recovered to better describe people.
Feedforward Sequential Memory Neural Networks without Recurrent Feedback
We introduce a new structure for memory neural networks, called feedforward sequential memory networks (FSMN), which can learn long-term dependency without using recurrent feedback.
A Fixed-Size Encoding Method for Variable-Length Sequences with its Application to Neural Network Language Models
In this paper, we propose the new fixed-size ordinally-forgetting encoding (FOFE) method, which can almost uniquely encode any variable-length sequence of words into a fixed-size representation.
Hybrid Orthogonal Projection and Estimation (HOPE): A New Framework to Probe and Learn Neural Networks
As a result, the HOPE framework can be used as a novel tool to probe why and how NNs work, more importantly, to learn NNs in either supervised or unsupervised ways.
Ranked #23 on
Image Classification
on MNIST
