Search Results for author:
Found 99 papers, 39 papers with code
The NiuTrans System for the WMT 2021 Efficiency Task
This paper describes the NiuTrans system for the WMT21 translation efficiency task.
The NiuTrans’s Submission to the IWSLT22 English-to-Chinese Offline Speech Translation Task
This paper describes NiuTrans’s submission to the IWSLT22 English-to-Chinese (En-Zh) offline speech translation task.
The NiuTrans System for the WMT20 Quality Estimation Shared Task
This paper describes the submissions of the NiuTrans Team to the WMT 2020 Quality Estimation Shared Task.
The NiuTrans Machine Translation Systems for WMT20
This paper describes NiuTrans neural machine translation systems of the WMT20 news translation tasks.
Efficient Prompting Methods for Large Language Models: A Survey
In this paper, we present a comprehensive overview of these methods.
Prior Constraints-based Reward Model Training for Aligning Large Language Models
Reinforcement learning with human feedback for aligning large language models (LLMs) trains a reward model typically using ranking loss with comparison pairs. However, the training procedure suffers from an inherent problem: the uncontrolled scaling of reward scores during reinforcement learning due to the lack of constraints while training the reward model. This paper proposes a Prior Constraints-based Reward Model (namely PCRM) training method to mitigate this problem.
RankPrompt: Step-by-Step Comparisons Make Language Models Better Reasoners
Our experiments across 11 arithmetic and commonsense reasoning tasks show that RankPrompt significantly enhances the reasoning performance of ChatGPT and GPT-4, with improvements of up to 13%.
Large Language Models are Parallel Multilingual Learners
In this study, we reveal an in-context learning (ICL) capability of multilingual large language models (LLMs): by translating the input to several languages, we provide Parallel Input in Multiple Languages (PiM) to LLMs, which significantly enhances their comprehension abilities.
Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation
The second step involves preserving dataset diversity through a clustering process. In our experiment, CaR selected a subset containing only 1. 96% of Alpaca's IT data, yet the underlying AlpaCaR model trained on this subset outperforms Alpaca by an average of 32. 1% in GPT-4 evaluations.
Effect of target signals and delays on spatially selective active noise control for open-fitting hearables
Spatially selective active noise control (ANC) hearables are designed to reduce unwanted noise from certain directions while preserving desired sounds from other directions.
Fairy: Fast Parallelized Instruction-Guided Video-to-Video Synthesis
In this paper, we introduce Fairy, a minimalist yet robust adaptation of image-editing diffusion models, enhancing them for video editing applications.
Soft Alignment of Modality Space for End-to-end Speech Translation
End-to-end Speech Translation (ST) aims to convert speech into target text within a unified model.
Introduction to Transformers: an NLP Perspective
Transformers have dominated empirical machine learning models of natural language processing.
Rethinking and Improving Multi-task Learning for End-to-end Speech Translation
Significant improvements in end-to-end speech translation (ST) have been achieved through the application of multi-task learning.
Incorporating Probing Signals into Multimodal Machine Translation via Visual Question-Answering Pairs
This paper presents an in-depth study of multimodal machine translation (MMT), examining the prevailing understanding that MMT systems exhibit decreased sensitivity to visual information when text inputs are complete.
PartialFormer: Modeling Part Instead of Whole
The design choices in Transformer feed-forward neural networks have resulted in significant computational and parameter overhead.
Ranked #23 on
Machine Translation
on WMT2014 English-German
Bridging the Gaps of Both Modality and Language: Synchronous Bilingual CTC for Speech Translation and Speech Recognition
In this study, we present synchronous bilingual Connectionist Temporal Classification (CTC), an innovative framework that leverages dual CTC to bridge the gaps of both modality and language in the speech translation (ST) task.
Learning Evaluation Models from Large Language Models for Sequence Generation
Large language models achieve state-of-the-art performance on sequence generation evaluation, but typically have a large number of parameters.
ESRL: Efficient Sampling-based Reinforcement Learning for Sequence Generation
Applying Reinforcement Learning (RL) to sequence generation models enables the direct optimization of long-term rewards (\textit{e. g.,} BLEU and human feedback), but typically requires large-scale sampling over a space of action sequences.
Towards Robust Aspect-based Sentiment Analysis through Non-counterfactual Augmentations
While state-of-the-art NLP models have demonstrated excellent performance for aspect based sentiment analysis (ABSA), substantial evidence has been presented on their lack of robustness.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+2
Recent Advances in Direct Speech-to-text Translation
Recently, speech-to-text translation has attracted more and more attention and many studies have emerged rapidly.
Understanding Parameter Sharing in Transformers
Inspired by this, we tune the training hyperparameters related to model convergence in a targeted manner.
Modality Adaption or Regularization? A Case Study on End-to-End Speech Translation
Pre-training and fine-tuning is a paradigm for alleviating the data scarcity problem in end-to-end speech translation (E2E ST).
MobileNMT: Enabling Translation in 15MB and 30ms
With the co-design of model and engine, compared with the existing system, we speed up 47. 0x and save 99. 5% of memory with only 11. 6% loss of BLEU.
Deliberate then Generate: Enhanced Prompting Framework for Text Generation
Large language models (LLMs) have shown remarkable success across a wide range of natural language generation tasks, where proper prompt designs make great impacts.
CTC-based Non-autoregressive Speech Translation
Combining end-to-end speech translation (ST) and non-autoregressive (NAR) generation is promising in language and speech processing for their advantages of less error propagation and low latency.
Bridging the Granularity Gap for Acoustic Modeling
While Transformer has become the de-facto standard for speech, modeling upon the fine-grained frame-level features remains an open challenge of capturing long-distance dependencies and distributing the attention weights.
Augmenting Large Language Model Translators via Translation Memories
Using translation memories (TMs) as prompts is a promising approach to in-context learning of machine translation models.
TranSFormer: Slow-Fast Transformer for Machine Translation
Learning multiscale Transformer models has been evidenced as a viable approach to augmenting machine translation systems.
Multi-Path Transformer is Better: A Case Study on Neural Machine Translation
For years the model performance in machine learning obeyed a power-law relationship with the model size.
Character, Word, or Both? Revisiting the Segmentation Granularity for Chinese Pre-trained Language Models
We conduct extensive experiments on various Chinese NLP tasks to evaluate existing PLMs as well as the proposed MigBERT.
A Novel Approach for Auto-Formulation of Optimization Problems
In the Natural Language for Optimization (NL4Opt) NeurIPS 2022 competition, competitors focus on improving the accessibility and usability of optimization solvers, with the aim of subtask 1: recognizing the semantic entities that correspond to the components of the optimization problem; subtask 2: generating formulations for the optimization problem.
Improved Knowledge Distillation for Pre-trained Language Models via Knowledge Selection
Knowledge distillation addresses the problem of transferring knowledge from a teacher model to a student model.
Prompting Neural Machine Translation with Translation Memories
Improving machine translation (MT) systems with translation memories (TMs) is of great interest to practitioners in the MT community.
EIT: Enhanced Interactive Transformer
In this paper, we propose a novel architecture, the Enhanced Interactive Transformer (EIT), to address the issue of head degradation in self-attention mechanisms.
Improving End-to-end Speech Translation by Leveraging Auxiliary Speech and Text Data
We present a method for introducing a text encoder into pre-trained end-to-end speech translation systems.
Spatially Selective Active Noise Control Systems
In this work, we propose a multi-channel ANC system that only reduces sound from undesired directions, and the system truly preserves the desired sound instead of reproducing it.
Learning Multiscale Transformer Models for Sequence Generation
In this work, we define those scales in different linguistic units, including sub-words, words and phrases.
On Vision Features in Multimodal Machine Translation
Previous work on multimodal machine translation (MMT) has focused on the way of incorporating vision features into translation but little attention is on the quality of vision models.
ODE Transformer: An Ordinary Differential Equation-Inspired Model for Sequence Generation
Inspired by this, we design a new architecture, {\it ODE Transformer}, which is analogous to the Runge-Kutta method that is well motivated in ODE.
The NiuTrans Machine Translation Systems for WMT21
This paper describes NiuTrans neural machine translation systems of the WMT 2021 news translation tasks.
The NiuTrans System for the WMT21 Efficiency Task
This paper describes the NiuTrans system for the WMT21 translation efficiency task (http://statmt. org/wmt21/efficiency-task. html).
The NiuTrans System for WNGT 2020 Efficiency Task
This paper describes the submissions of the NiuTrans Team to the WNGT 2020 Efficiency Shared Task.
RankNAS: Efficient Neural Architecture Search by Pairwise Ranking
This paper addresses the efficiency challenge of Neural Architecture Search (NAS) by formulating the task as a ranking problem.
Bag of Tricks for Optimizing Transformer Efficiency
Improving Transformer efficiency has become increasingly attractive recently.
The NiuTrans End-to-End Speech Translation System for IWSLT 2021 Offline Task
This paper describes the submission of the NiuTrans end-to-end speech translation system for the IWSLT 2021 offline task, which translates from the English audio to German text directly without intermediate transcription.
Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders
To our knowledge, we are the first to develop an end-to-end ST system that achieves comparable or even better BLEU performance than the cascaded ST counterpart when large-scale ASR and MT data is available.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
ODE Transformer: An Ordinary Differential Equation-Inspired Model for Neural Machine Translation
We show that a residual block of layers in Transformer can be described as a higher-order solution to ODEs.
Non-Autoregressive Translation by Learning Target Categorical Codes
Non-autoregressive Transformer is a promising text generation model.
Ranked #7 on
Machine Translation
on WMT2014 German-English
An Efficient Transformer Decoder with Compressed Sub-layers
The large attention-based encoder-decoder network (Transformer) has become prevailing recently due to its effectiveness.
Learning Light-Weight Translation Models from Deep Transformer
We proposed a novel group-permutation based knowledge distillation approach to compressing the deep Transformer model into a shallow model.
Dynamic Curriculum Learning for Low-Resource Neural Machine Translation
This eases training by highlighting easy samples that the current model has enough competence to learn.
A Simple and Effective Approach to Robust Unsupervised Bilingual Dictionary Induction
Our experiments show that this simple method does not hamper the performance of similar language pairs and achieves an accuracy of 13. 64~55. 53% between English and four distant languages, i. e., Chinese, Japanese, Vietnamese and Thai.
Layer-Wise Multi-View Learning for Neural Machine Translation
In this way, in addition to the topmost encoder layer (referred to as the primary view), we also incorporate an intermediate encoder layer as the auxiliary view.
Training Flexible Depth Model by Multi-Task Learning for Neural Machine Translation
The standard neural machine translation model can only decode with the same depth configuration as training.
Shallow-to-Deep Training for Neural Machine Translation
We find that stacking layers is helpful in improving the representation ability of NMT models and adjacent layers perform similarly.
Weight Distillation: Transferring the Knowledge in Neural Network Parameters
Inspired by this, we investigate methods of model acceleration and compression in another line of research.
Towards Fully 8-bit Integer Inference for the Transformer Model
8-bit integer inference, as a promising direction in reducing both the latency and storage of deep neural networks, has made great progress recently.
Geometric Correspondence Fields: Learned Differentiable Rendering for 3D Pose Refinement in the Wild
We present a novel 3D pose refinement approach based on differentiable rendering for objects of arbitrary categories in the wild.
MOOCCube: A Large-scale Data Repository for NLP Applications in MOOCs
The prosperity of Massive Open Online Courses (MOOCs) provides fodder for many NLP and AI research for education applications, e. g., course concept extraction, prerequisite relation discovery, etc.
Towards Differentially Private Text Representations
Most deep learning frameworks require users to pool their local data or model updates to a trusted server to train or maintain a global model.
Attention: to Better Stand on the Shoulders of Giants
In the progress of science, the previously discovered knowledge principally inspires new scientific ideas, and citation is a reasonably good reflection of this cumulative nature of scientific research.
Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation
In encoder-decoder neural models, multiple encoders are in general used to represent the contextual information in addition to the individual sentence.
Learning Architectures from an Extended Search Space for Language Modeling
Neural architecture search (NAS) has advanced significantly in recent years but most NAS systems restrict search to learning architectures of a recurrent or convolutional cell.
Neural Machine Translation with Joint Representation
Though early successes of Statistical Machine Translation (SMT) systems are attributed in part to the explicit modelling of the interaction between any two source and target units, e. g., alignment, the recent Neural Machine Translation (NMT) systems resort to the attention which partially encodes the interaction for efficiency.
Multi-layer Representation Fusion for Neural Machine Translation
In this paper, we propose a multi-layer representation fusion (MLRF) approach to fusing stacked layers.
Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition
In this paper, we study differentiable neural architecture search (NAS) methods for natural language processing.
Ranked #1 on
Language Modelling
on PTB Diagnostic ECG Database
Ultra-broadband local active noise control with remote acoustic sensing
One enduring challenge for controlling high frequency sound in local active noise control (ANC) systems is to obtain the acoustic signal at the specific location to be controlled.
The NiuTrans Machine Translation Systems for WMT19
We participated in 13 translation directions, including 11 supervised tasks, namely EN↔{ZH, DE, RU, KK, LT}, GU→EN and the unsupervised DE↔CS sub-track.
Sharing Attention Weights for Fast Transformer
This is even 16 times faster than the baseline with no use of the attention cache.
Shared-Private Bilingual Word Embeddings for Neural Machine Translation
For similar source and target words, their embeddings tend to share a part of the features and they cooperatively learn these common representation units.
Learning Deep Transformer Models for Machine Translation
Transformer is the state-of-the-art model in recent machine translation evaluations.
Modeling and Predicting Citation Count via Recurrent Neural Network with Long Short-Term Memory
The rapid evolution of scientific research has been creating a huge volume of publications every year.
Digital Libraries Physics and Society
The NiuTrans Machine Translation System for WMT18
This paper describes the submission of the NiuTrans neural machine translation system for the WMT 2018 Chinese ↔ English news translation tasks.
End-to-End Deep Kronecker-Product Matching for Person Re-identification
Person re-identification aims to robustly measure similarities between person images.
Deep Group-shuffling Random Walk for Person Re-identification
Person re-identification aims at finding a person of interest in an image gallery by comparing the probe image of this person with all the gallery images.
A Simple and Effective Approach to Coverage-Aware Neural Machine Translation
We offer a simple and effective method to seek a better balance between model confidence and length preference for Neural Machine Translation (NMT).
Video Person Re-Identification With Competitive Snippet-Similarity Aggregation and Co-Attentive Snippet Embedding
The attention weights are obtained based on a query feature, which is learned from the whole probe snippet by an LSTM network, making the resulting embeddings less affected by noisy frames.
Ranked #4 on
Person Re-Identification
on PRID2011
Implicit Syntactic Features for Target-dependent Sentiment Analysis
Targeted sentiment analysis investigates the sentiment polarities on given target mentions from input texts.
Learning Deep Neural Networks for Vehicle Re-ID with Visual-spatio-temporal Path Proposals
Vehicle re-identification is an important problem and has many applications in video surveillance and intelligent transportation.
Identity-Aware Textual-Visual Matching with Latent Co-attention
The stage-2 CNN-LSTM network refines the matching results with a latent co-attention mechanism.
Towards Bidirectional Hierarchical Representations for Attention-Based Neural Machine Translation
This paper proposes a hierarchical attentional neural translation model which focuses on enhancing source-side hierarchical representations by covering both local and global semantic information using a bidirectional tree-based encoder.
Object Detection in Videos with Tubelet Proposal Networks
Object detection in videos has drawn increasing attention recently with the introduction of the large-scale ImageNet VID dataset.
Person Search with Natural Language Description
Searching persons in large-scale image databases with the query of natural language description has important applications in video surveillance.
Crafting GBD-Net for Object Detection
The effectiveness of GBD-Net is shown through experiments on three object detection datasets, ImageNet, Pascal VOC2007 and Microsoft COCO.
Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification
Learning generic and robust feature representations with data from multiple domains for the same problem is of great value, especially for the problems that have multiple datasets but none of them are large enough to provide abundant data variations.
T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos
Temporal and contextual information of videos are not fully investigated and utilized.
Joint Detection and Identification Feature Learning for Person Search
Existing person re-identification benchmarks and methods mainly focus on matching cropped pedestrian images between queries and candidates.
Ranked #9 on
Person Re-Identification
on CUHK03
Convolutional neural networks with low-rank regularization
Recently, tensor decompositions have been used for speeding up CNNs.
Learning From Massive Noisy Labeled Data for Image Classification
To demonstrate the effectiveness of our approach, we collect a large-scale real-world clothing classification dataset with both noisy and clean labels.
DeepReID: Deep Filter Pairing Neural Network for Person Re-Identification
In this paper, we propose a novel filter pairing neural network (FPNN) to jointly handle misalignment, photometric and geometric transforms, occlusions and background clutter.
