Search Results for author: Bei Li

Found 35 papers, 17 papers with code

Large Language Models are Parallel Multilingual Learners

1 code implementation14 Mar 2024 Yongyu Mu, Peinan Feng, Zhiquan Cao, Yuzhang Wu, Bei Li, Chenglong Wang, Tong Xiao, Kai Song, Tongran Liu, Chunliang Zhang, Jingbo Zhu

In this study, we reveal an in-context learning (ICL) capability of multilingual large language models (LLMs): by translating the input to several languages, we provide Parallel Input in Multiple Languages (PiM) to LLMs, which significantly enhances their comprehension abilities.

In-Context Learning

Rethinking and Improving Multi-task Learning for End-to-end Speech Translation

1 code implementation7 Nov 2023 Yuhao Zhang, Chen Xu, Bei Li, Hao Chen, Tong Xiao, Chunliang Zhang, Jingbo Zhu

Significant improvements in end-to-end speech translation (ST) have been achieved through the application of multi-task learning.

Multi-Task Learning

Incorporating Probing Signals into Multimodal Machine Translation via Visual Question-Answering Pairs

1 code implementation26 Oct 2023 Yuxin Zuo, Bei Li, Chuanhao Lv, Tong Zheng, Tong Xiao, Jingbo Zhu

This paper presents an in-depth study of multimodal machine translation (MMT), examining the prevailing understanding that MMT systems exhibit decreased sensitivity to visual information when text inputs are complete.

Attribute Multimodal Machine Translation +2

PartialFormer: Modeling Part Instead of Whole

1 code implementation23 Oct 2023 Tong Zheng, Bei Li, Huiwen Bao, Weiqiao Shan, Tong Xiao, Jingbo Zhu

The design choices in Transformer feed-forward neural networks have resulted in significant computational and parameter overhead.

Abstractive Text Summarization Machine Translation +1

Conversational Speech Recognition by Learning Audio-textual Cross-modal Contextual Representation

no code implementations22 Oct 2023 Kun Wei, Bei Li, Hang Lv, Quan Lu, Ning Jiang, Lei Xie

By introducing both cross-modal and conversational representations into the decoder, our model retains context over longer sentences without information loss, achieving relative accuracy improvements of 8. 8% and 23% on Mandarin conversation datasets HKUST and MagicData-RAMC, respectively, compared to the standard Conformer model.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +1

Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers

1 code implementation15 Sep 2023 Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, Yujiu Yang

Large Language Models (LLMs) excel in various tasks, but they rely on carefully crafted prompts that often demand substantial human effort.

Evolutionary Algorithms

ESRL: Efficient Sampling-based Reinforcement Learning for Sequence Generation

2 code implementations4 Aug 2023 Chenglong Wang, Hang Zhou, Yimin Hu, Yifu Huo, Bei Li, Tongran Liu, Tong Xiao, Jingbo Zhu

Applying Reinforcement Learning (RL) to sequence generation models enables the direct optimization of long-term rewards (\textit{e. g.,} BLEU and human feedback), but typically requires large-scale sampling over a space of action sequences.

Abstractive Text Summarization Language Modelling +5

Deliberate then Generate: Enhanced Prompting Framework for Text Generation

no code implementations31 May 2023 Bei Li, Rui Wang, Junliang Guo, Kaitao Song, Xu Tan, Hany Hassan, Arul Menezes, Tong Xiao, Jiang Bian, Jingbo Zhu

Large language models (LLMs) have shown remarkable success across a wide range of natural language generation tasks, where proper prompt designs make great impacts.

Text Generation

ManagerTower: Aggregating the Insights of Uni-Modal Experts for Vision-Language Representation Learning

1 code implementation31 May 2023 Xiao Xu, Bei Li, Chenfei Wu, Shao-Yen Tseng, Anahita Bhiwandiwalla, Shachar Rosenman, Vasudev Lal, Wanxiang Che, Nan Duan

With only 4M VLP data, ManagerTower achieves superior performances on various downstream VL tasks, especially 79. 15% accuracy on VQAv2 Test-Std, 86. 56% IR@1 and 95. 64% TR@1 on Flickr30K.

Representation Learning

TranSFormer: Slow-Fast Transformer for Machine Translation

no code implementations26 May 2023 Bei Li, Yi Jing, Xu Tan, Zhen Xing, Tong Xiao, Jingbo Zhu

Learning multiscale Transformer models has been evidenced as a viable approach to augmenting machine translation systems.

Machine Translation Translation

EIT: Enhanced Interactive Transformer

1 code implementation20 Dec 2022 Tong Zheng, Bei Li, Huiwen Bao, Tong Xiao, Jingbo Zhu

In this paper, we propose a novel architecture, the Enhanced Interactive Transformer (EIT), to address the issue of head degradation in self-attention mechanisms.

Abstractive Text Summarization Language Modelling +2

Learning Multiscale Transformer Models for Sequence Generation

1 code implementation19 Jun 2022 Bei Li, Tong Zheng, Yi Jing, Chengbo Jiao, Tong Xiao, Jingbo Zhu

In this work, we define those scales in different linguistic units, including sub-words, words and phrases.

On Vision Features in Multimodal Machine Translation

2 code implementations ACL 2022 Bei Li, Chuanhao Lv, Zefan Zhou, Tao Zhou, Tong Xiao, Anxiang Ma, Jingbo Zhu

Previous work on multimodal machine translation (MMT) has focused on the way of incorporating vision features into translation but little attention is on the quality of vision models.

Image Captioning Multimodal Machine Translation +3

The NiuTrans System for the WMT21 Efficiency Task

1 code implementation16 Sep 2021 Chenglong Wang, Chi Hu, Yongyu Mu, Zhongxiang Yan, Siming Wu, Minyi Hu, Hang Cao, Bei Li, Ye Lin, Tong Xiao, Jingbo Zhu

This paper describes the NiuTrans system for the WMT21 translation efficiency task (http://statmt. org/wmt21/efficiency-task. html).

Knowledge Distillation Translation

High-resolution chirplet transform: from parameters analysis to parameters combination

no code implementations2 Aug 2021 Xiangxiang Zhu, Bei Li, Kunde Yang, Zhuosheng Zhang, Wenting Li

The standard chirplet transform (CT) with a chirp-modulated Gaussian window provides a valuable tool for analyzing linear chirp signals.

Vocal Bursts Intensity Prediction

Learning Light-Weight Translation Models from Deep Transformer

1 code implementation27 Dec 2020 Bei Li, Ziyang Wang, Hui Liu, Quan Du, Tong Xiao, Chunliang Zhang, Jingbo Zhu

We proposed a novel group-permutation based knowledge distillation approach to compressing the deep Transformer model into a shallow model.

Knowledge Distillation Machine Translation +2

Shallow-to-Deep Training for Neural Machine Translation

1 code implementation EMNLP 2020 Bei Li, Ziyang Wang, Hui Liu, Yufan Jiang, Quan Du, Tong Xiao, Huizhen Wang, Jingbo Zhu

We find that stacking layers is helpful in improving the representation ability of NMT models and adjacent layers perform similarly.

Machine Translation NMT +2

A multilayer interstitial fluid flow along vascular adventitia

no code implementations23 Sep 2020 Hongyi Li, You Lv, Xiaoliang Chen, Bei Li, Qi Hua, Fusui Ji, Yajun Yin, Hua Li

In real-time observations, the calculated velocity of a continuous ISF flow along fibers of a PACT pathway was 3. 6-15. 6 mm/sec.

Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation

1 code implementation ACL 2020 Bei Li, Hui Liu, Ziyang Wang, Yufan Jiang, Tong Xiao, Jingbo Zhu, Tongran Liu, Changliang Li

In encoder-decoder neural models, multiple encoders are in general used to represent the contextual information in addition to the individual sentence.

Machine Translation NMT +2

The NiuTrans Machine Translation System for WMT18

no code implementations WS 2018 Qiang Wang, Bei Li, Jiqiang Liu, Bojian Jiang, Zheyang Zhang, Yinqiao Li, Ye Lin, Tong Xiao, Jingbo Zhu

This paper describes the submission of the NiuTrans neural machine translation system for the WMT 2018 Chinese ↔ English news translation tasks.

Machine Translation Translation

Cannot find the paper you are looking for? You can Submit a new open access paper.