Search Results for author: Xuzheng He

Found 3 papers, 1 papers with code

Demonstrating Mutual Reinforcement Effect through Information Flow

no code implementations • 5 Mar 2024 • Chengguang Gan, Xuzheng He, Qinghao Zhang, Tatsunori Mori

The Mutual Reinforcement Effect (MRE) investigates the synergistic relationship between word-level and text-level classifications in text classification tasks.

text-classification Text Classification

Paper
Add Code

StableMask: Refining Causal Masking in Decoder-only Transformer

no code implementations • 7 Feb 2024 • Qingyu Yin, Xuzheng He, Xiang Zhuang, Yu Zhao, Jianhua Yao, Xiaoyu Shen, Qiang Zhang

The decoder-only Transformer architecture with causal masking and relative position encoding (RPE) has become the de facto choice in language modeling.

Language Modelling Position

Paper
Add Code

RWKV: Reinventing RNNs for the Transformer Era

5 code implementations • 22 May 2023 • Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, Kranthi Kiran GV, Xuzheng He, Haowen Hou, Jiaju Lin, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartlomiej Koptyra, Hayden Lau, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Bolun Wang, Johan S. Wind, Stanislaw Wozniak, Ruichong Zhang, Zhenyuan Zhang, Qihang Zhao, Peng Zhou, Qinghua Zhou, Jian Zhu, Rui-Jie Zhu

This work presents a significant step towards reconciling trade-offs between computational efficiency and model performance in sequence processing tasks.

Ranked #22 on Natural Language Inference on WNLI

Computational Efficiency Natural Language Inference

11,631

Paper
Code

Cannot find the paper you are looking for? You can Submit a new open access paper.