no code implementations • 26 Jan 2024 • Sicong Cao, Xiaobing Sun, Ratnadira Widyasari, David Lo, Xiaoxue Wu, Lili Bo, Jiale Zhang, Bin Li, Wei Liu, Di wu, Yixin Chen
The remarkable achievements of Artificial Intelligence (AI) algorithms, particularly in Machine Learning (ML) and Deep Learning (DL), have fueled their extensive deployment across multiple sectors, including Software Engineering (SE).
1 code implementation • 25 Oct 2023 • Xiaobing Sun, Jiaxi Li, Wei Lu
The underlying mechanism of neural networks in capturing precise knowledge has been the subject of consistent research efforts.
1 code implementation • NAACL 2022 • Xiaobing Sun, Wei Lu
Although self-attention based models such as Transformers have achieved remarkable successes on natural language processing (NLP) tasks, recent studies reveal that they have limitations on modeling sequential transformations (Hahn, 2020), which may prompt re-examinations of recurrent neural networks (RNNs) that demonstrated impressive results on handling sequential data.
no code implementations • 1 Jan 2021 • Xiaobing Sun, Wei Lu
Based on the closed-form approximations of the hidden states, we argue that the effectiveness of the cells may be attributed to a type of sequence-level representations brought in by the gating mechanism, which enables the cells to encode sequence-level features along with token-level features.
no code implementations • ACL 2020 • Xiaobing Sun, Wei Lu
Attention has been proven successful in many natural language processing (NLP) tasks.