Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey

1 code implementation • 21 Nov 2023 • Yunpeng Huang, Jingwei Xu, Junyu Lai, Zixu Jiang, Taolue Chen, Zenan Li, Yuan YAO, Xiaoxing Ma, Lijuan Yang, Hao Chen, Shupeng Li, Penghao Zhao

Transformer-based Large Language Models (LLMs) have been applied in diverse areas such as knowledge bases, human interfaces, and dynamic agents, and marking a stride towards achieving Artificial General Intelligence (AGI).

Navigate