1 code implementation • 31 Dec 2023 • Zhouhan Lin, Cheng Deng, Le Zhou, Tianhang Zhang, Yi Xu, Yutong Xu, Zhongmou He, Yuanyuan Shi, Beiya Dai, Yunchong Song, Boyi Zeng, Qiyuan Chen, Yuxun Miao, Bo Xue, Shu Wang, Luoyi Fu, Weinan Zhang, Junxian He, Yunqiang Zhu, Xinbing Wang, Chenghu Zhou
To our best knowledge, it is the largest language model for the geoscience domain.
no code implementations • 13 Nov 2023 • Ziwei He, Jian Yuan, Le Zhou, Jingwen Leng, Bo Jiang
The quadratic complexity of self-attention in Transformers has hindered the processing of long text.