1 code implementation • 21 Nov 2022 • Yongyu Yan, Kui Xue, Xiaoming Shi, Qi Ye, Jingping Liu, Tong Ruan
Continual pretraining is a popular way of building a domain-specific pretrained language model from a general-domain language model.
Continual Pretraining Language Modelling