Taking Notes on the Fly Helps BERT Pre-training

4 Aug 2020Qiyu WuChen XingYatao LiGuolin KeDi HeTie-Yan Liu

How to make unsupervised language pre-training more efficient and less resource-intensive is an important research direction in NLP. In this paper, we focus on improving the efficiency of language pre-training methods through providing better data utilization... (read more)

PDF Abstract


No code implementations yet. Submit your code now


Results from the Paper

  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

Methods used in the Paper