Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work?

ACL 2020 Yada PruksachatkunJason PhangHaokun LiuPhu Mon HtutXiaoyi ZhangRichard Yuanzhe PangClara VaniaKatharina KannSamuel R. Bowman

While pretrained models such as BERT have shown large gains across natural language understanding tasks, their performance can be improved by further training the model on a data-rich intermediate task, before fine-tuning it on a target task. However, it is still poorly understood when and why intermediate-task training is beneficial for a given target task... (read more)

PDF Abstract

Results from the Paper


  Submit results from this paper to get state-of-the-art GitHub badges and help the community compare results to other papers.

Methods used in the Paper