Rethinking Exposure Bias In Language Modeling

no code implementations • 13 Oct 2019 • Yifan Xu, Kening Zhang, Haoyu Dong, Yuezhou Sun, Wenlong Zhao, Zhuowen Tu

Exposure bias describes the phenomenon that a language model trained under the teacher forcing schema may perform poorly at the inference stage when its predictions are conditioned on its previous predictions unseen from the training corpus.

Language Modelling Reinforcement Learning (RL)

Neural Program Synthesis By Self-Learning

no code implementations • 13 Oct 2019 • Yifan Xu, Lu Dai, Udaikaran Singh, Kening Zhang, Zhuowen Tu

Neural inductive program synthesis is a task generating instructions that can produce desired outputs from given inputs.

Program Synthesis Reinforcement Learning (RL) +1