Time-aware Large Kernel Convolutions

8 Feb 2020Vasileios LioutasYuhong Guo

To date, most state-of-the-art sequence modeling architectures use attention to build generative models for language based tasks. Some of these models use all the available sequence tokens to generate an attention distribution which results in time complexity of $O(n^2)$... (read more)

PDF Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Document Summarization CNN / Daily Mail TaLK Convolutions (Standard) ROUGE-1 40.03 # 11
ROUGE-2 18.45 # 11
ROUGE-L 36.13 # 14
Document Summarization CNN / Daily Mail TaLK Convolutions (Deep) ROUGE-1 40.59 # 10
ROUGE-2 18.97 # 8
ROUGE-L 36.81 # 11
Machine Translation IWSLT2014 German-English TaLK Convolutions BLEU score 35.5 # 4
Machine Translation WMT2014 English-French TaLK Convolutions BLEU score 43.2 # 6
Machine Translation WMT2014 English-German TaLK Convolutions BLEU score 29.6 # 10

Methods used in the Paper