Multi-Head Attention

Introduced by Vaswani et al. in Attention Is All You Need

Multi-head Attention is a module for attention mechanisms which runs through an attention mechanism several times in parallel. The independent attention outputs are then concatenated and linearly transformed into the expected dimension. Intuitively, multiple attention heads allows for attending to parts of the sequence differently (e.g. longer-term dependencies versus shorter-term dependencies).

$$ \text{MultiHead}\left(\textbf{Q}, \textbf{K}, \textbf{V}\right) = \left[\text{head}_{1},\dots,\text{head}_{h}\right]\textbf{W}_{0}$$

$$\text{where} \text{ head}_{i} = \text{Attention} \left(\textbf{Q}\textbf{W}_{i}^{Q}, \textbf{K}\textbf{W}_{i}^{K}, \textbf{V}\textbf{W}_{i}^{V} \right) $$

Above $\textbf{W}$ are all learnable parameter matrices.

Note that scaled dot-product attention is most commonly used in this module, although in principle it can be swapped out for other types of attention mechanism.

Source: Lilian Weng

Source: Attention Is All You Need

Latest Papers

PAPER DATE
MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering
Shayne LongpreYi LuJoachim Daiber
2020-07-30
Depressive, Drug Abusive, or Informative: Knowledge-aware Study of News Exposure during COVID-19 Outbreak
Amanuel AlamboManas GaurKrishnaprasad Thirunarayan
2020-07-30
What does BERT know about books, movies and music? Probing BERT for Conversational Recommendation
| Gustavo PenhaClaudia Hauff
2020-07-30
Interpretable Contextual Team-aware Item Recommendation: Application in Multiplayer Online Battle Arena Games
Andrés VillaVladimir AraujoFrancisca CattanDenis Parra
2020-07-30
Deep Multi-View Spatiotemporal Virtual Graph Neural Network for Significant Citywide Ride-hailing Demand Prediction
Guangyin JinZhexu XiHengyu ShaYanghe FengJincai Huang
2020-07-30
Composer Style Classification of Piano Sheet Music Images Using Language Model Pretraining
TJ TsaiKevin Ji
2020-07-29
GUIR at SemEval-2020 Task 12: Domain-Tuned Contextualized Models for Offensive Language Detection
Sajad SotudehTong XiangHao-Ren YaoSean MacAvaneyEugene YangNazli GoharianOphir Frieder
2020-07-28
Variants of BERT, Random Forests and SVM approach for Multimodal Emotion-Target Sub-challenge
Hoang Manh HungHyung-Jeong YangSoo-Hyung KimGuee-Sang Lee
2020-07-28
BUT-FIT at SemEval-2020 Task 5: Automatic detection of counterfactual statements with deep pre-trained language representation models
Martin FajcikJosef JonMartin DocekalPavel Smrz
2020-07-28
Improving Results on Russian Sentiment Datasets
| Anton GolubevNatalia Loukachevitch
2020-07-28
Reed at SemEval-2020 Task 9: Sentiment Analysis on Code-Mixed Tweets
Vinay GopalanMark Hopkins
2020-07-26
KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media
| Ali SafayaMoutasem AbdullatifDeniz Yuret
2020-07-26
FiSSA at SemEval-2020 Task 9: Fine-tuned For Feelings
| Bertelt BraaksmaRichard ScholtensStan van SuijlekomRemy WangAhmet Üstün
2020-07-24
MULTISEM at SemEval-2020 Task 3: Fine-tuning BERT for Lexical Meaning
Aina Garí SolerMarianna Apidianaki
2020-07-24
Product Title Generation for Conversational Systems using BERT
Mansi Ranjit ManeShashank KediaAditya ManthaStephen GuoKannan Achan
2020-07-23
The Lottery Ticket Hypothesis for Pre-trained BERT Networks
| Tianlong ChenJonathan FrankleShiyu ChangSijia LiuYang ZhangZhangyang WangMichael Carbin
2020-07-23
CrossTransformers: spatially-aware few-shot transfer
Carl DoerschAnkush GuptaAndrew Zisserman
2020-07-22
IITK at the FinSim Task: Hypernym Detection in Financial Domain via Context-Free and Contextualized Word Embeddings
Vishal KeswaniSakshi SinghAshutosh Modi
2020-07-22
Analogical Reasoning for Visually Grounded Language Acquisition
Bo WuHaoyu QinAlireza ZareianCarl VondrickShih-Fu Chang
2020-07-22
Neural Machine Translation with Error Correction
Kaitao SongXu TanJianfeng Lu
2020-07-21
SliceOut: Training Transformers and CNNs faster while using less memory
Pascal NotinAidan N. GomezJoanna YooYarin Gal
2020-07-21
problemConquero at SemEval-2020 Task 12: Transformer and Soft label-based approaches
Karishma LaudJagriti SinghRandeep Kumar SahuAshutosh Modi
2020-07-21
newsSweeper at SemEval-2020 Task 11: Context-Aware Rich Feature Representations For Propaganda Classification
| Paramansh SinghSiraj SandhuSubham KumarAshutosh Modi
2020-07-21
Word Representation for Rhythms
Tongyu LuLyucheng YanGus Xia
2020-07-21
Understanding BERT Rankers Under Distillation
Luyu GaoZhuyun DaiJamie Callan
2020-07-21
Learning Joint Spatial-Temporal Transformations for Video Inpainting
| Yanhong ZengJianlong FuHongyang Chao
2020-07-20
A Comparison of Supervised Learning to Match Methods for Product Search
| Fatemeh SarviNikos VoskaridesLois MooimanSebastian SchelterMaarten de Rijke
2020-07-20
Conformer-Kernel with Query Term Independence for Document Retrieval
| Bhaskar MitraSebastian HofstatterHamed ZamaniNick Craswell
2020-07-20
Mono vs Multilingual Transformer-based Models: a Comparison across Several Language Tasks
Diego de Vargas FeijoViviane Pereira Moreira
2020-07-19
Temporal Pointwise Convolutional Networks for Length of Stay Prediction in the Intensive Care Unit
| Emma RocheteauPietro LiòStephanie Hyland
2020-07-18
Feature Pyramid Transformer
Dong ZhangHanwang ZhangJinhui TangMeng WangXiansheng HuaQianru Sun
2020-07-18
Generative Pretraining from Pixels
| Mark ChenAlec RadfordRewon ChildJeff WuHeewoo JunPrafulla DhariwalDavid LuanIlya Sutskever
2020-07-17
Deep Learning Based Traffic Surveillance System For Missing and Suspicious Car Detection
K. V. KadambariVishnu Vardhan Nimmalapudi
2020-07-17
CTC-Segmentation of Large Corpora for German End-to-end Speech Recognition
Ludwig KürzingerDominik WinkelbauerLujun LiTobias WatzelGerhard Rigoll
2020-07-17
Investigating Pretrained Language Models for Graph-to-Text Generation
Leonardo F. R. RibeiroMartin SchmittHinrich SchützeIryna Gurevych
2020-07-16
Towards Debiasing Sentence Representations
Paul Pu LiangIrene Mengze LiEmily ZhengYao Chong LimRuslan SalakhutdinovLouis-Philippe Morency
2020-07-16
Translate Reverberated Speech to Anechoic Ones: Speech Dereverberation with BERT
Yang Jiao
2020-07-16
AdapterHub: A Framework for Adapting Transformers
| Jonas PfeifferAndreas RückléClifton PothAishwarya KamathIvan VulićSebastian RuderKyunghyun ChoIryna Gurevych
2020-07-15
Multimodal Word Sense Disambiguation in Creative Practice
Manuel Ladron de GuevaraChristopher GeorgeAkshat GuptaDaragh ByrneRamesh Krishnamurti
2020-07-15
Logic Constrained Pointer Networks for Interpretable Textual Similarity
| Subhadeep MajiRohan KumarManish BansalKalyani RoyPawan Goyal
2020-07-15
Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks
Pavel BlinovManvel AvetisianVladimir KokhDmitry UmerenkovAlexander Tuzhilin
2020-07-15
Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media
Alberto Barron-CedenoTamer ElsayedPreslav NakovGiovanni Da San MartinoMaram HasanainReem SuwailehFatima HaouariNikolay BabulkovBayan HamdanAlex NikolovShaden ShaarZien Sheikh Ali
2020-07-15
Deep Reinforced Query Reformulation for Information Retrieval
Xiao WangCraig MacdonaldIadh Ounis
2020-07-15
The Monte Carlo Transformer: a stochastic self-attention model for sequence prediction
Alice MartinCharles OllionFlorian StrubSylvain Le CorffOlivier Pietquin
2020-07-15
Add a SideNet to your MainNet
Adrien Morisot
2020-07-14
Fast and Accurate Neural CRF Constituency Parsing
| Yu ZhangHouquan ZhouZhenghua Li
2020-07-14
Deep Transformer based Data Augmentation with Subword Units for Morphologically Rich Online ASR
Balázs TarjánGyörgy SzaszákTibor FegyóPéter Mihajlik
2020-07-14
Contextualized Code Representation Learning for Commit Message Generation
Lun Yiu NieCuiyun GaoZhicong ZhongWai LamYang LiuZenglin Xu
2020-07-14
What's in a Name? Are BERT Named Entity Representations just as Good for any other Name?
Sriram BalasubramanianNaman JainGaurav JindalAbhijeet AwasthiSunita Sarawagi
2020-07-14
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models
Lifu TuGarima LalwaniSpandana GellaHe He
2020-07-14
Can neural networks acquire a structural bias from raw linguistic data?
Alex WarstadtSamuel R. Bowman
2020-07-14
Emoji Prediction: Extensions and Benchmarking
Weicheng MaRuibo LiuLili WangSoroush Vosoughi
2020-07-14
Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning
Shauharda KhadkaEstelle AflaloMattias MarderAvrech Ben-DavidSantiago MiretHanlin TangShie MannorTamir HazanSomdeb Majumdar
2020-07-14
Transformer with Depth-Wise LSTM
Hongfei XuQiuhui LiuDeyi XiongJosef van Genabith
2020-07-13
An Enhanced Text Classification to Explore Health based Indian Government Policy Tweets
Aarzoo DhimanDurga Toshniwal
2020-07-13
Paranoid Transformer: Reading Narrative of Madness as Computational Approach to Creativity
Yana AgafonovaAlexey TikhonovIvan P. Yamshchikov
2020-07-13
Sparse Graph to Sequence Learning for Vision Conditioned Long Textual Sequence Generation
Aditya MogadalaMarius MosbachDietrich Klakow
2020-07-12
TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech
| Andy T. LiuShang-Wen LiHung-yi Lee
2020-07-12
HyperGrid: Efficient Multi-Task Transformers with Grid-wise Decomposable Hyper Projections
Yi TayZhe ZhaoDara BahriDonald MetzlerDa-Cheng Juan
2020-07-12
Sequence Generation with Mixed Representations
Lijun Wu Shufang Xie Yingce Xia Fan Yang Tao Qin Jianhuang Lai Tie-Yan Liu
2020-07-11
Generative Graph Perturbations for Scene Graph Prediction
Boris KnyazevHarm de VriesCătălina CangeaGraham W. TaylorAaron CourvilleEugene Belilovsky
2020-07-11
BISON:BM25-weighted Self-Attention Framework for Multi-Fields Document Search
Xuan ShanChuanjie LiuYiqian XiaQi ChenYusi ZhangAngen LuoYuxiang Luo
2020-07-10
To BAN or not to BAN: Bayesian Attention Networks for Reliable Hate Speech Detection
Kristian MiokBlaz SkrljDaniela ZaharieMarko Robnik-Sikonja
2020-07-10
Multi-Dialect Arabic BERT for Country-Level Dialect Identification
| Bashar TalafhaMohammad AliMuhy Eddin Za'terHaitham SeelawiIbraheem TuffahaMostafa SamirWael FarhanHussein T. Al-Natsheh
2020-07-10
DeepSinger: Singing Voice Synthesis with Data Mined From the Web
Yi RenXu TanTao QinJian LuanZhou ZhaoTie-Yan Liu
2020-07-09
Contrastive Code Representation Learning
| Paras JainAjay JainTianjun ZhangPieter AbbeelJoseph E. GonzalezIon Stoica
2020-07-09
Fast Transformers with Clustered Attention
| Apoorv VyasAngelos KatharopoulosFrançois Fleuret
2020-07-09
Advances of Transformer-Based Models for News Headline Generation
| Alexey BukhtiyarovIlya Gusev
2020-07-09
The Go Transformer: Natural Language Modeling for Game Play
Matthew CiolinoDavid NoeverJosh Kalin
2020-07-07
Continual BERT: Continual Learning for Adaptive Extractive Summarization of COVID-19 Literature
Jong Won Park
2020-07-07
Do Transformers Need Deep Long-Range Memory
Jack W. RaeAli Razavi
2020-07-07
Exploring Heterogeneous Information Networks via Pre-Training
Yang FangXiang ZhaoWeidong Xiao
2020-07-07
Relevance Transformer: Generating Concise Code Snippets with Relevance Feedback
Carlos GemmellFederico RossettoJeffrey Dalton
2020-07-06
LMVE at SemEval-2020 Task 4: Commonsense Validation and Explanation using Pretraining Language Model
Shilei LiuYu GuoBochao LiFeiliang Ren
2020-07-06
Learning to Segment Anatomical Structures Accurately from One Exemplar
Yuhang LuWeijian LiKang ZhengYirui WangAdam P. HarrisonChihung LinSong WangJing XiaoLe LuChang-Fu KuoShun Miao
2020-07-06
Deep Contextual Embeddings for Address Classification in E-commerce
Shreyas MangalgiLakshya KumarRavindra Babu Tallamraju
2020-07-06
You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion
Roei SchusterCongzheng SongEran TromerVitaly Shmatikov
2020-07-05
Text Data Augmentation: Towards better detection of spear-phishing emails
Mehdi ReginaMaxime MeyerSébastien Goutal
2020-07-04
Robust Prediction of Punctuation and Truecasing for Medical ASR
Monica SunkaraSrikanth RonankiKalpit DixitSravan BodapatiKatrin Kirchhoff
2020-07-04
Language-agnostic BERT Sentence Embedding
| Fangxiaoyu FengYinfei YangDaniel CerNaveen ArivazhaganWei Wang
2020-07-03
Pretrained Semantic Speech Embeddings for End-to-End Spoken Language Understanding via Cross-Modal Teacher-Student Learning
Pavel DenisovNgoc Thang Vu
2020-07-03
Reading Comprehension in Czech via Machine Translation and Cross-lingual Transfer
Kateřina MackováMilan Straka
2020-07-03
Playing with Words at the National Library of Sweden -- Making a Swedish BERT
| Martin MalmstenLove BörjesonChris Haffenden
2020-07-03
On-The-Fly Information Retrieval Augmentation for Language Models
Hai WangDavid McAllester
2020-07-03
MIRA: Leveraging Multi-Intention Co-click Information in Web-scale Document Retrieval using Deep Neural Networks
Yusi ZhangChuanjie LiuAngen LuoHui XueXuan ShanYuxiang LuoYiqian XiaYuanchi YanHaidong Wang
2020-07-03
Abstractive and mixed summarization for long-single documents
Roger BarrullJugal Kalita
2020-07-03
Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey
Shivaji AlaparthiManit Mishra
2020-07-02
The Impact of Explanations on AI Competency Prediction in VQA
Kamran AlipourArijit RayXiao LinJurgen P. SchulzeYi YaoGiedrius T. Burachas
2020-07-02
Improving Event Detection using Contextual Word and Sentence Embeddings
Mariano MaisonnaveFernando DelbiancoFernando TohméAna MaguitmanEvangelos Milios
2020-07-02
Interpretable Operational Risk Classification with Semi-Supervised Variational Autoencoder
Fan ZhouShengming ZhangYi Yang
2020-07-01
Self-Attention Guided Copy Mechanism for Abstractive Summarization
Song XuHaoran LiPeng YuanYouzheng WuXiaodong HeBowen Zhou
2020-07-01
Integrating Multimodal Information in Large Pretrained Transformers
Wasifur RahmanMd Kamrul HasanSangwu LeeAmirAli Bagher ZadehChengfeng MaoLouis-Philippe MorencyEhsan Hoque
2020-07-01
Roles and Utilization of Attention Heads in Transformer-based Neural Language Models
Jae-young JoSung-Hyon Myaeng
2020-07-01
Towards Holistic and Automatic Evaluation of Open-Domain Dialogue Generation
Bo PangErik NijkampWenjuan HanLinqi ZhouYixian LiuKewei Tu
2020-07-01
Multimodal Transformer for Multimodal Machine Translation
Shaowei YaoXiaojun Wan
2020-07-01
Paraphrase Generation by Learning How to Edit from Samples
Amirhossein KazemnejadMohammadreza SalehiMahdieh Soleymani Baghshah
2020-07-01
A Mixture of h - 1 Heads is Better than h Heads
Hao PengRoy SchwartzDianqi LiNoah A. Smith
2020-07-01
Dependency Graph Enhanced Dual-transformer Structure for Aspect-based Sentiment Classification
Hao TangDonghong JiChenliang LiQiji Zhou
2020-07-01
Do Transformers Need Deep Long-Range Memory?
Jack RaeAli Razavi
2020-07-01
In Neural Machine Translation, What Does Transfer Learning Transfer?
Alham Fikri AjiNikolay BogoychevKenneth HeafieldRico Sennrich
2020-07-01
Feature Projection for Improved Text Classification
Qi QinWenpeng HuBing Liu
2020-07-01
Addressing Posterior Collapse with Mutual Information for Improved Variational Neural Machine Translation
Arya D. McCarthyXian LiJiatao GuNing Dong
2020-07-01
DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation
| Yizhe ZhangSiqi SunMichel GalleyYen-Chun ChenChris BrockettXiang GaoJianfeng GaoJingjing LiuBill Dolan
2020-07-01
Combining Subword Representations into Word-level Representations in the Transformer Architecture
Noe CasasMarta R. Costa-juss{\`a}Jos{\'e} A. R. Fonollosa
2020-07-01
Unsupervised Multilingual Sentence Embeddings for Parallel Corpus Mining
Ivana Kvapil{\'\i}kov{\'a}Mikel ArtetxeGorka LabakaEneko AgirreOnd{\v{r}}ej Bojar
2020-07-01
Robust Neural Machine Translation with ASR Errors
Haiyang XueYang FengShuhao GuWei Chen
2020-07-01
An empirical investigation of neural methods for content scoring of science explanations
Brian RiordanSarah BichlerAllison BradfordJennifer King ChenKorah WileyLibby GerardMarcia C. Linn
2020-07-01
Neural Transduction of Letter Position Dyslexia using an Anagram Matrix Representation
Avi Bleiweiss
2020-07-01
Detecting Sarcasm in Conversation Context Using Transformer-Based Models
Adithya AvvaruSanath VobilisettyRadhika Mamidi
2020-07-01
Character aware models with similarity learning for metaphor detection
Tarun KumarYashvardhan Sharma
2020-07-01
Metaphor Detection Using Contextual Word Embeddings From Transformers
Jerry LiuNathan O{'}HaraAlex RubinerRachel DraelosCynthia Rudin
2020-07-01
A Transformer Approach to Contextual Sarcasm Detection in Twitter
Hunter GregorySteven LiPouya MohammadiNatalie TarnRachel DraelosCynthia Rudin
2020-07-01
Adaptation of Multilingual Transformer Encoder for Robust Enhanced Universal Dependency Parsing
Han HeJinho D. Choi
2020-07-01
KIT's IWSLT 2020 SLT Translation System
Ngoc-Quan PhamFelix SchneiderTuan-Nam NguyenThanh-Le HaThai Son NguyenMaximilian AwiszusSebastian St{\"u}kerAlex Waibeler
2020-07-01
End-to-End Simultaneous Translation System for IWSLT2020 Using Modality Agnostic Meta-Learning
Hou Jeung HanMohd Abbas ZaidiSathish Reddy IndurthiNikhil Kumar LakumarapuBeomseok LeeSangha Kim
2020-07-01
End-to-End Offline Speech Translation System for IWSLT 2020 using Modality Agnostic Meta-Learning
Nikhil Kumar LakumarapuBeomseok LeeSathish Reddy IndurthiHou Jeung HanMohd Abbas ZaidiSangha Kim
2020-07-01
SRPOL's System for the IWSLT 2020 End-to-End Speech Translation Task
Tomasz PotapczykPawel Przybysz
2020-07-01
The AFRL IWSLT 2020 Systems: Work-From-Home Edition
Brian OreEric HansenTim AndersonJeremy Gwinnup
2020-07-01
OPPO's Machine Translation System for the IWSLT 2020 Open Domain Translation Task
Qian ZhangXiaopu LiDawei DangTingxun ShiDi AiZhengshan XueJie Hao
2020-07-01
CASIA's System for IWSLT 2020 Open Domain Translation
Qian WangYuchen LiuCong MaYu LuYining WangLong ZhouYang ZhaoJiajun ZhangChengqing Zong
2020-07-01
Deep Blue Sonics' Submission to IWSLT 2020 Open Domain Translation Task
Enmin SuYi Ren
2020-07-01
University of Tsukuba's Machine Translation System for IWSLT20 Open Domain Translation Task
Hongyi CuiYizhen WeiShohei IidaTakehito UtsuroMasaaki Nagata
2020-07-01
Xiaomi's Submissions for IWSLT 2020 Open Domain Translation Task
Yuhui SunMengxue GuoXiang LiJianwei CuiBin Wang
2020-07-01
The HW-TSC Video Speech Translation System at IWSLT 2020
Minghan WangHao YangYao DengYing QinLizhi LeiDaimeng WeiHengchao ShangNing XieXiaochun LiJiaxian Guo
2020-07-01
Towards Stream Translation: Adaptive Computation Time for Simultaneous Machine Translation
Felix SchneiderAlex Waibeler
2020-07-01
Compressing Neural Machine Translation Models with 4-bit Precision
Alham Fikri AjiKenneth Heafield
2020-07-01
Training and Inference Methods for High-Coverage Neural Machine Translation
Michael YangYixin LiuRahul Mayuranath
2020-07-01
POSTECH Submission on Duolingo Shared Task
Junsu ParkHongseok KwonJong-Hyeok Lee
2020-07-01
Expand and Filter: CUNI and LMU Systems for the WNGT 2020 Duolingo Shared Task
Jind{\v{r}}ich Libovick{\'y}Zden{\v{e}}k KasnerJind{\v{r}}ich HelclOnd{\v{r}}ej Du{\v{s}}ek
2020-07-01
The NiuTrans System for WNGT 2020 Efficiency Task
Chi HuBei LiYinqiao LiYe LinYanyang LiChenglong WangTong XiaoJingbo Zhu
2020-07-01
Efficient and High-Quality Neural Machine Translation with OpenNMT
Guillaume KleinDakun ZhangCl{\'e}ment ChouteauJosep CregoJean Senellart
2020-07-01
Improving Document-Level Neural Machine Translation with Domain Adaptation
Sami Ul HaqSadaf Abdul RaufArslan ShoukatNoor-e- Hira
2020-07-01
CopyBERT: A Unified Approach to Question Generation with Self-Attention
Stalin VaranasiSaadullah AminGuenter Neumann
2020-07-01
How to Tame Your Data: Data Augmentation for Dialog State Tracking
Adam SummervilleJordan HashemiJames Ryanwilliam ferguson
2020-07-01
Methods for Extracting Information from Messages from Primary Care Providers to Specialists
Xiyu DingMichael BarnettAteev MehrotraTimothy Miller
2020-07-01
Generating Medical Reports from Patient-Doctor Conversations Using Sequence-to-Sequence Models
Seppo EnarviMarilisa AmoiaMiguel Del-Agua TebaBrian DelaneyFrank DiehlStefan HahnKristina HarrisLiam McGrathYue PanJoel PintoLuca RubiniMiguel RuizGag SingheepFabian StemmerWeiyi SunPaul VozilaThomas LinRanjani Ramamurthy
2020-07-01
Enhancing Transformer with Sememe Knowledge
Yuhui ZhangChenghao YangZhengping ZhouZhiyuan Liu
2020-07-01
Grapheme-to-Phoneme Conversion with a Multilingual Transformer Model
Omnia ElSaadanyBenjamin Suter
2020-07-01
Frustratingly Easy Multilingual Grapheme-to-Phoneme Conversion
Nikhil PrabhuKatharina Kann
2020-07-01
Leveraging Principal Parts for Morphological Inflection
Ling LiuMans Hulden
2020-07-01
Data Augmentation for Transformer-based G2P
Zach RyanMans Hulden
2020-07-01
HausaMT v1.0: Towards English--Hausa Neural Machine Translation
Adewale Akinfaderin
2020-07-01
On-The-Fly Information Retrieval Augmentation for Language Models
Hai WangDavid McAllester
2020-07-01
Investigating the effect of auxiliary objectives for the automated grading of learner English speech transcriptions
Hannah CraigheadAndrew CainesPaula ButteryHelen Yannakoudakis
2020-07-01
Unsupervised FAQ Retrieval with Question Generation and BERT
Yosi MassBoaz CarmeliHaggai RoitmanDavid Konopnicki
2020-07-01
GAN-BERT: Generative Adversarial Learning for Robust Text Classification with a Bunch of Labeled Examples
Danilo CroceGiuseppe CastellucciRoberto Basili
2020-07-01
Modelling Context and Syntactical Features for Aspect-based Sentiment Analysis
Minh Hieu PhanPhilip O. Ogunbona
2020-07-01
Adversarial and Domain-Aware BERT for Cross-Domain Sentiment Analysis
Chunning DuHaifeng SunJingyu WangQi QiJianxin Liao
2020-07-01
How does BERT's attention change when you fine-tune? An analysis methodology and a case study in negation scope
Yiyun ZhaoSteven Bethard
2020-07-01
Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work?
Yada PruksachatkunJason PhangHaokun LiuPhu Mon HtutXiaoyi ZhangRichard Yuanzhe PangClara VaniaKatharina KannSamuel R. Bowman
2020-07-01
Would you Rather? A New Benchmark for Learning Machine Alignment with Cultural Values and Social Preferences
Yi TayDonovan OngJie FuAlvin ChanNancy ChenAnh Tuan LuuChris Pal
2020-07-01
Towards Debiasing Sentence Representations
Paul Pu LiangIrene Mengze LiEmily ZhengYao Chong LimRuslan SalakhutdinovLouis-Philippe Morency
2020-07-01
Automatic Generation of Citation Texts in Scholarly Papers: A Pilot Study
Xinyu XingXiaosheng FanXiaojun Wan
2020-07-01
Transition-based Semantic Dependency Parsing with Pointer Networks
Daniel Fern{\'a}ndez-Gonz{\'a}lezCarlos G{\'o}mez-Rodr{\'\i}guez
2020-07-01
tBERT: Topic Models and BERT Joining Forces for Semantic Similarity Detection
Nicole PeineltDong NguyenMaria Liakata
2020-07-01
Understanding Advertisements with BERT
Kanika KalraBhargav KurmaSilpa Vadakkeeveetil SreelathaManasi PatwardhanKarShirish e
2020-07-01
A Generate-and-Rank Framework with Semantic Type Regularization for Biomedical Concept Normalization
Dongfang XuZeyu ZhangSteven Bethard
2020-07-01
Revisiting Higher-Order Dependency Parsers
Erick FonsecaAndr{\'e} F. T. Martins
2020-07-01
SUPP.AI: finding evidence for supplement-drug interactions
Lucy WangOyvind TafjordArman CohanSarthak JainSam SkjonsbergCarissa SchoenickNick BotnerWaleed Ammar
2020-07-01
Why is penguin more similar to polar bear than to sea gull? Analyzing conceptual knowledge in distributional models
Pia Sommerauer
2020-07-01
A Simple and Effective Dependency Parser for Telugu
Sneha NallaniManish ShrivastavaDipti Sharma
2020-07-01
Cross-Lingual Disaster-related Multi-label Tweet Classification with Manifold Mixup
Jishnu Ray ChowdhuryCornelia CarageaDoina Caragea
2020-07-01
Should You Fine-Tune BERT for Automated Essay Scoring?
Elijah MayfieldAlan W Black
2020-07-01
A BERT-based One-Pass Multi-Task Model for Clinical Temporal Relation Extraction
Chen LinTimothy MillerDmitriy DligachFarig SadequeSteven BethardGuergana Savova
2020-07-01
Evaluating the Utility of Model Configurations and Data Augmentation on Clinical Semantic Textual Similarity
Yuxia WangFei LiuKarin VerspoorTimothy Baldwin
2020-07-01
Item-based Collaborative Filtering with BERT
Tian WangYuyangzi Fu
2020-07-01
Sarcasm Identification and Detection in Conversion Context using BERT
Kalaivani A.Thenmozhi D.
2020-07-01
Neural Sarcasm Detection using Conversation Context
Nikhil Jaiswal
2020-07-01
Context-Aware Sarcasm Detection Using BERT
Arup BaruahKaushik DasFerdous BarbhuiyaKuntal Dey
2020-07-01
IlliniMet: Illinois System for Metaphor Detection with Contextual and Linguistic Information
Hongyu GongKshitij GuptaAkriti JainSuma Bhat
2020-07-01
Go Figure! Multi-task transformer-based architecture for metaphor detection using idioms: ETS team in 2020 metaphor shared task
Xianyang ChenChee Wee (Ben) LeongMichael FlorBeata Beigman Klebanov
2020-07-01
Turku Enhanced Parser Pipeline: From Raw Text to Enhanced Graphs in the IWPT 2020 Shared Task
Jenna KanervaFilip GinterSampo Pyysalo
2020-07-01
K\opsala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding
Daniel HershcovichMiryam de LhoneuxArtur KulmizevElham PejhanJoakim Nivre
2020-07-01
RobertNLP at the IWPT 2020 Shared Task: Surprisingly Simple Enhanced UD Parsing for English
Stefan Gr{\"u}newaldAnnemarie Friedrich
2020-07-01
Robust Prediction of Punctuation and Truecasing for Medical ASR
Monica SunkaraSrikanth RonankiKalpit DixitSravan BodapatiKatrin Kirchhoff
2020-07-01
Exploring the Limits of Simple Learners in Knowledge Distillation for Document Classification with DocBERT
Ashutosh AdhikariAchyudh RamRaphael TangWilliam L. HamiltonJimmy Lin
2020-07-01
Joint Training with Semantic Role Labeling for Better Generalization in Natural Language Inference
Cemil CengizDeniz Yuret
2020-07-01
A Metric Learning Approach to Misogyny Categorization
Juan Manuel CoriaSahar GhannaySophie RossetHerv{\'e} Bredin
2020-07-01
Contextual and Non-Contextual Word Embeddings: an in-depth Linguistic Investigation
Alessio MiaschiFelice Dell{'}Orletta
2020-07-01
What's in a Name? Are BERT Named Entity Representations just as Good for any other Name?
Sriram BalasubramanianNaman JainGaurav JindalAbhijeet AwasthiSunita Sarawagi
2020-07-01
Getting the \#\#life out of living: How Adequate Are Word-Pieces for Modelling Complex Morphology?
Stav KleinReut Tsarfaty
2020-07-01
SentiTel: TABSA for Twitter reviews on Uganda Telecoms
David KabiitoJoyce Nakatumba Nabende
2020-07-01
Adversarial Evaluation of BERT for Biomedical Named Entity Recognition
Vladimir AraujoAndr{\'e}s CarvalloDenis Parra
2020-07-01
Improving Multimodal Named Entity Recognition via Entity Span Detection with Unified Multimodal Transformer
Jianfei YuJing JiangLi YangRui Xia
2020-07-01
Probing for Referential Information in Language Models
Ionut-Teodor SorodocKristina GulordavaGemma Boleda
2020-07-01
LSTM and GPT-2 Synthetic Speech Transfer Learning for Speaker Recognition to Overcome Data Scarcity
Jordan J. BirdDiego R. FariaAnikó EkártCristiano PremebidaPedro P. S. Ayrosa
2020-07-01
The Summary Loop: Learning to Write Abstractive Summaries Without Examples
| Philippe LabanAndrew HsiJohn CannyMarti A. Hearst
2020-07-01
Image-level Harmonization of Multi-Site Data using Image-and-Spatial Transformer Networks
| R. RobinsonQ. DouD. C. CastroK. KamnitsasM. de GrootR. M. SummersD. RueckertB. Glocker
2020-06-30
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
| Dmitry LepikhinHyoukJoong LeeYuanzhong XuDehao ChenOrhan FiratYanping HuangMaxim KrikunNoam ShazeerZhifeng Chen
2020-06-30
Correction of Faulty Background Knowledge based on Condition Aware and Revise Transformer for Question Answering
Xinyan ZhaoXiao FengHaoming ZhongJun YaoHuanhuan Chen
2020-06-30
SE3M: A Model for Software Effort Estimation Using Pre-trained Embedding Models
Eliane M. De Bortoli FáveroDalcimar CasanovaAndrey Ricardo Pimentel
2020-06-30
Data Movement Is All You Need: A Case Study on Optimizing Transformers
Andrei IvanovNikoli DrydenTal Ben-NunShigang LiTorsten Hoefler
2020-06-30
Segmentation Approach for Coreference Resolution Task
Aref JafariAli Ghodsi
2020-06-30
BERTERS: Multimodal Representation Learning for Expert Recommendation System with Transformer
N. Nikzad-KhasmakhiM. A. BalafarM. Reza Feizi-DerakhshiCina Motamed
2020-06-30
Simplifying Models with Unlabeled Output Data
Sang Michael XieTengyu MaPercy Liang
2020-06-29
Predicting Length of Stay in the Intensive Care Unit with Temporal Pointwise Convolutional Networks
| Emma RocheteauPietro LiòStephanie Hyland
2020-06-29
A Transformer-based joint-encoding for Emotion Recognition and Sentiment Analysis
| Jean-Benoit DelbrouckNoé TitsMathilde BrousmicheStéphane Dupont
2020-06-29
Multi-Head Attention: Collaborate Instead of Concatenate
| Jean-Baptiste CordonnierAndreas LoukasMartin Jaggi
2020-06-29
Want to Identify, Extract and Normalize Adverse Drug Reactions in Tweets? Use RoBERTa
Katikapalli Subramanyam KalyanS. Sangeetha
2020-06-29
Improving Sequence Tagging for Vietnamese Text Using Transformer-based Neural Models
Viet Bui TheOanh Tran ThiPhuong Le-Hong
2020-06-29
Knowledge-Aware Language Model Pretraining
Corby RossetChenyan XiongMinh PhanXia SongPaul BennettSaurabh Tiwary
2020-06-29
Interpreting Hierarchical Linguistic Interactions in DNNs
Die ZhangHuilin ZhouXiaoyi BaoDa HuoRuizhao ChenXu ChengHao ZhangMengyue WuQuanshi Zhang
2020-06-29
Progressive Generation of Long Text
| Bowen TanZichao YangMaruan AI-ShedivatEric P. XingZhiting Hu
2020-06-28
Rethinking Positional Encoding in Language Pre-training
| Guolin KeDi HeTie-Yan Liu
2020-06-28
Self-Attention Networks for Intent Detection
Sevinj YolchuyevaGéza NémethBálint Gyires-Tóth
2020-06-28
Bottom-Up Human Pose Estimation by Ranking Heatmap-Guided Adaptive Keypoint Estimates
| Ke SunZigang GengDepu MengBin XiaoDong LiuZhaoxiang ZhangJingdong Wang
2020-06-28
BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision
| Chen LiangYue YuHaoming JiangSiawpeng ErRuijia WangTuo ZhaoChao Zhang
2020-06-28
Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarization
Beliz GunelChenguang ZhuMichael ZengXuedong Huang
2020-06-27
Video-Grounded Dialogues with Pretrained Generation Language Models
Hung LeSteven C. H. Hoi
2020-06-27
Normalizador Neural de Datas e Endereços
Gustavo PlensackPaulo Finardi
2020-06-27
What they do when in doubt: a study of inductive biases in seq2seq learners
Eugene KharitonovRahma Chaabouni
2020-06-26
TURL: Table Understanding through Representation Learning
| Xiang DengHuan SunAlyssa LeesYou WuCong Yu
2020-06-26
BERTology Meets Biology: Interpreting Attention in Protein Language Models
| Jesse VigAli MadaniLav R. VarshneyCaiming XiongRichard SocherNazneen Fatema Rajani
2020-06-26
Conditional Set Generation with Transformers
Adam R KosiorekHyunjik KimDanilo J Rezende
2020-06-26
Learning Source Phrase Representations for Neural Machine Translation
Hongfei XuJosef van GenabithDeyi XiongQiuhui LiuJingyi Zhang
2020-06-25
Self-Segregating and Coordinated-Segregating Transformer for Focused Deep Multi-Modular Network for Visual Question Answering
Chiranjib Sur
2020-06-25
SACT: Self-Aware Multi-Space Feature Composition Transformer for Multinomial Attention for Video Captioning
Chiranjib Sur
2020-06-25
FastSpec: Scalable Generation and Detection of Spectre Gadgets Using Neural Embeddings
| M. Caner TolKoray YurtsevenBerk GulmezogluBerk Sunar
2020-06-25
Normalizing Text using Language Modelling based on Phonetics and String Similarity
Fenil DoshiJimit GandhiDeep GosaliaSudhir Bagul
2020-06-25
LSBert: A Simple Framework for Lexical Simplification
| Jipeng QiangYun LiYi ZhuYunhao YuanXindong Wu
2020-06-25
Differentiable Window for Dynamic Local Attention
Thanh-Tung NguyenXuan-Phi NguyenShafiq JotyXiaoli Li
2020-06-24
Accelerated Large Batch Optimization of BERT Pretraining in 54 minutes
Shuai ZhengHaibin LinSheng ZhaMu Li
2020-06-24
Efficient Constituency Parsing by Pointing
Thanh-Tung NguyenXuan-Phi NguyenShafiq JotyXiaoli Li
2020-06-24
Hybrid Spatio-Temporal Graph Convolutional Network: Improving Traffic Prediction with Navigation Data
| Rui DaiShenkun XuQian GuChenguang JiKaikui Liu
2020-06-23
Bach or Mock? A Grading Function for Chorales in the Style of J.S. Bach
| Alexander FangAlisa LiuPrem SeetharamanBryan Pardo
2020-06-23
Self-supervised edge features for improved Graph Neural Network training
| Arijit SehanobishNeal G. RavindraDavid van Dijk
2020-06-23
A Self-Attention Network based Node Embedding Model
Dai Quoc NguyenTu Dinh NguyenDinh Phung
2020-06-22
Exploring Software Naturalness through Neural Language Models
Luca BurattiSaurabh PujarMihaela BorneaScott McCarleyYunhui ZhengGaetano RossielloAlessandro MorariJim LaredoVeronika ThostYufan ZhuangGiacomo Domeniconi
2020-06-22
ReCO: A Large Scale Chinese Reading Comprehension Dataset on Opinion
| BingningWangTing YaoQi ZhangJingfang XuXiaochuan Wang
2020-06-22
Students Need More Attention: BERT-based AttentionModel for Small Data with Application to AutomaticPatient Message Triage
Shijing SiRui WangJedrek WosikHao ZhangDavid DovGuoyin WangRicardo HenaoLawrence Carin
2020-06-22
AdvAug: Robust Adversarial Augmentation for Neural Machine Translation
Yong ChengLu JiangWolfgang MachereyJacob Eisenstein
2020-06-21
The NYU-CUBoulder Systems for SIGMORPHON 2020 Task 0 and Task 2
Assaf SingerKatharina Kann
2020-06-21
Off-Policy Self-Critical Training for Transformer in Visual Paragraph Generation
Shiyang YanYang HuaNeil M. Robertson
2020-06-21
A Universal Representation Transformer Layer for Few-Shot Image Classification
| Lu LiuWilliam HamiltonGuodong LongJing JiangHugo Larochelle
2020-06-21
Memory Transformer
Mikhail S. BurtsevGrigory V. Sapunov
2020-06-20
Sarcasm Detection in Tweets with BERT and GloVe Embeddings
Akshay KhatriPranav PDr. Anand Kumar M
2020-06-20
A Qualitative Evaluation of Language Models on Automatic Question-Answering for COVID-19
| David OnianiYanshan Wang
2020-06-19
End-to-end deep metamodeling to calibrate and optimize energy loads
Max CohenMaurice CharbitSylvain Le CorffMarius PredaGilles Nozière
2020-06-19
New Vietnamese Corpus for Machine ReadingComprehension of Health News Articles
Kiet Van NguyenDuc-Vu NguyenAnh Gia-Tuan NguyenNgan Luu-Thuy Nguyen
2020-06-19
SqueezeBERT: What can computer vision teach NLP about efficient neural networks?
Forrest N. IandolaAlbert E. ShawRavi KrishnaKurt W. Keutzer
2020-06-19
Infinite attention: NNGP and NTK for deep attention networks
Jiri HronYasaman BahriJascha Sohl-DicksteinRoman Novak
2020-06-18
Boosting Objective Scores of Speech Enhancement Model through MetricGAN Post-Processing
Szu-Wei FuChien-Feng LiaoTsun-An HsiehKuo-Hsuan HungSyu-Siang WangCheng YuHeng-Cheng KuoRyandhimas E. ZezarioYou-Jin LiShang-Yi ChuangYen-Ju LuYu Tsao
2020-06-18
Multi-branch Attentive Transformer
| Yang FanShufang XieYingce XiaLijun WuTao QinXiang-Yang LiTie-Yan Liu
2020-06-18
I-BERT: Inductive Generalization of Transformer to Arbitrary Context Lengths
| Hyoungwook NamSeung Byum SeoVikram Sharma MailthodyNoor MichaelLan Li
2020-06-18
SEAL: Segment-wise Extractive-Abstractive Long-form Text Summarization
Yao ZhaoMohammad SalehPeter J. Liu
2020-06-18
Sparse GPU Kernels for Deep Learning
Trevor GaleMatei ZahariaCliff YoungErich Elsen
2020-06-18
SenWave: Monitoring the Global Sentiments under the COVID-19 Pandemic
Qiang YangHind AlamroSomayah AlbaradeiAdil SalhiXiaoting LvChangsheng MaManal AlshehriInji JaberFaroug TifrateneWei WangTakashi GojoboriCarlos M. DuarteXin GaoXiangliang Zhang
2020-06-18
Intelligent Protection & Classification of Transients in Two-Core Symmetric Phase Angle Regulating Transformers
Pallav Kumar BeraCan Isik
2020-06-17
Automatically Ranked Russian Paraphrase Corpus for Text Generation
Vadim GudkovOlga MitrofanovaElizaveta Filippskikh
2020-06-17
Learning Visual Commonsense for Robust Scene Graph Generation
Alireza ZareianZhecan WangHaoxuan YouShih-Fu Chang
2020-06-17
Exploring the BERT Cross-Lingual Transferability: a Case Study in Reading Comprehension
Konovalov V. P.Gulyaev P. A.Sorokin A. A.Kuratov Y. M.Burtsev M. S.
2020-06-17
Tagging and parsing of multidomain collections
| Alexey SorokinIvan SmurovDenis Kirianov
2020-06-17
Modeling Graph Structure via Relative Position for Better Text Generation from Knowledge Graphs
Martin SchmittLeonardo F. R. RibeiroPhilipp DufterIryna GurevychHinrich Schütze
2020-06-16
Improving accuracy and speeding up Document Image Classification through parallel systems
Javier FerrandoJuan Luis DominguezJordi TorresRaul GarciaDavid GarciaDaniel GarridoJordi CortadaMateo Valero
2020-06-16
PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models
| Eyal Ben-DavidCarmel RabinovitzRoi Reichart
2020-06-16
The SPPD System for Schema Guided Dialogue State Tracking Challenge
Miao LiHaoqi XiongYunbo Cao
2020-06-16
Scalable Cross Lingual Pivots to Model Pronoun Gender for Translation
Kellie WebsterEmily Pitler
2020-06-16
End-to-End Code Switching Language Models for Automatic Speech Recognition
Ahan M. R.Shreyas Sunil Kulkarni
2020-06-16
Fine-grained Human Evaluation of Transformer and Recurrent Approaches to Neural Machine Translation for English-to-Chinese
| Yuying YeAntonio Toral
2020-06-15
On the Multi-Property Extraction and Beyond
Tomasz DwojakMichał PietruszkaŁukasz BorchmannFilip GralińskiJakub Chłędowski
2020-06-15
Exploration of End-to-End ASR for OpenSTT -- Russian Open Speech-to-Text Dataset
Andrei AndrusenkoAleksandr LaptevIvan Medennikov
2020-06-15
Differentiable Neural Architecture Transformation for Reproducible Architecture Improvement
Do-Guk KimHeung-Chang Lee
2020-06-15
Multi-Image Summarization: Textual Summary from a Set of Cohesive Images
Nicholas TrieuSebastian GoodmanPradyumna NarayanaKazoo SoneRadu Soricut
2020-06-15
Document Classification for COVID-19 Literature
Bernal Jiménez GutiérrezJuncheng ZengDongdong ZhangPing ZhangYu Su
2020-06-15
FinBERT: A Pretrained Language Model for Financial Communications
| Yi YangMark Christopher Siy UYAllen Huang
2020-06-15
Cooking Is All About People: Comment Classification On Cookery Channels Using BERT and Classification Models (Malayalam-English Mix-Code)
Subramaniam KazhuparambilAbhishek Kaushik
2020-06-15
FinEst BERT and CroSloEngual BERT: less is more in multilingual models
Matej UlčarMarko Robnik-Šikonja
2020-06-14
Transferring Monolingual Model to Low-Resource Language: The Case of Tigrinya
Abrhalei TelaAbraham WoubieVille Hautamaki
2020-06-13
Guided Transformer: Leveraging Multiple External Sources for Representation Learning in Conversational Search
Helia HashemiHamed ZamaniW. Bruce Croft
2020-06-13
Temporal Fusion Network for Temporal Action Localization:Submission to ActivityNet Challenge 2020 (Task E)
Zhiwu QingXiang WangYongpeng SangChangxin GaoShiwei ZhangNong Sang
2020-06-13
Modelling High-Level Mathematical Reasoning in Mechanised Declarative Proofs
Wenda LiLei YuYuhuai WuLawrence C. Paulson
2020-06-13
Comparing Natural Language Processing Techniques for Alzheimer's Dementia Prediction in Spontaneous Speech
Thomas SearleZina IbrahimRichard Dobson
2020-06-12
Unmasking the Inductive Biases of Unsupervised Object Representations for Video Sequences
Marissa A. WeisKashyap ChittaYash SharmaWieland BrendelMatthias BethgeAndreas GeigerAlexander S. Ecker
2020-06-12
Implicit Kernel Attention
Kyungwoo SongYohan JungDongjun KimIl-Chul Moon
2020-06-11
Dance Revolution: Long Sequence Dance Generation with Music via Curriculum Learning
| Ruozi HuangHuang HuWei WuKei SawadaMi Zhang
2020-06-11
FastPitch: Parallel Text-to-speech with Pitch Prediction
Adrian Łańcucki
2020-06-11
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages
Pedro Javier Ortiz SuárezLaurent RomaryBenoît Sagot
2020-06-11
MC-BERT: Efficient Language Pre-Training via a Meta Controller
| Zhenhui XuLinyuan GongGuolin KeDi HeShuxin ZhengLiwei WangJiang BianTie-Yan Liu
2020-06-10
Extrapolation for Large-batch Training in Deep Learning
Tao LinLingjing KongSebastian U. StichMartin Jaggi
2020-06-10
Revisiting Few-sample BERT Fine-tuning
| Tianyi ZhangFelix WuArzoo KatiyarKilian Q. WeinbergerYoav Artzi
2020-06-10
Unsupervised Paraphrase Generation using Pre-trained Language Models
Chaitra HegdeShrikumar Patil
2020-06-09
Graph-Aware Transformer: Is Attention All Graphs Need?
Sanghyun YooYoung-Seok KimKang Hyun LeeKuhwan JeongJunhwi ChoiHoshik LeeYoung Sang Choi
2020-06-09
DyHGCN: A Dynamic Heterogeneous Graph Convolutional Network to Learn Users' Dynamic Preferences for Information Diffusion Prediction
Chunyuan YuanJiacheng LiWei ZhouYijun LuXiaodan ZhangSonglin Hu
2020-06-09
HausaMT v1.0: Towards English-Hausa Neural Machine Translation
Adewale Akinfaderin
2020-06-09
Few-Shot Generative Conversational Query Rewriting
| Shi YuJiahua LiuJingqin YangChenyan XiongPaul BennettJianfeng GaoZhiyuan Liu
2020-06-09
Linformer: Self-Attention with Linear Complexity
| Sinong WangBelinda Z. LiMadian KhabsaHan FangHao Ma
2020-06-08
Modeling Discourse Structure for Document-level Neural Machine Translation
Junxuan ChenXiang LiJiarui ZhangChulun ZhouJianwei CuiBin WangJinsong Su
2020-06-08
MultiSpeech: Multi-Speaker Text to Speech with Transformer
Mingjian ChenXu TanYi RenJin XuHao SunSheng ZhaoTao Qin
2020-06-08
Learning to Count Words in Fluent Speech enables Online Speech Recognition
| George SterpuChristian SaamNaomi Harte
2020-06-08
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines
| Marius MosbachMaksym AndriushchenkoDietrich Klakow
2020-06-08
Wat zei je? Detecting Out-of-Distribution Translations with Variational Transformers
Tim Z. XiaoAidan N. GomezYarin Gal
2020-06-08
BERT Loses Patience: Fast and Robust Inference with Early Exit
| Wangchunshu ZhouCanwen XuTao GeJulian McAuleyKe XuFuru Wei
2020-06-07
Learning Texture Transformer Network for Image Super-Resolution
| Fuzhi YangHuan YangJianlong FuHongtao LuBaining Guo
2020-06-07
Pre-training Polish Transformer-based Language Models at Scale
| Sławomir DadasMichał PerełkiewiczRafał Poświata
2020-06-07
Medical Concept Normalization in User Generated Texts by Learning Target Concept Embeddings
Katikapalli Subramanyam KalyanS. Sangeetha
2020-06-07
Challenges and Thrills of Legal Arguments
Anurag PallaproluRadha VaidyaAditya Swaroop Attawar
2020-06-06
Masked Language Modeling for Proteins via Linearly Scalable Long-Context Transformers
Krzysztof ChoromanskiValerii LikhosherstovDavid DohanXingyou SongJared DavisTamas SarlosDavid BelangerLucy ColwellAdrian Weller
2020-06-05
GMAT: Global Memory Augmentation for Transformers
| Ankit GuptaJonathan Berant
2020-06-05
Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing
| Zihang DaiGuokun LaiYiming YangQuoc V. Le
2020-06-05
An Overview of Neural Network Compression
James O' Neill
2020-06-05
Accelerating Natural Language Understanding in Task-Oriented Dialog
Ojas AhujaShrey Desai
2020-06-05
UDPipe at EvaLatin 2020: Contextualized Embeddings and Treebank Embeddings
Milan StrakaJana Straková
2020-06-05
DeBERTa: Decoding-enhanced BERT with Disentangled Attention
| Pengcheng HeXiaodong LiuJianfeng GaoWeizhu Chen
2020-06-05
End-to-End Speech-Translation with Knowledge Distillation: [email protected]
Marco GaidoMattia Antonino Di GangiMatteo NegriMarco Turchi
2020-06-04
The SOFC-Exp Corpus and Neural Approaches to Information Extraction in the Materials Science Domain
Annemarie FriedrichHeike AdelFederico TomazicJohannes HingerlRenou BenteauAnika MaruscykLukas Lange
2020-06-04
Automatic Text Summarization of COVID-19 Medical Research Articles using BERT and GPT-2
| Virapat KieuvongngamBowen TanYiming Niu
2020-06-03
On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior
Ethan Gotlieb WilcoxJon GauthierJennifer HuPeng QianRoger Levy
2020-06-02
Subjective Question Answering: Deciphering the inner workings of Transformers in the realm of subjectivity
Lukas Muttenthaler
2020-06-02
WikiBERT models: deep transfer learning for many languages
Sampo PyysaloJenna KanervaAntti VirtanenFilip Ginter
2020-06-02
Question Answering on Scholarly Knowledge Graphs
Mohamad Yaser JaradehMarkus StockerSören Auer
2020-06-02
A Pairwise Probe for Understanding BERT Fine-Tuning on Machine Reading Comprehension
Jie CaiZhengzhou ZhuPing NieQian Liu
2020-06-02
BERT Based Multilingual Machine Comprehension in English and Hindi
| Somil GuptaNilesh Khade
2020-06-02
Exploring Cross-sentence Contexts for Named Entity Recognition with BERT
Jouni LuomaSampo Pyysalo
2020-06-02
Position Masking for Language Models
Andy WagnerTiyasa MitraMrinal IyerGodfrey Da CostaMarc Tremblay
2020-06-02
Emergence of Separable Manifolds in Deep Language Representations
Jonathan MamouHang LeMiguel Del RioCory StephensonHanlin TangYoon KimSueYeon Chung
2020-06-01
Online Versus Offline NMT Quality: An In-depth Analysis on English-German and German-English
Maha ElbayadMichael UstaszewskiEmmanuelle Esperança-RodierFrancis Brunet ManquatLaurent Besacier
2020-06-01
Translating Natural Language Instructions for Behavioral Robot Navigation with a Multi-Head Attention Mechanism
Patricio Cerda-MardiniVladimir AraujoAlvaro Soto
2020-06-01
Context-based Transformer Models for Answer Sentence Selection
Ivano LauriolaAlessandro Moschitti
2020-06-01
Unsupervised Sparse-view Backprojection via Convolutional and Spatial Transformer Networks
Xueqing LiuPaul Sajda
2020-06-01
BERT-based Ensembles for Modeling Disclosure and Support in Conversational Social Media Text
Tanvi DaduKartikey PantRadhika Mamidi
2020-06-01
Image Search With Text Feedback by Visiolinguistic Attention Learning
| Yanbei Chen Shaogang Gong Loris Bazzani
2020-06-01
Few-Shot Learning of Part-Specific Probability Space for 3D Shape Segmentation
Lingjing Wang Xiang Li Yi Fang
2020-06-01
RDCFace: Radial Distortion Correction for Face Recognition
He Zhao Xianghua Ying Yongjie Shi Xin Tong Jingsi Wen Hongbin Zha
2020-06-01
ActBERT: Learning Global-Local Video-Text Representations
Linchao Zhu Yi Yang
2020-06-01
Conversational Machine Comprehension: a Literature Review
Somil GuptaBhanu Pratap Singh Rawat
2020-06-01
When Bert Forgets How To POS: Amnesic Probing of Linguistic Properties and MLM Predictions
Yanai ElazarShauli RavfogelAlon JacoviYoav Goldberg
2020-06-01
An Effective Contextual Language Modeling Framework for Speech Summarization with Augmented Features
Shi-Yan WengTien-Hong LoBerlin Chen
2020-06-01
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
| Zhewei YaoAmir GholamiSheng ShenKurt KeutzerMichael W. Mahoney
2020-06-01
BPGC at SemEval-2020 Task 11: Propaganda Detection in News Articles with Multi-Granularity Knowledge Sharing and Linguistic Features based Ensemble Learning
Rajaswa PatilSomesh SinghSwati Agarwal
2020-05-31
CNRL at SemEval-2020 Task 5: Modelling Causal Reasoning in Language with Multi-Head Self-Attention Weights based Counterfactual Detection
Rajaswa PatilVeeky Baths
2020-05-31
Neural Entity Linking: A Survey of Models based on Deep Learning
| Ozge SevgiliArtem ShelmanovMikhail ArkhipovAlexander PanchenkoChris Biemann
2020-05-31
"Judge me by my size (noun), do you?'' YodaLib: A Demographic-Aware Humor Generation Framework
Aparna GarimellaCarmen BaneaNabil HossainRada Mihalcea
2020-05-31
LRG at SemEval-2020 Task 7: Assessing the Ability of BERT and Derivative Models to Perform Short-Edits based Humor Grading
Siddhant MahurkarRajaswa Patil
2020-05-31
Detecting Problem Statements in Peer Assessments
Yunkai XiaoGabriel ZingleQinjin JiaHarsh R. ShahYi ZhangTianyi LiMohsin KarovaliyaWeixiang ZhaoYang SongJie JiAshwin BalasubramaniamHarshit PatelPriyankha BhalasubbramanianVikram PatelEdward F. Gehringer
2020-05-30
First Neural Conjecturing Datasets and Experiments
Josef UrbanJan Jakubův
2020-05-29
Using Large Pretrained Language Models for Answering User Queries from Product Specifications
Kalyani RoySmit ShahNithish PaiJaidam RamtejPrajit Prashant NadkarnJyotirmoy BanerjeePawan GoyalSurender Kumar
2020-05-29
A Comparative Study of Lexical Substitution Approaches based on Neural Language Models
Nikolay ArefyevBoris SheludkoAlexander PodolskiyAlexander Panchenko
2020-05-29
SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions
Mao YeChengyue GongQiang Liu
2020-05-29
Stance Prediction for Contemporary Issues: Data and Experiments
| Marjan HosseiniaEduard DragutArjun Mukherjee
2020-05-29
HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
| Hanrui WangZhanghao WuZhijian LiuHan CaiLigeng ZhuChuang GanSong Han
2020-05-28
Variational Neural Machine Translation with Normalizing Flows
Hendra SetiawanMatthias SperberUdhay NallasamyMatthias Paulik
2020-05-28
Empirical Evaluation of Pretraining Strategies for Supervised Entity Linking
Thibault FévryNicholas FitzGeraldLivio Baldini SoaresTom Kwiatkowski
2020-05-28
On Incorporating Structural Information to improve Dialogue Response Generation
| Nikita MoghePriyesh VijayanBalaraman RavindranMitesh M. Khapra
2020-05-28
Language Models are Few-Shot Learners
| Tom B. BrownBenjamin MannNick RyderMelanie SubbiahJared KaplanPrafulla DhariwalArvind NeelakantanPranav ShyamGirish SastryAmanda AskellSandhini AgarwalAriel Herbert-VossGretchen KruegerTom HenighanRewon ChildAditya RameshDaniel M. ZieglerJeffrey WuClemens WinterChristopher HesseMark ChenEric SiglerMateusz LitwinScott GrayBenjamin ChessJack ClarkChristopher BernerSam McCandlishAlec RadfordIlya SutskeverDario Amodei
2020-05-28
General-Purpose User Embeddings based on Mobile App Usage
| Junqi ZhangBing BaiYe LinJian LiangKun BaiFei Wang
2020-05-27
Permutation Matters: Anisotropic Convolutional Layer for Learning on Point Clouds
| Zhongpai GaoGuangtao ZhaiJunchi YanXiaokang Yang
2020-05-27
Language Representation Models for Fine-Grained Sentiment Classification
Brian CheangBailey WeiDavid KoganHowey QiuMasud Ahmed
2020-05-27
Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Adhiguna KuncoroLingpeng KongDaniel FriedDani YogatamaLaura RimellChris DyerPhil Blunsom
2020-05-27
CausaLM: Causal Model Explanation Through Counterfactual Language Models
Amir FederNadav OvedUri ShalitRoi Reichart
2020-05-27
Transition-based Semantic Dependency Parsing with Pointer Networks
Daniel Fernández-GonzálezCarlos Gómez-Rodríguez
2020-05-27
Network Fusion for Content Creation with Conditional INNs
Robin RombachPatrick EsserBjörn Ommer
2020-05-27
Insertion-Based Modeling for End-to-End Automatic Speech Recognition
Yuya FujitaShinji WatanabeMotoi OmachiXuankai Chan
2020-05-27
End-to-End Object Detection with Transformers
| Nicolas CarionFrancisco MassaGabriel SynnaeveNicolas UsunierAlexander KirillovSergey Zagoruyko
2020-05-26
GECToR -- Grammatical Error Correction: Tag, Not Rewrite
| Kostiantyn OmelianchukVitaliy AtrasevychArtem ChernodubOleksandr Skurzhanskyi
2020-05-26
Guiding Symbolic Natural Language Grammar Induction via Transformer-Based Sequence Probabilities
Ben GoertzelAndres Suarez MadrigalGino Yu
2020-05-26
Pay Attention to What You Read: Non-recurrent Handwritten Text-Line Recognition
Lei KangPau RibaMarçal RusiñolAlicia FornésMauricio Villegas
2020-05-26
A Data-driven Approach for Noise Reduction in Distantly Supervised Biomedical Relation Extraction
Saadullah AminKatherine Ann DunfieldAnna VechkaevaGünter Neumann
2020-05-26
What Are People Asking About COVID-19? A Question Classification Dataset
| Jerry WeiChengyu HuangSoroush VosoughiJason Wei
2020-05-26
ParsBERT: Transformer-based Model for Persian Language Understanding
| Mehrdad FarahaniMohammad GharachorlooMarzieh FarahaniMohammad Manthouri
2020-05-26
BEEP! Korean Corpus of Online News Comments for Toxic Speech Detection
| Jihyung MoonWon Ik ChoJunbum Lee
2020-05-26
Comparing BERT against traditional machine learning text classification
Santiago González-CarvajalEduardo C. Garrido-Merchán
2020-05-26
BERT-XML: Large Scale Automated ICD Coding Using BERT Pretraining
Zachariah ZhangJingshu LiuNarges Razavian
2020-05-26
Deep Learning Models for Automatic Summarization
Pirmin Lemberger
2020-05-25
The Unreasonable Volatility of Neural Machine Translation Models
| Marzieh FadaeeChristof Monz
2020-05-25
An Audio-enriched BERT-based Framework for Spoken Multiple-choice Question Answering
Chia-Chih KuoShang-Bao LuoKuan-Yu Chen
2020-05-25
Køpsala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding
| Daniel HershcovichMiryam de LhoneuxArtur KulmizevElham PejhanJoakim Nivre
2020-05-25
Pointwise Paraphrase Appraisal is Potentially Problematic
Hannah ChenYangfeng JiDavid Evans
2020-05-25
Stronger Baselines for Grammatical Error Correction Using Pretrained Encoder-Decoder Model
Satoru KatsumataMamoru Komachi
2020-05-24
Adversarial NLI for Factual Correctness in Text Summarisation Models
Mario BarrantesBenedikt HerudekRichard Wang
2020-05-24
Jointly Encoding Word Confusion Network and Dialogue Context with BERT for Spoken Language Understanding
Chen LiuSu ZhuZijian ZhaoRuisheng CaoLu ChenKai Yu
2020-05-24
Devising Malware Characterstics using Transformers
Simra ShahidTanmay SinghYash SharmaKapil Sharma
2020-05-23
Character-level Transformer-based Neural Machine Translation
Nikolay BanarWalter DaelemansMike Kestemont
2020-05-22
A Generative Approach to Titling and Clustering Wikipedia Sections
Anjalie FieldSascha RotheSimon BaumgartnerCong YuAbe Ittycheriah
2020-05-22
Low-Latency Sequence-to-Sequence Speech Recognition and Translation by Partial Hypothesis Selection
Danni LiuGerasimos SpanakisJan Niehues
2020-05-22
Transformer-based Context-aware Sarcasm Detection in Conversation Threads from Social Media
Xiangjue DongChangmao LiJinho D. Choi
2020-05-22
Comparative Study of Machine Learning Models and BERT on SQuAD
Devshree PatelParam RavalRatnam ParikhYesha Shastri
2020-05-22
L2R2: Leveraging Ranking for Abductive Reasoning
| Yunchang ZhuLiang PangYanyan LanXueqi Cheng
2020-05-22
Living Machines: A study of atypical animacy
Mariona Coll ArdanuyFederico NanniKaspar BeelenKasra HosseiniRuth AhnertJon LawrenceKatherine McDonoughGiorgia TolfoDaniel CS WilsonBarbara McGillivray
2020-05-22
Robust Layout-aware IE for Visually Rich Documents with Pre-trained Language Models
Mengxi WeiYifan HeQiong Zhang
2020-05-22
Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction
Laila RasmyYang XiangZiqian XieCui TaoDegui Zhi
2020-05-22
Simplified Self-Attention for Transformer-based End-to-End Speech Recognition
Haoneng LuoShiliang ZhangMing LeiLei Xie
2020-05-21
Text-to-Text Pre-Training for Data-to-Text Tasks
| Mihir Kale
2020-05-21
Leveraging Text Data Using Hybrid Transformer-LSTM Based End-to-End ASR in Transfer Learning
Zhiping ZengVan Tung PhamHaihua XuYerbolat KhassanovEng Siong ChngChongjia NiBin Ma
2020-05-21
Applying the Transformer to Character-level Transduction
Shijie WuRyan CotterellMans Hulden
2020-05-20
Creative Artificial Intelligence -- Algorithms vs. humans in an incentivized writing competition
Nils KöbisLuca Mossink
2020-05-20
Relative Positional Encoding for Speech Recognition and Direct Translation
Ngoc-Quan PhamThanh-Le HaTuan-Nam NguyenThai-Son NguyenElizabeth SaleskySebastian StuekerJan NiehuesAlexander Waibel
2020-05-20
A Further Study of Unsupervised Pre-training for Transformer Based Speech Recognition
Dongwei JiangWubo LiRuixiong ZhangMiao CaoNe LuoYang HanWei ZouXiangang Li
2020-05-20
BERTweet: A pre-trained language model for English Tweets
| Dat Quoc NguyenThanh VuAnh Tuan Nguyen
2020-05-20
FashionBERT: Text and Image Matching with Adaptive Loss for Cross-modal Retrieval
Dehong GaoLinbo JinBen ChenMinghui QiuPeng LiYi WeiYi HuHao Wang
2020-05-20
Comparing Transformers and RNNs on predicting human sentence processing data
Danny MerkxStefan L. Frank
2020-05-19
Should we hard-code the recurrence concept or learn it instead ? Exploring the Transformer architecture for Audio-Visual Speech Recognition
| George SterpuChristian SaamNaomi Harte
2020-05-19
Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestalt
Hangyu LinYanwei FuYu-Gang JiangXiangyang Xue
2020-05-19
Exploring Transformers for Large-Scale Speech Recognition
Liang LuChangliang LiuJinyu LiYifan Gong
2020-05-19
Cross-lingual Transfer Learning for Dialogue Act Recognition
Jiří MartínekChristophe CerisaraPavel KrálLadislav Lenc
2020-05-19
Table Search Using a Deep Contextualized Language Model
| Zhiyu ChenMohamed TrabelsiJeff HeflinYinan XuBrian D. Davison
2020-05-19
A Transformer-based Embedding Model for Personalized Product Search
Keping BiQingyao AiW. Bruce Croft
2020-05-18
Efficient Wait-k Models for Simultaneous Machine Translation
Maha ElbayadLaurent BesacierJakob Verbeek
2020-05-18
Audio ALBERT: A Lite BERT for Self-supervised Learning of Audio Representation
Po-Han ChiPei-Hung ChungTsung-Han WuChun-Cheng HsiehShang-Wen LiHung-yi Lee
2020-05-18
Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction
Cunjun YuXiao MaJiawei RenHaiyu ZhaoShuai Yi
2020-05-18
Many-to-Many Voice Transformer Network
Hirokazu KameokaWen-Chin HuangKou TanakaTakuhiro KanekoNobukatsu HojoTomoki Toda
2020-05-18
GPT-too: A language-model-first approach for AMR-to-text generation
| Manuel MagerRamon Fernandez AstudilloTahira NaseemMd Arafat SultanYoung-Suk LeeRadu FlorianSalim Roukos
2020-05-18
Weak-Attention Suppression For Transformer Based Speech Recognition
Yangyang ShiYongqiang WangChunyang WuChristian FuegenFrank ZhangDuc LeChing-Feng YehMichael L. Seltzer
2020-05-18
Mask CTC: Non-Autoregressive End-to-End ASR with CTC and Mask Predict
Yosuke HiguchiShinji WatanabeNanxin ChenTetsuji OgawaTetsunori Kobayashi
2020-05-18
Are All Languages Created Equal in Multilingual BERT?
Shijie WuMark Dredze
2020-05-18
A Better Use of Audio-Visual Cues: Dense Video Captioning with Bi-modal Transformer
| Vladimir IashinEsa Rahtu
2020-05-17
Building a Hebrew Semantic Role Labeling Lexical Resource from Parallel Movie Subtitles
Ben EyalMichael Elhadad
2020-05-17
Context-Based Quotation Recommendation
Ansel MacLaughlinTao ChenBurcu Karagol AyanDan Roth
2020-05-17
Support-BERT: Predicting Quality of Question-Answer Pairs in MSDN using Deep Bidirectional Transformer
Bhaskar SenNikhil GopalXinwei Xue
2020-05-17
Cross-Lingual Low-Resource Set-to-Description Retrieval for Global E-Commerce
| Juntao LiChang LiuJian WangLidong BingHongsong LiXiaozhong LiuDongyan ZhaoRui Yan
2020-05-17
Adversarial Training for Commonsense Inference
Lis PereiraXiaodong LiuFei ChengMasayuki AsaharaIchiro Kobayashi
2020-05-17
TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data
| Pengcheng YinGraham NeubigWen-tau YihSebastian Riedel
2020-05-17
Conformer: Convolution-augmented Transformer for Speech Recognition
| Anmol GulatiJames QinChung-Cheng ChiuNiki ParmarYu ZhangJiahui YuWei HanShibo WangZhengdong ZhangYonghui WuRuoming Pang
2020-05-16
Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension
Hongyu GongYelong ShenDian YuJianshu ChenDong Yu
2020-05-16
Streaming Transformer-based Acoustic Models Using Self-attention with Augmented Memory
Chunyang WuYongqiang WangYangyang ShiChing-Feng YehFrank Zhang
2020-05-16
IntelliCode Compose: Code Generation Using Transformer
Alexey SvyatkovskiyShao Kun DengShengyu FuNeel Sundaresan
2020-05-16
Spike-Triggered Non-Autoregressive Transformer for End-to-End Speech Recognition
Zhengkun TianJiangyan YiJianhua TaoYe BaiShuai ZhangZhengqi Wen
2020-05-16
CERT: Contrastive Self-supervised Learning for Language Understanding
Hongchao FangSicheng WangMeng ZhouJiayuan DingPengtao Xie
2020-05-16
Leveraging Affective Bidirectional Transformers for Offensive Language Detection
AbdelRahim ElmadanyChiyu ZhangMuhammad Abdul-MageedAzadeh Hashemi
2020-05-16
COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter
| Martin MüllerMarcel SalathéPer E Kummervold
2020-05-15
Neural Entity Linking on Technical Service Tickets
Nadja KurzFelix HamannAdrian Ulges
2020-05-15
Finding Experts in Transformer Models
Xavier SuauLuca ZappellaNicholas Apostoloff
2020-05-15
JDI-T: Jointly trained Duration Informed Transformer for Text-To-Speech without Explicit Alignment
Dan LimWon JangGyeonghwan OHyeyeong ParkBongwan KimJesam Yoon
2020-05-15
Spelling Error Correction with Soft-Masked BERT
| Shaohua ZhangHaoran HuangJicong LiuHang Li
2020-05-15
Challenges in Emotion Style Transfer: An Exploration with a Lexical Substitution Pipeline
David HelbigEnrica TroianoRoman Klinger
2020-05-15
[email protected] at SemEval-2020 Task 12: Identifying Multilingual Offensive Tweets Using Weighted Ensemble and Fine-Tuned BERT
Saja Khaled TawalbehMahmoud HammadMohammad AL-Smadi
2020-05-15
NIT-Agartala-NLP-Team at SemEval-2020 Task 8: Building Multimodal Classifiers to tackle Internet Humor
Steve Durairaj SwamyShubham LaddhaBasil AbdussalamDebayan DattaAnupam Jamatia
2020-05-14
A pre-training technique to localize medical BERT and enhance BioBERT
| Shoya WadaToshihiro TakedaShiro ManabeShozo KonishiJun KamoharaYasushi Matsumura
2020-05-14
The Unstoppable Rise of Computational Linguistics in Deep Learning
James Henderson
2020-05-13
Large Scale Multi-Actor Generative Dialog Modeling
Alex BoydRaul PuriMohammad ShoeybiMostofa PatwaryBryan Catanzaro
2020-05-13
A Mixture of $h-1$ Heads is Better than $h$ Heads
Hao PengRoy SchwartzDianqi LiNoah A. Smith
2020-05-13
Parallel Corpus Filtering via Pre-trained Language Models
Boliang ZhangAjay NageshKevin Knight
2020-05-13
Entity-Enriched Neural Models for Clinical Question Answering
| Bhanu Pratap Singh RawatWei-Hung WengPreethi RaghavanPeter Szolovits
2020-05-13
On the Robustness of Language Encoders against Grammatical Errors
Fan YinQuanyu LongTao MengKai-Wei Chang
2020-05-12
Discriminative Multi-modality Speech Recognition
| Bo XuCheng LuYandong GuoJacob Wang
2020-05-12
Simultaneous paraphrasing and translation by fine-tuning Transformer models
Rakesh Chada
2020-05-12
SOLOIST: Few-shot Task-Oriented Dialog with A Single Pre-trained Auto-regressive Model
Baolin PengChunyuan LiJinchao LiShahin ShayandehLars LidenJianfeng Gao
2020-05-11
Hierarchical Attention Transformer Architecture For Syntactic Spell Correction
Abhishek NiranjanM Ali Basha ShaikKushal Verma
2020-05-11
Listen Attentively, and Spell Once: Whole Sentence Generation via a Non-Autoregressive Architecture for Low-Latency Speech Recognition
Ye BaiJiangyan YiJianhua TaoZhengkun TianZhengqi WenShuai Zhang
2020-05-11
MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioning
| Jie LeiLiwei WangYelong ShenDong YuTamara L. BergMohit Bansal
2020-05-11
On the Generation of Medical Dialogues for COVID-19
| Wenmian YangGuangtao ZengBowen TanZeqian JuSubrato ChakravortyXuehai HeShu ChenXingyi YangQingyang WuZhou YuEric XingPengtao Xie
2020-05-11
Detecting Adverse Drug Reactions from Twitter through Domain-Specific Preprocessing and BERT Ensembling
Amy BredenLee Moore
2020-05-11
Epipolar Transformers
| Yihui HeRui YanKaterina FragkiadakiShoou-I Yu
2020-05-10
How Context Affects Language Models' Factual Predictions
Fabio PetroniPatrick LewisAleksandra PiktusTim RocktäschelYuxiang WuAlexander H. MillerSebastian Riedel
2020-05-10
Transformer Based Language Models for Similar Text Retrieval and Ranking
Javed Qadrud-DinAshraf Bah RabiouRyan WalkerRavi SoniMartin GajekGabriel PackAkhil Rangaraj
2020-05-10
Finding Universal Grammatical Relations in Multilingual BERT
Ethan A. ChiJohn HewittChristopher D. Manning
2020-05-09
It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations
| Samson TanShafiq JotyMin-Yen KanRichard Socher
2020-05-09
SocialTrans: A Deep Sequential Model with Social Information for Web-Scale Recommendation Systems
Qiaoan ChenHao GuLingling YiYishi LinPeng HeChuan ChenYangqiu Song
2020-05-09
LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluation
Gustavo AguilarSudipta KarThamar Solorio
2020-05-09
schuBERT: Optimizing Elements of BERT
Ashish KhetanZohar Karnin
2020-05-09
Character Matters: Video Story Understanding with Character-Aware Relations
Shijie GengJi ZhangZuohui FuPeng GaoHang ZhangGerard de Melo
2020-05-09
SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics
| Da YinTao MengKai-Wei Chang
2020-05-08
Distilling Knowledge from Pre-trained Language Models via Text Smoothing
Xing WuYibing LiuXiangyang ZhouDianhai Yu
2020-05-08
GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference
Ali Hadi ZadehAndreas Moshovos
2020-05-08
Temporal Common Sense Acquisition with Minimal Supervision
Ben ZhouQiang NingDaniel KhashabiDan Roth
2020-05-08
Comparative Analysis of Text Classification Approaches in Electronic Health Records
Aurelie MascioZeljko KraljevicDaniel BeanRichard DobsonRobert StewartRebecca BendayanAngus Roberts
2020-05-08
Beyond Accuracy: Behavioral Testing of NLP models with CheckList
| Marco Tulio RibeiroTongshuang WuCarlos GuestrinSameer Singh
2020-05-08
Mapping Natural Language Instructions to Mobile UI Action Sequences
| Yang LiJiacong HeXin ZhouYuan ZhangJason Baldridge
2020-05-07
LIIR at SemEval-2020 Task 12: A Cross-Lingual Augmentation Approach for Multilingual Offensive Language Identification
Erfan GhaderyMarie-Francine Moens
2020-05-07
A Systematic Assessment of Syntactic Generalization in Neural Language Models
Jennifer HuJon GauthierPeng QianEthan WilcoxRoger P. Levy
2020-05-07
Trajectory Prediction for Autonomous Driving based on Multi-Head Attention with Joint Agent-Map Representation
Kaouther MessaoudNachiket DeoMohan M. TrivediFawzi Nashashibi
2020-05-06
Harvesting and Refining Question-Answer Pairs for Unsupervised QA
| Zhongli LiWenhui WangLi DongFuru WeiKe Xu
2020-05-06
An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining
Yifan PengQingyu ChenZhiyong Lu
2020-05-06
Autoencoding Pixies: Amortised Variational Inference with Graph Convolutions for Functional Distributional Semantics
Guy Emerson
2020-05-06
Categorical Vector Space Semantics for Lambek Calculus with a Relevant Modality
Lachlan McPheatMehrnoosh SadrzadehHadi WazniGijs Wijnholds
2020-05-06
The Cascade Transformer: an Application for Efficient Answer Sentence Selection
| Luca SoldainiAlessandro Moschitti
2020-05-05
MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models
| Mandy GuoYinfei YangDaniel CerQinlan ShenNoah Constant
2020-05-05
Contextualizing Hate Speech Classifiers with Post-hoc Explanation
Brendan KennedyXisen JinAida Mostafazadeh DavaniMorteza DehghaniXiang Ren
2020-05-05
Establishing Baselines for Text Classification in Low-Resource Languages
| Jan Christian Blaise CruzCharibeth Cheng
2020-05-05
Dynamically Adjusting Transformer Batch Size by Monitoring Gradient Direction Change
Hongfei XuJosef van GenabithDeyi XiongQiuhui Liu
2020-05-05
ExpBERT: Representation Engineering with Natural Language Explanations
| Shikhar MurtyPang Wei KohPercy Liang
2020-05-05
OpinionDigest: A Simple Framework for Opinion Summarization
Yoshihiko SuharaXiaolan WangStefanos AngelidisWang-Chiew Tan
2020-05-05
ImpactCite: An XLNet-based method for Citation Impact Analysis
Dominique MercierSyed Tahseen Raza RizviVikas RajashekarAndreas DengelSheraz Ahmed
2020-05-05
Distributional Discrepancy: A Metric for Unconditional Text Generation
| Ping CaiXingyuan ChenPeng JinHongjun WangTianrui Li
2020-05-04
Robust Encodings: A Framework for Combating Adversarial Typos
Erik JonesRobin JiaAditi RaghunathanPercy Liang
2020-05-04
Successfully Applying the Stabilized Lottery Ticket Hypothesis to the Transformer Architecture
Christopher BrixParnia BaharHermann Ney
2020-05-04
Spying on your neighbors: Fine-grained probing of contextual embeddings for information about surrounding words
Josef KlafkaAllyson Ettinger
2020-05-04
Unsupervised Alignment-based Iterative Evidence Retrieval for Multi-hop Question Answering
| Vikas YadavSteven BethardMihai Surdeanu
2020-05-04
Code and Named Entity Recognition in StackOverflow
| Jeniya TabassumMounica MaddelaWei XuAlan Ritter
2020-05-04
An Accurate Model for Predicting the (Graded) Effect of Context in Word Similarity Based on Bert
Wei BaoHongshu CheJiandong Zhang
2020-05-03
Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints
Zhenyi WangXiaoyang WangBang AnDong YuChangyou Chen
2020-05-03
Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction
| Masahiro KanekoMasato MitaShun KiyonoJun SuzukiKentaro Inui
2020-05-03
Dynamic Programming Encoding for Subword Segmentation in Neural Machine Translation
| Xuanli HeGholamreza HaffariMohammad Norouzi
2020-05-03
Transformer-based End-to-End Question Generation
| Luis Enrico LopezDiane Kathryn CruzJan Christian Blaise CruzCharibeth Cheng
2020-05-03
Quantifying Attention Flow in Transformers
| Samira AbnarWillem Zuidema
2020-05-02
Measuring and Reducing Non-Multifact Reasoning in Multi-hop Question Answering
Harsh TrivediNiranjan BalasubramanianTushar KhotAshish Sabharwal
2020-05-02
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA
Nora KassnerHinrich Schütze
2020-05-02
Synthesizer: Rethinking Self-Attention in Transformer Models
Yi TayDara BahriDonald MetzlerDa-Cheng JuanZhe ZhaoChe Zheng
2020-05-02
Hard-Coded Gaussian Attention for Neural Machine Translation
Weiqiu YouSimeng SunMohit Iyyer
2020-05-02
DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering
| Qingqing CaoHarsh TrivediAruna BalasubramanianNiranjan Balasubramanian
2020-05-02
Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models
Bill Yuchen LinSeyeon LeeRahul KhannaXiang Ren
2020-05-02
Generating Derivational Morphology with BERT
Valentin HofmannJanet B. PierrehumbertHinrich Schütze
2020-05-02
Contrastive Self-Supervised Learning for Commonsense Reasoning
| Tassilo KleinMoin Nabi
2020-05-02
IsoBN: Fine-Tuning BERT with Isotropic Batch Normalization
| Wenxuan ZhouBill Yuchen LinXiang Ren
2020-05-02
A Simple Language Model for Task-Oriented Dialogue
| Ehsan Hosseini-AslBryan McCannChien-Sheng WuSemih YavuzRichard Socher
2020-05-02
HipoRank: Incorporating Hierarchical and Positional Information into Graph-based Unsupervised Long Document Extractive Summarization
Yue DongAndrei RomascanuJackie C. K. Cheung
2020-05-01
Identifying Necessary Elements for BERT's Multilinguality
| Philipp DufterHinrich Schütze
2020-05-01
Hitachi at SemEval-2020 Task 12: Offensive Language Identification with Noisy Labels using Statistical Sampling and Post-Processing
Manikandan RavikiranAmin Ekant MuljibhaiToshinori MiyoshiHiroaki OzakiYuta KoreedaSakata Masayuki
2020-05-01
The AVA-Kinetics Localized Human Actions Video Dataset
Ang LiMeghana ThotakuriDavid A. RossJoão CarreiraAlexander VostrikovAndrew Zisserman
2020-05-01
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
Linjie LiYen-Chun ChenYu ChengZhe GanLicheng YuJingjing Liu
2020-05-01
Cross-Linguistic Syntactic Evaluation of Word Prediction Models
| Aaron MuellerGarrett NicolaiPanayiota Petrou-ZeniouNatalia TalminaTal Linzen
2020-05-01
A Transformer-based Approach for Source Code Summarization
| Wasi Uddin AhmadSaikat ChakrabortyBaishakhi RayKai-Wei Chang
2020-05-01
Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?
Yada PruksachatkunJason PhangHaokun LiuPhu Mon HtutXiaoyi ZhangRichard Yuanzhe PangClara VaniaKatharina KannSamuel R. Bowman
2020-05-01
A Controllable Model of Grounded Response Generation
Zeqiu WuMichel GalleyChris BrockettYizhe ZhangXiang GaoChris QuirkRik Koncel-KedziorskiJianfeng GaoHannaneh HajishirziMari OstendorfBill Dolan
2020-05-01
Multi-scale Transformer Language Models
Sandeep SubramanianRonan CollobertMarc'Aurelio RanzatoY-Lan Boureau
2020-05-01
Clinical Reading Comprehension: A Thorough Analysis of the emrQA Dataset
Xiang YueBernal Jimenez GutierrezHuan Sun
2020-05-01
When BERT Plays the Lottery, All Tickets Are Winning
Sai PrasannaAnna RogersAnna Rumshisky
2020-05-01
POINTER: Constrained Text Generation via Insertion-based Generative Pre-training
| Yizhe ZhangGuoyin WangChunyuan LiZhe GanChris BrockettBill Dolan
2020-05-01
Probing Text Models for Common Ground with Visual Representations
Gabriel IlharcoRowan ZellersAli FarhadiHannaneh Hajishirzi
2020-05-01
Event Clustering within News Articles
Faik Kerem {\"O}rsS{\"u}veyda YeniterziReyyan Yeniterzi
2020-05-01
Detecting Direct Speech in Multilingual Collection of 19th-century Novels
Joanna ByszukMicha{\l} Wo{\'z}niakMike KestemontAlbert Le{\'s}niakWojciech {\L}ukasikArtjoms {\v{S}}e{\c{l}}aMaciej Eder
2020-05-01
Transfer learning applied to text classification in Spanish radiological reports
Pilar L{\'o}pez {\'U}bedaManuel Carlos D{\'\i}az-GalianoL. Alfonso Urena LopezMaite MartinTeodoro Mart{\'\i}n-NoguerolAntonio Luna
2020-05-01
ASU\_OPTO at OSACT4 - Offensive Language Detection for Arabic text
Amr KelegSamhaa R. El-BeltagyMahmoud Khalil
2020-05-01
Scaling Language Data Import/Export with a Data Transformer Interface
Nicholas BuckeridgeBen Foley
2020-05-01
``A Passage to India'': Pre-trained Word Embeddings for Indian Languages
Saurav KumarSaunack KumarDiptesh KanojiaPushpak Bhattacharyya
2020-05-01
Aggression Identification in Social Media: a Transfer Learning Based Approach
RamiFaneva risoaJosiane Mothe
2020-05-01
IRIT at TRAC 2020
RamiFaneva risoaJosiane Mothe
2020-05-01
Multilingual Joint Fine-tuning of Transformer models for identifying Trolling, Aggression and Cyberbullying at TRAC 2020
| Sudhanshu MishraShivangi PrasadShubhanshu Mishra
2020-05-01
On the Influence of Coreference Resolution on Word Embeddings in Lexical-semantic Evaluation Tasks
Alex HenleinerAlex Mehlerer
2020-05-01
Chinese Discourse Parsing: Model and Evaluation
Lin Chuan-AnShyh-Shiun HungHen-Hsen HuangHsin-Hsi Chen
2020-05-01
DecOp: A Multilingual and Multi-domain Corpus For Detecting Deception In Typed Text
Pasquale CapuozzoIvano LauriolaCarlo StrapparavaFabio AiolliGiuseppe Sartori
2020-05-01
Evaluation Metrics for Headline Generation Using Deep Pre-Trained Embeddings
Abdul MoeedYang AnGerhard HagererGeorg Groh
2020-05-01
Paraphrase Generation and Evaluation on Colloquial-Style Sentences
Eetu Sj{\"o}blomMathias CreutzYves Scherrer
2020-05-01
Building a Task-oriented Dialog System for Languages with no Training Data: the Case for Basque
Maddalen L{\'o}pez de LacalleXabier SaralegiI{\~n}aki San Vicente
2020-05-01
Linguistically Informed Hindi-English Neural Machine Translation
Vikrant GoyalPruthwik MishraDipti Misra Sharma
2020-05-01
Corpora for Document-Level Neural Machine Translation
Siyou LiuXiaojun Zhang
2020-05-01
Multilingual Corpus Creation for Multilingual Semantic Similarity Task
Mahtab AhmedChahna DixitRobert E. MercerAtif KhanMuhammad Rifayat SameeFelipe Urra
2020-05-01
Exploring Transformer Text Generation for Medical Dataset Augmentation
Ali Amin-NejadJulia IveSumithra Velupillai
2020-05-01
Much Ado About Nothing -- Identification of Zero Copulas in Hungarian Using an NMT Model
Andrea D{\"o}m{\"o}t{\"o}rZijian Gy{\H{o}}z{\H{o}} YangAttila Nov{\'a}k
2020-05-01
ParlVote: A Corpus for Sentiment Analysis of Political Debates
Gavin AbercrombieRiza Batista-Navarro
2020-05-01
Cross-lingual and Cross-domain Evaluation of Machine Reading Comprehension with Squad and CALOR-Quest Corpora
Delphine CharletGeraldine DamnatiFrederic Bechetgabriel marzinottoJohannes Heinecke
2020-05-01
Contextualized Embeddings based Transformer Encoder for Sentence Similarity Modeling in Answer Selection Task
| Md Tahmid Rahman LaskarJimmy Xiangji HuangEnamul Hoque
2020-05-01
Evaluation of Dataset Selection for Pre-Training and Fine-Tuning Transformer Language Models for Clinical Question Answering
Sarvesh SoniKirk Roberts
2020-05-01
Sign Language Recognition with Transformer Networks
Mathieu De CosterMieke Van HerrewegheJoni Dambre
2020-05-01
Minority Positive Sampling for Switching Points - an Anecdote for the Code-Mixing Language Modeling
Arindam ChatterjereVineeth GupthaParul ChopraAmitava Das
2020-05-01
Seq2SeqPy: A Lightweight and Customizable Toolkit for Neural Sequence-to-Sequence Modeling
Raheel QaderFran{\c{c}}ois PortetCyril Labbe
2020-05-01
Text Categorization for Conflict Event Annotation
Fredrik OlssonMagnus SahlgrenFehmi ben AbdesslemAriel EkgrenKristine Eck
2020-05-01
TF-IDF Character N-grams versus Word Embedding-based Models for Fine-grained Event Classification: A Preliminary Study
Jakub PiskorskiGuillaume Jacquet
2020-05-01
TermEval 2020: TALN-LS2N System for Automatic Term Extraction
Amir HazemBouhM{\'e}rieme iFlorian BoudinBeatrice Daille
2020-05-01
FrameNet Annotations Alignment using Attention-based Machine Translation
Gabriel Marzinotto
2020-05-01
Implementation of Supervised Training Approaches for Monolingual Word Sense Alignment: ACDH-CH System Description for the MWSA Shared Task at GlobaLex 2020
Lenka BajceticSeung-bin Yim
2020-05-01
Bagging BERT Models for Robust Aggression Identification
Julian RischRalf Krestel
2020-05-01
Scmhl5 at TRAC-2 Shared Task on Aggression Identification: Bert Based Ensemble Learning Approach
Han LiuPete BurnapWafa AlorainyMatthew Williams
2020-05-01
Aggression Identification in English, Hindi and Bangla Text using BERT, RoBERTa and SVM
| Arup BaruahKaushik DasFerdous BarbhuiyaKuntal Dey
2020-05-01
Aggression and Misogyny Detection using BERT: A Multi-Task Approach
| Niloofar Safi SamghabadiParth PatwaSrinivas PYKLPrerana MukherjeeAmitava DasThamar Solorio
2020-05-01
From Web Crawl to Clean Register-Annotated Corpora
Veronika LaippalaSamuel R{\"o}nnqvistSaara Hellstr{\"o}mJuhani LuotolahtiLiina RepoAnna SalmelaValtteri SkantsiSampo Pyysalo
2020-05-01
Cross-lingual Zero Pronoun Resolution
Abdulrahman AlorainiMassimo Poesio
2020-05-01
Understanding User Utterances in a Dialog System for Caregiving
Yoshihiko AsaoJulien KloetzerJunta MizunoDai SaikiKazuma KadowakiKentaro Torisawa
2020-05-01
Joint Learning of Syntactic Features Helps Discourse Segmentation
Takshak DesaiParag Pravin DakleDan Moldovan
2020-05-01
Adapting BERT to Implicit Discourse Relation Classification with a Focus on Discourse Connectives
Yudai KishimotoYugo MurawakiSadao Kurohashi
2020-05-01
Automated Essay Scoring System for Nonnative Japanese Learners
Reo HiraoMio AraiHiroki ShimanakaSatoru KatsumataMamoru Komachi
2020-05-01
Development and Validation of a Corpus for Machine Humor Comprehension
Yuen-Hsien TsengWun-Syuan WuChia-Yueh ChangHsueh-Chih ChenWei-Lun Hsu
2020-05-01
Abusive language in Spanish children and young teenager's conversations: data preparation and short text classification with contextual word embeddings
Marta R. Costa-juss{\`a}Esther Gonz{\'a}lezAsuncion MorenoEudald Cumalat
2020-05-01
An Evaluation Dataset for Identifying Communicative Functions of Sentences in English Scholarly Papers
Kenichi IwatsukiFlorian BoudinAkiko Aizawa
2020-05-01
SiBert: Enhanced Chinese Pre-trained Language Model with Sentence Insertion
Jiahao ChenChenjie CaoXiuyan Jiang
2020-05-01
Adaptation of Deep Bidirectional Transformers for Afrikaans Language
Sello Ralethe
2020-05-01
Massive vs. Curated Embeddings for Low-Resourced Languages: the Case of Yor\`ub\'a and Twi
Jesujoba AlabiKwabena Amponsah-KaakyireDavid AdelaniCristina Espa{\~n}a-Bonet
2020-05-01
Introducing a Large-Scale Dataset for Vietnamese POS Tagging on Conversational Texts
Oanh TranTu PhamVu DangBang Nguyen
2020-05-01
DaNE: A Named Entity Resource for Danish
Rasmus HvingelbyAmalie Brogaard PauliMaria BarrettChristina RostedLasse Malm LidegaardAnders S{\o}gaard
2020-05-01
Is Language Modeling Enough? Evaluating Effective Embedding Combinations
Rudolf SchneiderTom OberhauserPaul GrundmannFelix Alex GerserAlex LoesererSteffen Staab
2020-05-01
Parsing as Tagging
Robert VacareanuGeorge Caique Gouveia BarbosaMarco A. Valenzuela-Esc{\'a}rcegaMihai Surdeanu
2020-05-01
AIA-BDE: A Corpus of FAQs in Portuguese and their Variations
Hugo Gon{\c{c}}alo OliveiraJo{\~a}o FerreiraJos{\'e} SantosPedro FialhoRicardo RodriguesLuisa CoheurAna Alves
2020-05-01
One Classifier for All Ambiguous Words: Overcoming Data Sparsity by Utilizing Sense Correlations Across Words
Prafulla Kumar ChoubeyRuihong Huang
2020-05-01
A Summarization Dataset of Slovak News Articles
| Marek SuppaJergus Adamec
2020-05-01
KLEJ: Comprehensive Benchmark for Polish Language Understanding
| Piotr RybakRobert MroczkowskiJanusz TraczIreneusz Gawlik
2020-05-01
Analyzing ELMo and DistilBERT on Socio-political News Classification
Berfu B{\"u}y{\"u}k{\"o}zAli H{\"u}rriyeto{\u{g}}luArzucan {\"O}zg{\"u}r
2020-05-01
SciREX: A Challenge Dataset for Document-Level Information Extraction
| Sarthak JainMadeleine van ZuylenHannaneh HajishirziIz Beltagy
2020-05-01
WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context
Anna BreitArtem RevenkoKiamehr RezaeeMohammad Taher PilehvarJose Camacho-Collados
2020-04-30
On the Evaluation of Contextual Embeddings for Zero-Shot Cross-Lingual Transfer Learning
Phillip KeungYichao LuJulian SalazarVikas Bhardwaj
2020-04-30
A Matter of Framing: The Impact of Linguistic Formalism on Probing Results
Ilia KuznetsovIryna Gurevych
2020-04-30
SegaBERT: Pre-training of Segment-aware BERT for Language Understanding
He BaiPeng ShiJimmy LinLuchen TanKun XiongWen GaoMing Li
2020-04-30
How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking
| Nicola De CaoMichael SchlichtkrullWilker AzizIvan Titov
2020-04-30
Investigating Transferability in Pretrained Language Models
Alex TamkinTrisha SinghDavide GiovanardiNoah Goodman
2020-04-30
PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking
Hannah RashkinAsli CelikyilmazYejin ChoiJianfeng Gao
2020-04-30
Addressing Zero-Resource Domains Using Document-Level Context in Neural Machine Translation
Dario StojanovskiAlexander Fraser
2020-04-30
Progressive Transformers for End-to-End Sign Language Production
| Ben SaundersNecati Cihan CamgozRichard Bowden
2020-04-30
Enriched Pre-trained Transformers for Joint Slot Filling and Intent Detection
Momchil HardalovIvan KoychevPreslav Nakov
2020-04-30
Accurate Word Alignment Induction from Neural Machine Translation
Yun ChenYang LiuGuanhua ChenXin JiangQun Liu
2020-04-30
Character-Level Translation with Self-attention
Yingqiang GaoNikola I. NikolovYuhuang HuRichard H. R. Hahnloser
2020-04-30
Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT
Zhiyong WuYun ChenBen KaoQun Liu
2020-04-30
Semantic Triple Encoder for Fast Open-Set Link Prediction
Bo WangTao ShenGuodong LongTianyi ZhouYi Chang
2020-04-30
Self-Supervised and Controlled Multi-Document Opinion Summarization
Hady ElsaharMaximin CoavouxMatthias GalléJos Rozen
2020-04-30
End-to-End Neural Word Alignment Outperforms GIZA++
Thomas ZenkelJoern WuebkerJohn DeNero
2020-04-30
Capsule-Transformer for Neural Machine Translation
Sufeng DuanJuncheng CaoHai Zhao
2020-04-30
Robust Question Answering Through Sub-part Alignment
Jifan ChenGreg Durrett
2020-04-30
Modular Representation Underlies Systematic Generalization in Neural Natural Language Inference Models
Atticus GeigerKyle RichardsonChristopher Potts
2020-04-30
Universal Dependencies according to BERT: both more specific and more general
| Tomasz LimisiewiczRudolf RosaDavid Mareček
2020-04-30
Look at the First Sentence: Position Bias in Question Answering
Miyoung KoJinhyuk LeeHyunjae KimGangwoo KimJaewoo Kang
2020-04-30
Exploring Contextualized Neural Language Models for Temporal Dependency Parsing
Hayley RossJonathan CaiBonan Min
2020-04-30
Interpretable Entity Representations through Large-Scale Typing
Yasumasa OnoeGreg Durrett
2020-04-30
Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging
Shigang LiTal Ben-NunGiorgi NadiradzeSalvatore Di GirolamoNikoli DrydenDan AlistarhTorsten Hoefler
2020-04-30
MAD-X: An Adapter-based Framework for Multi-task Cross-lingual Transfer
Jonas PfeifferIvan VulićIryna GurevychSebastian Ruder
2020-04-30
Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation
Asa Cooper SticklandXian LiMarjan Ghazvininejad
2020-04-30
End-to-End Slot Alignment and Recognition for Cross-Lingual NLU
Weijia XuBatool HaiderSaab Mansour
2020-04-29
Detecting Perceived Emotions in Hurricane Disasters
Shrey DesaiCornelia CarageaJunyi Jessy Li
2020-04-29
Towards Character-Level Transformer NMT by Finetuning Subword Systems
Jindřich LibovickýAlexander Fraser
2020-04-29
Training Curricula for Open Domain Answer Re-Ranking
| Sean MacAvaneyFranco Maria NardiniRaffaele PeregoNicola TonellottoNazli GoharianOphir Frieder
2020-04-29
Efficient Document Re-Ranking for Transformers by Precomputing Term Representations
| Sean MacAvaneyFranco Maria NardiniRaffaele PeregoNicola TonellottoNazli GoharianOphir Frieder
2020-04-29
GePpeTto Carves Italian into a Language Model
| Lorenzo De MatteiMichele CafagnaFelice Dell'OrlettaMalvina NissimMarco Guerini
2020-04-29
Image Captioning through Image Transformer
Sen HeWentong LiaoHamed R. TavakoliMichael YangBodo RosenhahnNicolas Pugeault
2020-04-29
Analysing Lexical Semantic Change with Contextualised Word Representations
Mario GiulianelliMarco Del TrediciRaquel Fernández
2020-04-29
Do Neural Language Models Show Preferences for Syntactic Formalisms?
Artur KulmizevVinit RavishankarMostafa AbdouJoakim Nivre
2020-04-29
Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning
Alexandre TamborrinoNicola PellicanoBaptiste PannierPascal VoitotLouise Naudin
2020-04-29
Image Morphing with Perceptual Constraints and STN Alignment
Noa FishRichard ZhangLilach PerryDaniel Cohen-OrEli ShechtmanConnelly Barnes
2020-04-29
Learning Better Universal Representations from Pre-trained Contextualized Language Models
Yian LiHai Zhao
2020-04-29
Bilingual Text Extraction as Reading Comprehension
Katsuki ChousaMasaaki NagataMasaaki Nishino
2020-04-29
What Happens To BERT Embeddings During Fine-tuning?
Amil MerchantElahe RahimtoroghiEllie PavlickIan Tenney
2020-04-29
Distantly-Supervised Neural Relation Extraction with Side Information using BERT
| Johny MoreiraChaina OliveiraDavid MacêdoCleber ZanchettinLuciano Barbosa
2020-04-29
Multiresolution and Multimodal Speech Recognition with Transformers
Georgios ParaskevopoulosSrinivas ParthasarathyAparna KhareShiva Sundaram
2020-04-29
Revisiting Pre-Trained Models for Chinese Natural Language Processing
| Yiming CuiWanxiang CheTing LiuBing QinShijin WangGuoping Hu
2020-04-29
A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERT
Masaaki NagataChousa KatsukiMasaaki Nishino
2020-04-29
Asking without Telling: Exploring Latent Ontologies in Contextual Representations
Julian MichaelJan A. BothaIan Tenney
2020-04-29
TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP
| John X. MorrisEli LiflandJin Yong YooJake GrigsbyDi JinYanjun Qi
2020-04-29
Scheduled DropHead: A Regularization Method for Transformer Models
Wangchunshu ZhouTao GeKe XuFuru WeiMing Zhou
2020-04-28
Extending Multilingual BERT to Low-Resource Languages
Zihan WangKarthikeyan KStephen MayhewDan Roth
2020-04-28
Joint Keyphrase Chunking and Salience Ranking with BERT
| Si SunChenyan XiongZhenghao LiuZhiyuan LiuJie Bao
2020-04-28
EARL: Speedup Transformer-based Rankers with Pre-computed Representation
Luyu GaoZhuyun DaiJamie Callan
2020-04-28
VD-BERT: A Unified Vision and Dialog Transformer with BERT
Yue WangShafiq JotyMichael R. LyuIrwin KingCaiming XiongSteven C. H. Hoi
2020-04-28
DomBERT: Domain-oriented Language Model for Aspect-based Sentiment Analysis
Hu XuBing LiuLei ShuPhilip S. Yu
2020-04-28
Kungfupanda at SemEval-2020 Task 12: BERT-Based Multi-Task Learning for Offensive Language Detection
| Wenliang DaiTiezheng YuZihan LiuPascale Fung
2020-04-28
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference
| Ji XinRaphael TangJaejun LeeYaoliang YuJimmy Lin
2020-04-27
ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Omar KhattabMatei Zaharia
2020-04-27
LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning
Kaitao SongHao SunXu TanTao QinJianfeng LuHongzhi LiuTie-Yan Liu
2020-04-27
ColBERT: Using BERT Sentence Embedding for Humor Detection
| Issa Annamoradnejad
2020-04-27
Augmenting Transformers with KNN-Based Composite Memory for Dialogue
Angela FanClaire GardentChloe BraudAntoine Bordes
2020-04-27
Lexically Constrained Neural Machine Translation with Levenshtein Transformer
| Raymond Hendy SusantoShamil ChollampattLiling Tan
2020-04-27
On the Importance of Word and Sentence Representation Learning in Implicit Discourse Relation Classification
| Xin LiuJiefu OuYangqiu SongXin Jiang
2020-04-27
Explicitly Modeling Adaptive Depths for Transformer
Yijin LiuFandong MengJie ZhouYufeng ChenJinan Xu
2020-04-27
Choppy: Cut Transformer For Ranked List Truncation
Dara BahriYi TayChe ZhengDonald MetzlerAndrew Tomkins
2020-04-26
Assessing Discourse Relations in Language Generation from Pre-trained Language Models
Wei-Jen KoJunyi Jessy Li
2020-04-26
Experiments with LVT and FRE for Transformer model
Ilshat GibadullinAidar Valeev
2020-04-26
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models
Mengjie ZhaoTao LinMartin JaggiHinrich Schütze
2020-04-26
Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Document Matching
Liu YangMingyang ZhangCheng LiMichael BenderskyMarc Najork
2020-04-26
Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias
Jesse VigSebastian GehrmannYonatan BelinkovSharon QianDaniel NevoYaron SingerStuart Shieber
2020-04-26
Research on Modeling Units of Transformer Transducer for Mandarin Speech Recognition
Li FuXiaoxiao LiLibo Zi
2020-04-26
Classification of Cuisines from Sequentially Structured Recipes
Tript Sharma