Weight Decay

Weight Decay, or $L_{2}$ Regularization, is a regularization technique applied to the weights of a neural network. We minimize a loss function compromising both the primary loss function and a penalty on the $L_{2}$ Norm of the weights:

$$L_{new}\left(w\right) = L_{original}\left(w\right) + \lambda{w^{T}w}$$

where $\lambda$ is a value determining the strength of the penalty (encouraging smaller weights).

Weight decay can be incorporated directly into the weight update rule, rather than just implicitly by defining it through to objective function. Often weight decay refers to the implementation where we specify it directly in the weight update rule (whereas L2 regularization is usually the implementation which is specified in the objective function).

Image Source: Deep Learning, Goodfellow et al

Latest Papers

PAPER DATE
Improve Generalization and Robustness of Neural Networks via Weight Scale Shifting Invariant Regularizations
Ziquan LiuYufei CuiAntoni B. Chan
2020-08-07
ConvBERT: Improving BERT with Span-based Dynamic Convolution
Zihang JiangWeihao YuDaquan ZhouYunpeng ChenJiashi FengShuicheng Yan
2020-08-06
DeText: A Deep Text Ranking Framework with BERT
| Weiwei GuoXiaowei LiuSida WangHuiji GaoAnanth SankarZimeng YangQi GuoLiang ZhangBo LongBee-Chung ChenDeepak Agarwal
2020-08-06
Taking Notes on the Fly Helps BERT Pre-training
Qiyu WuChen XingYatao LiGuolin KeDi HeTie-Yan Liu
2020-08-04
NLPDove at SemEval-2020 Task 12: Improving Offensive Language Detection with Cross-lingual Transfer
Hwijeen AhnJimin SunChan Young ParkJungyun Seo
2020-08-04
Improving One-stage Visual Grounding by Recursive Sub-query Construction
| Zhengyuan YangTianlang ChenLiwei WangJiebo Luo
2020-08-03
[email protected] at SemEval-2020 Task 12: Multilingual or language-specific BERT?
Marc PàmiesEmily ÖhmanKaisla KajavaJörg Tiedemann
2020-08-03
Trojaning Language Models for Fun and Profit
Xinyang ZhangZheng ZhangTing Wang
2020-08-01
Multi-node Bert-pretraining: Cost-efficient Approach
Jiahuang LinXin LiGennady Pekhimenko
2020-08-01
Finite Versus Infinite Neural Networks: an Empirical Study
Jaehoon LeeSamuel S. SchoenholzJeffrey PenningtonBen AdlamLechao XiaoRoman NovakJascha Sohl-Dickstein
2020-07-31
On Learning Universal Representations Across Languages
Xiangpeng WeiYue HuRongxiang WengLuxi XingHeng YuWeihua Luo
2020-07-31
Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
Yu GuRobert TinnHao ChengMichael LucasNaoto UsuyamaXiaodong LiuTristan NaumannJianfeng GaoHoifung Poon
2020-07-31
TweepFake: about Detecting Deepfake Tweets
Tiziano FagniFabrizio FalchiMargherita GambiniAntonio MartellaMaurizio Tesconi
2020-07-31
Model Reduction of Shallow CNN Model for Reliable Deployment of Information Extraction from Medical Reports
Abhishek K DubeyAlina PelusoJacob HinkleDevanshu AgarawalZilong Tan
2020-07-31
What does BERT know about books, movies and music? Probing BERT for Conversational Recommendation
| Gustavo PenhaClaudia Hauff
2020-07-30
Depressive, Drug Abusive, or Informative: Knowledge-aware Study of News Exposure during COVID-19 Outbreak
Amanuel AlamboManas GaurKrishnaprasad Thirunarayan
2020-07-30
MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering
| Shayne LongpreYi LuJoachim Daiber
2020-07-30
Composer Style Classification of Piano Sheet Music Images Using Language Model Pretraining
TJ TsaiKevin Ji
2020-07-29
Improving Results on Russian Sentiment Datasets
| Anton GolubevNatalia Loukachevitch
2020-07-28
BUT-FIT at SemEval-2020 Task 5: Automatic detection of counterfactual statements with deep pre-trained language representation models
Martin FajcikJosef JonMartin DocekalPavel Smrz
2020-07-28
Variants of BERT, Random Forests and SVM approach for Multimodal Emotion-Target Sub-challenge
Hoang Manh HungHyung-Jeong YangSoo-Hyung KimGuee-Sang Lee
2020-07-28
GUIR at SemEval-2020 Task 12: Domain-Tuned Contextualized Models for Offensive Language Detection
Sajad SotudehTong XiangHao-Ren YaoSean MacAvaneyEugene YangNazli GoharianOphir Frieder
2020-07-28
Deep Learning Brasil -- NLP at SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets
Manoel Veríssimo dos Santos NetoAyrton Denner da Silva AmaralNádia Félix Felipe da SilvaAnderson da Silva Soares
2020-07-28
KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media
| Ali SafayaMoutasem AbdullatifDeniz Yuret
2020-07-26
Reed at SemEval-2020 Task 9: Fine-Tuning and Bag-of-Words Approaches to Code-Mixed Sentiment Analysis
Vinay GopalanMark Hopkins
2020-07-26
To BERT or Not To BERT: Comparing Speech and Language-based Approaches for Alzheimer's Disease Detection
Aparna BalagopalanBenjamin EyreFrank RudziczJekaterina Novikova
2020-07-26
MULTISEM at SemEval-2020 Task 3: Fine-tuning BERT for Lexical Meaning
Aina Garí SolerMarianna Apidianaki
2020-07-24
Product Title Generation for Conversational Systems using BERT
Mansi Ranjit ManeShashank KediaAditya ManthaStephen GuoKannan Achan
2020-07-23
The Lottery Ticket Hypothesis for Pre-trained BERT Networks
| Tianlong ChenJonathan FrankleShiyu ChangSijia LiuYang ZhangZhangyang WangMichael Carbin
2020-07-23
IITK at the FinSim Task: Hypernym Detection in Financial Domain via Context-Free and Contextualized Word Embeddings
Vishal KeswaniSakshi SinghAshutosh Modi
2020-07-22
Multi-task learning for natural language processing in the 2020s: where are we going?
Joseph WorshamJugal Kalita
2020-07-22
problemConquero at SemEval-2020 Task 12: Transformer and Soft label-based approaches
Karishma LaudJagriti SinghRandeep Kumar SahuAshutosh Modi
2020-07-21
newsSweeper at SemEval-2020 Task 11: Context-Aware Rich Feature Representations For Propaganda Classification
| Paramansh SinghSiraj SandhuSubham KumarAshutosh Modi
2020-07-21
Word Representation for Rhythms
Tongyu LuLyucheng YanGus Xia
2020-07-21
Understanding BERT Rankers Under Distillation
Luyu GaoZhuyun DaiJamie Callan
2020-07-21
A Comparison of Supervised Learning to Match Methods for Product Search
| Fatemeh SarviNikos VoskaridesLois MooimanSebastian SchelterMaarten de Rijke
2020-07-20
Mono vs Multilingual Transformer-based Models: a Comparison across Several Language Tasks
Diego de Vargas FeijoViviane Pereira Moreira
2020-07-19
On regularization of gradient descent, layer imbalance and flat minima
Boris Ginsburg
2020-07-18
Generative Pretraining from Pixels
| Mark ChenAlec RadfordRewon ChildJeff WuHeewoo JunPrafulla DhariwalDavid LuanIlya Sutskever
2020-07-17
Multi-Perspective Semantic Information Retrieval in the Biomedical Domain
Samarth Rawal
2020-07-17
Towards Debiasing Sentence Representations
Paul Pu LiangIrene Mengze LiEmily ZhengYao Chong LimRuslan SalakhutdinovLouis-Philippe Morency
2020-07-16
Translate Reverberated Speech to Anechoic Ones: Speech Dereverberation with BERT
Yang Jiao
2020-07-16
Human-like Energy Management Based on Deep Reinforcement Learning and Historical Driving Experiences
Teng LiuXiaolin TangXiaosong HuWenhao TanJinwei Zhang
2020-07-16
Hopfield Networks is All You Need
| Hubert RamsauerBernhard SchäflJohannes LehnerPhilipp SeidlMichael WidrichLukas GruberMarkus HolzleitnerMilena PavlovićGeir Kjetil SandveVictor GreiffDavid KreilMichael KoppGünter KlambauerJohannes BrandstetterSepp Hochreiter
2020-07-16
Non-greedy Gradient-based Hyperparameter Optimization Over Long Horizons
| Paul MicaelliAmos Storkey
2020-07-15
AdapterHub: A Framework for Adapting Transformers
| Jonas PfeifferAndreas RückléClifton PothAishwarya KamathIvan VulićSebastian RuderKyunghyun ChoIryna Gurevych
2020-07-15
Multimodal Word Sense Disambiguation in Creative Practice
Manuel Ladron de GuevaraChristopher GeorgeAkshat GuptaDaragh ByrneRamesh Krishnamurti
2020-07-15
Logic Constrained Pointer Networks for Interpretable Textual Similarity
| Subhadeep MajiRohan KumarManish BansalKalyani RoyPawan Goyal
2020-07-15
Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks
Pavel BlinovManvel AvetisianVladimir KokhDmitry UmerenkovAlexander Tuzhilin
2020-07-15
Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media
Alberto Barron-CedenoTamer ElsayedPreslav NakovGiovanni Da San MartinoMaram HasanainReem SuwailehFatima HaouariNikolay BabulkovBayan HamdanAlex NikolovShaden ShaarZien Sheikh Ali
2020-07-15
Deep Reinforced Query Reformulation for Information Retrieval
Xiao WangCraig MacdonaldIadh Ounis
2020-07-15
Fast and Accurate Neural CRF Constituency Parsing
| Yu ZhangHouquan ZhouZhenghua Li
2020-07-14
Deep Transformer based Data Augmentation with Subword Units for Morphologically Rich Online ASR
Balázs TarjánGyörgy SzaszákTibor FegyóPéter Mihajlik
2020-07-14
What's in a Name? Are BERT Named Entity Representations just as Good for any other Name?
Sriram BalasubramanianNaman JainGaurav JindalAbhijeet AwasthiSunita Sarawagi
2020-07-14
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models
Lifu TuGarima LalwaniSpandana GellaHe He
2020-07-14
Can neural networks acquire a structural bias from raw linguistic data?
Alex WarstadtSamuel R. Bowman
2020-07-14
Emoji Prediction: Extensions and Benchmarking
Weicheng MaRuibo LiuLili WangSoroush Vosoughi
2020-07-14
Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning
Shauharda KhadkaEstelle AflaloMattias MarderAvrech Ben-DavidSantiago MiretHanlin TangShie MannorTamir HazanSomdeb Majumdar
2020-07-14
Add a SideNet to your MainNet
Adrien Morisot
2020-07-14
An Enhanced Text Classification to Explore Health based Indian Government Policy Tweets
Aarzoo DhimanDurga Toshniwal
2020-07-13
Generative Graph Perturbations for Scene Graph Prediction
Boris KnyazevHarm de VriesCătălina CangeaGraham W. TaylorAaron CourvilleEugene Belilovsky
2020-07-11
BERT Learns (and Teaches) Chemistry
Josh PayneMario SroujiDian Ang YapVineet Kosaraju
2020-07-11
To BAN or not to BAN: Bayesian Attention Networks for Reliable Hate Speech Detection
Kristian MiokBlaz SkrljDaniela ZaharieMarko Robnik-Sikonja
2020-07-10
BISON:BM25-weighted Self-Attention Framework for Multi-Fields Document Search
| Xuan ShanChuanjie LiuYiqian XiaQi ChenYusi ZhangAngen LuoYuxiang Luo
2020-07-10
Multi-Dialect Arabic BERT for Country-Level Dialect Identification
| Bashar TalafhaMohammad AliMuhy Eddin Za'terHaitham SeelawiIbraheem TuffahaMostafa SamirWael FarhanHussein T. Al-Natsheh
2020-07-10
Contrastive Code Representation Learning
| Paras JainAjay JainTianjun ZhangPieter AbbeelJoseph E. GonzalezIon Stoica
2020-07-09
Fast Transformers with Clustered Attention
| Apoorv VyasAngelos KatharopoulosFrançois Fleuret
2020-07-09
The Go Transformer: Natural Language Modeling for Game Play
Matthew CiolinoDavid NoeverJosh Kalin
2020-07-07
Continual BERT: Continual Learning for Adaptive Extractive Summarization of COVID-19 Literature
Jong Won Park
2020-07-07
Exploring Heterogeneous Information Networks via Pre-Training
Yang FangXiang ZhaoWeidong Xiao
2020-07-07
Deep Contextual Embeddings for Address Classification in E-commerce
Shreyas MangalgiLakshya KumarRavindra Babu Tallamraju
2020-07-06
You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion
Roei SchusterCongzheng SongEran TromerVitaly Shmatikov
2020-07-05
Text Data Augmentation: Towards better detection of spear-phishing emails
Mehdi ReginaMaxime MeyerSébastien Goutal
2020-07-04
Robust Prediction of Punctuation and Truecasing for Medical ASR
Monica SunkaraSrikanth RonankiKalpit DixitSravan BodapatiKatrin Kirchhoff
2020-07-04
Language-agnostic BERT Sentence Embedding
| Fangxiaoyu FengYinfei YangDaniel CerNaveen ArivazhaganWei Wang
2020-07-03
Pretrained Semantic Speech Embeddings for End-to-End Spoken Language Understanding via Cross-Modal Teacher-Student Learning
Pavel DenisovNgoc Thang Vu
2020-07-03
Reading Comprehension in Czech via Machine Translation and Cross-lingual Transfer
Kateřina MackováMilan Straka
2020-07-03
Playing with Words at the National Library of Sweden -- Making a Swedish BERT
| Martin MalmstenLove BörjesonChris Haffenden
2020-07-03
On-The-Fly Information Retrieval Augmentation for Language Models
Hai WangDavid McAllester
2020-07-03
MIRA: Leveraging Multi-Intention Co-click Information in Web-scale Document Retrieval using Deep Neural Networks
Yusi ZhangChuanjie LiuAngen LuoHui XueXuan ShanYuxiang LuoYiqian XiaYuanchi YanHaidong Wang
2020-07-03
Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey
Shivaji AlaparthiManit Mishra
2020-07-02
The Impact of Explanations on AI Competency Prediction in VQA
Kamran AlipourArijit RayXiao LinJurgen P. SchulzeYi YaoGiedrius T. Burachas
2020-07-02
On Dropout, Overfitting, and Interaction Effects in Deep Neural Networks
Benjamin LengerichEric P. XingRich Caruana
2020-07-02
Improving Event Detection using Contextual Word and Sentence Embeddings
Mariano MaisonnaveFernando DelbiancoFernando TohméAna MaguitmanEvangelos Milios
2020-07-02
On-The-Fly Information Retrieval Augmentation for Language Models
Hai WangDavid McAllester
2020-07-01
Unsupervised FAQ Retrieval with Question Generation and BERT
Yosi MassBoaz CarmeliHaggai RoitmanDavid Konopnicki
2020-07-01
GAN-BERT: Generative Adversarial Learning for Robust Text Classification with a Bunch of Labeled Examples
| Danilo CroceGiuseppe CastellucciRoberto Basili
2020-07-01
Integrating Multimodal Information in Large Pretrained Transformers
Wasifur RahmanMd Kamrul HasanSangwu LeeAmirAli Bagher ZadehChengfeng MaoLouis-Philippe MorencyEhsan Hoque
2020-07-01
Modelling Context and Syntactical Features for Aspect-based Sentiment Analysis
Minh Hieu PhanPhilip O. Ogunbona
2020-07-01
Roles and Utilization of Attention Heads in Transformer-based Neural Language Models
Jae-young JoSung-Hyon Myaeng
2020-07-01
Towards Holistic and Automatic Evaluation of Open-Domain Dialogue Generation
Bo PangErik NijkampWenjuan HanLinqi ZhouYixian LiuKewei Tu
2020-07-01
Adversarial and Domain-Aware BERT for Cross-Domain Sentiment Analysis
Chunning DuHaifeng SunJingyu WangQi QiJianxin Liao
2020-07-01
How does BERT's attention change when you fine-tune? An analysis methodology and a case study in negation scope
Yiyun ZhaoSteven Bethard
2020-07-01
Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work?
Yada PruksachatkunJason PhangHaokun LiuPhu Mon HtutXiaoyi ZhangRichard Yuanzhe PangClara VaniaKatharina KannSamuel R. Bowman
2020-07-01
Would you Rather? A New Benchmark for Learning Machine Alignment with Cultural Values and Social Preferences
Yi TayDonovan OngJie FuAlvin ChanNancy ChenAnh Tuan LuuChris Pal
2020-07-01
Towards Debiasing Sentence Representations
Paul Pu LiangIrene Mengze LiEmily ZhengYao Chong LimRuslan SalakhutdinovLouis-Philippe Morency
2020-07-01
Automatic Generation of Citation Texts in Scholarly Papers: A Pilot Study
Xinyu XingXiaosheng FanXiaojun Wan
2020-07-01
Transition-based Semantic Dependency Parsing with Pointer Networks
Daniel Fern{\'a}ndez-Gonz{\'a}lezCarlos G{\'o}mez-Rodr{\'\i}guez
2020-07-01
tBERT: Topic Models and BERT Joining Forces for Semantic Similarity Detection
Nicole PeineltDong NguyenMaria Liakata
2020-07-01
Understanding Advertisements with BERT
Kanika KalraBhargav KurmaSilpa Vadakkeeveetil SreelathaManasi PatwardhanKarShirish e
2020-07-01
Feature Projection for Improved Text Classification
Qi QinWenpeng HuBing Liu
2020-07-01
A Generate-and-Rank Framework with Semantic Type Regularization for Biomedical Concept Normalization
Dongfang XuZeyu ZhangSteven Bethard
2020-07-01
Revisiting Higher-Order Dependency Parsers
Erick FonsecaAndr{\'e} F. T. Martins
2020-07-01
SUPP.AI: finding evidence for supplement-drug interactions
Lucy WangOyvind TafjordArman CohanSarthak JainSam SkjonsbergCarissa SchoenickNick BotnerWaleed Ammar
2020-07-01
Why is penguin more similar to polar bear than to sea gull? Analyzing conceptual knowledge in distributional models
Pia Sommerauer
2020-07-01
A Simple and Effective Dependency Parser for Telugu
Sneha NallaniManish ShrivastavaDipti Sharma
2020-07-01
Cross-Lingual Disaster-related Multi-label Tweet Classification with Manifold Mixup
Jishnu Ray ChowdhuryCornelia CarageaDoina Caragea
2020-07-01
Should You Fine-Tune BERT for Automated Essay Scoring?
Elijah MayfieldAlan W Black
2020-07-01
A BERT-based One-Pass Multi-Task Model for Clinical Temporal Relation Extraction
Chen LinTimothy MillerDmitriy DligachFarig SadequeSteven BethardGuergana Savova
2020-07-01
Evaluating the Utility of Model Configurations and Data Augmentation on Clinical Semantic Textual Similarity
Yuxia WangFei LiuKarin VerspoorTimothy Baldwin
2020-07-01
Item-based Collaborative Filtering with BERT
Tian WangYuyangzi Fu
2020-07-01
Sarcasm Identification and Detection in Conversion Context using BERT
Kalaivani A.Thenmozhi D.
2020-07-01
Neural Sarcasm Detection using Conversation Context
Nikhil Jaiswal
2020-07-01
Context-Aware Sarcasm Detection Using BERT
Arup BaruahKaushik DasFerdous BarbhuiyaKuntal Dey
2020-07-01
Character aware models with similarity learning for metaphor detection
Tarun KumarYashvardhan Sharma
2020-07-01
IlliniMet: Illinois System for Metaphor Detection with Contextual and Linguistic Information
Hongyu GongKshitij GuptaAkriti JainSuma Bhat
2020-07-01
Go Figure! Multi-task transformer-based architecture for metaphor detection using idioms: ETS team in 2020 metaphor shared task
Xianyang ChenChee Wee (Ben) LeongMichael FlorBeata Beigman Klebanov
2020-07-01
Metaphor Detection Using Contextual Word Embeddings From Transformers
Jerry LiuNathan O{'}HaraAlex RubinerRachel DraelosCynthia Rudin
2020-07-01
A Transformer Approach to Contextual Sarcasm Detection in Twitter
Hunter GregorySteven LiPouya MohammadiNatalie TarnRachel DraelosCynthia Rudin
2020-07-01
Turku Enhanced Parser Pipeline: From Raw Text to Enhanced Graphs in the IWPT 2020 Shared Task
Jenna KanervaFilip GinterSampo Pyysalo
2020-07-01
K\opsala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding
Daniel HershcovichMiryam de LhoneuxArtur KulmizevElham PejhanJoakim Nivre
2020-07-01
RobertNLP at the IWPT 2020 Shared Task: Surprisingly Simple Enhanced UD Parsing for English
Stefan Gr{\"u}newaldAnnemarie Friedrich
2020-07-01
The HW-TSC Video Speech Translation System at IWSLT 2020
Minghan WangHao YangYao DengYing QinLizhi LeiDaimeng WeiHengchao ShangNing XieXiaochun LiJiaxian Guo
2020-07-01
CopyBERT: A Unified Approach to Question Generation with Self-Attention
Stalin VaranasiSaadullah AminGuenter Neumann
2020-07-01
Robust Prediction of Punctuation and Truecasing for Medical ASR
Monica SunkaraSrikanth RonankiKalpit DixitSravan BodapatiKatrin Kirchhoff
2020-07-01
Exploring the Limits of Simple Learners in Knowledge Distillation for Document Classification with DocBERT
Ashutosh AdhikariAchyudh RamRaphael TangWilliam L. HamiltonJimmy Lin
2020-07-01
Joint Training with Semantic Role Labeling for Better Generalization in Natural Language Inference
Cemil CengizDeniz Yuret
2020-07-01
A Metric Learning Approach to Misogyny Categorization
Juan Manuel CoriaSahar GhannaySophie RossetHerv{\'e} Bredin
2020-07-01
Contextual and Non-Contextual Word Embeddings: an in-depth Linguistic Investigation
Alessio MiaschiFelice Dell{'}Orletta
2020-07-01
What's in a Name? Are BERT Named Entity Representations just as Good for any other Name?
Sriram BalasubramanianNaman JainGaurav JindalAbhijeet AwasthiSunita Sarawagi
2020-07-01
Getting the \#\#life out of living: How Adequate Are Word-Pieces for Modelling Complex Morphology?
Stav KleinReut Tsarfaty
2020-07-01
SentiTel: TABSA for Twitter reviews on Uganda Telecoms
David KabiitoJoyce Nakatumba Nabende
2020-07-01
Adversarial Evaluation of BERT for Biomedical Named Entity Recognition
Vladimir AraujoAndr{\'e}s CarvalloDenis Parra
2020-07-01
Improving Multimodal Named Entity Recognition via Entity Span Detection with Unified Multimodal Transformer
Jianfei YuJing JiangLi YangRui Xia
2020-07-01
Investigating the effect of auxiliary objectives for the automated grading of learner English speech transcriptions
Hannah CraigheadAndrew CainesPaula ButteryHelen Yannakoudakis
2020-07-01
Regularly Updated Deterministic Policy Gradient Algorithm
Shuai HanWenbo ZhouShuai LüJiayu Yu
2020-07-01
LSTM and GPT-2 Synthetic Speech Transfer Learning for Speaker Recognition to Overcome Data Scarcity
Jordan J. BirdDiego R. FariaAnikó EkártCristiano PremebidaPedro P. S. Ayrosa
2020-07-01
The Summary Loop: Learning to Write Abstractive Summaries Without Examples
| Philippe LabanAndrew HsiJohn CannyMarti A. Hearst
2020-07-01
SE3M: A Model for Software Effort Estimation Using Pre-trained Embedding Models
Eliane M. De Bortoli FáveroDalcimar CasanovaAndrey Ricardo Pimentel
2020-06-30
Data Movement Is All You Need: A Case Study on Optimizing Transformers
Andrei IvanovNikoli DrydenTal Ben-NunShigang LiTorsten Hoefler
2020-06-30
Segmentation Approach for Coreference Resolution Task
Aref JafariAli Ghodsi
2020-06-30
Want to Identify, Extract and Normalize Adverse Drug Reactions in Tweets? Use RoBERTa
Katikapalli Subramanyam KalyanS. Sangeetha
2020-06-29
Improving Sequence Tagging for Vietnamese Text Using Transformer-based Neural Models
Viet Bui TheOanh Tran ThiPhuong Le-Hong
2020-06-29
Knowledge-Aware Language Model Pretraining
Corby RossetChenyan XiongMinh PhanXia SongPaul BennettSaurabh Tiwary
2020-06-29
Interpreting Hierarchical Linguistic Interactions in DNNs
Die ZhangHuilin ZhouXiaoyi BaoDa HuoRuizhao ChenXu ChengHao ZhangMengyue WuQuanshi Zhang
2020-06-29
Progressive Generation of Long Text
| Bowen TanZichao YangMaruan AI-ShedivatEric P. XingZhiting Hu
2020-06-28
Rethinking Positional Encoding in Language Pre-training
| Guolin KeDi HeTie-Yan Liu
2020-06-28
BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision
| Chen LiangYue YuHaoming JiangSiawpeng ErRuijia WangTuo ZhaoChao Zhang
2020-06-28
Video-Grounded Dialogues with Pretrained Generation Language Models
Hung LeSteven C. H. Hoi
2020-06-27
Distributed Uplink Beamforming in Cell-Free Networks Using Deep Reinforcement Learning
Firas FredjYasser Al-EryaniSetareh MaghsudiMohamed AkroutEkram Hossain
2020-06-26
Noise, overestimation and exploration in Deep Reinforcement Learning
Rafael Stekolshchik
2020-06-25
FastSpec: Scalable Generation and Detection of Spectre Gadgets Using Neural Embeddings
| M. Caner TolKoray YurtsevenBerk GulmezogluBerk Sunar
2020-06-25
Normalizing Text using Language Modelling based on Phonetics and String Similarity
Fenil DoshiJimit GandhiDeep GosaliaSudhir Bagul
2020-06-25
LSBert: A Simple Framework for Lexical Simplification
| Jipeng QiangYun LiYi ZhuYunhao YuanXindong Wu
2020-06-25
Accelerated Large Batch Optimization of BERT Pretraining in 54 minutes
Shuai ZhengHaibin LinSheng ZhaMu Li
2020-06-24
Efficient Constituency Parsing by Pointing
Thanh-Tung NguyenXuan-Phi NguyenShafiq JotyXiaoli Li
2020-06-24
The Effect of Multi-step Methods on Overestimation in Deep Reinforcement Learning
Lingheng MengRob GorbetDana Kulić
2020-06-23
ReCO: A Large Scale Chinese Reading Comprehension Dataset on Opinion
| BingningWangTing YaoQi ZhangJingfang XuXiaochuan Wang
2020-06-22
Students Need More Attention: BERT-based AttentionModel for Small Data with Application to AutomaticPatient Message Triage
Shijing SiRui WangJedrek WosikHao ZhangDavid DovGuoyin WangRicardo HenaoLawrence Carin
2020-06-22
Sarcasm Detection in Tweets with BERT and GloVe Embeddings
Akshay KhatriPranav PDr. Anand Kumar M
2020-06-20
New Vietnamese Corpus for Machine ReadingComprehension of Health News Articles
Kiet Van NguyenDuc-Vu NguyenAnh Gia-Tuan NguyenNgan Luu-Thuy Nguyen
2020-06-19
A Qualitative Evaluation of Language Models on Automatic Question-Answering for COVID-19
| David OnianiYanshan Wang
2020-06-19
SqueezeBERT: What can computer vision teach NLP about efficient neural networks?
Forrest N. IandolaAlbert E. ShawRavi KrishnaKurt W. Keutzer
2020-06-19
Reducing Estimation Bias via Weighted Delayed Deep Deterministic Policy Gradient
Qiang HeXinwen Hou
2020-06-18
Exploring the BERT Cross-Lingual Transferability: a Case Study in Reading Comprehension
Konovalov V. P.Gulyaev P. A.Sorokin A. A.Kuratov Y. M.Burtsev M. S.
2020-06-17
Tagging and parsing of multidomain collections
| Alexey SorokinIvan SmurovDenis Kirianov
2020-06-17
Improving accuracy and speeding up Document Image Classification through parallel systems
| Javier FerrandoJuan Luis DominguezJordi TorresRaul GarciaDavid GarciaDaniel GarridoJordi CortadaMateo Valero
2020-06-16
PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models
| Eyal Ben-DavidCarmel RabinovitzRoi Reichart
2020-06-16
The SPPD System for Schema Guided Dialogue State Tracking Challenge
Miao LiHaoqi XiongYunbo Cao
2020-06-16
Scalable Cross Lingual Pivots to Model Pronoun Gender for Translation
Kellie WebsterEmily Pitler
2020-06-16
End-to-End Code Switching Language Models for Automatic Speech Recognition
Ahan M. R.Shreyas Sunil Kulkarni
2020-06-16
Spherical Motion Dynamics of Deep Neural Networks with Batch Normalization and Weight Decay
Ruosi WanZhanxing ZhuXiangyu ZhangJian Sun
2020-06-15
An online evolving framework for advancing reinforcement-learning based automated vehicle control
Teawon HanSubramanya NageshraoDimitar P. FilevUmit Ozguner
2020-06-15
Document Classification for COVID-19 Literature
Bernal Jiménez GutiérrezJuncheng ZengDongdong ZhangPing ZhangYu Su
2020-06-15
FinBERT: A Pretrained Language Model for Financial Communications
| Yi YangMark Christopher Siy UYAllen Huang
2020-06-15
Cooking Is All About People: Comment Classification On Cookery Channels Using BERT and Classification Models (Malayalam-English Mix-Code)
Subramaniam KazhuparambilAbhishek Kaushik
2020-06-15
FinEst BERT and CroSloEngual BERT: less is more in multilingual models
Matej UlčarMarko Robnik-Šikonja
2020-06-14
Transferring Monolingual Model to Low-Resource Language: The Case of Tigrinya
Abrhalei TelaAbraham WoubieVille Hautamaki
2020-06-13
Human and Multi-Agent collaboration in a human-MARL teaming framework
Neda NavidiFrancois ChabotSagar KurandwadIrv LustigmanVincent RobertGregory SzriftgiserAndrea Schuch
2020-06-12
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages
Pedro Javier Ortiz SuárezLaurent RomaryBenoît Sagot
2020-06-11
All Local Minima are Global for Two-Layer ReLU Neural Networks: The Hidden Convex Optimization Landscape
Jonathan LacotteMert Pilanci
2020-06-10
MC-BERT: Efficient Language Pre-Training via a Meta Controller
| Zhenhui XuLinyuan GongGuolin KeDi HeShuxin ZhengLiwei WangJiang BianTie-Yan Liu
2020-06-10
Neural Networks, Ridge Splines, and TV Regularization in the Radon Domain
Rahul ParhiRobert D. Nowak
2020-06-10
Revisiting Few-sample BERT Fine-tuning
| Tianyi ZhangFelix WuArzoo KatiyarKilian Q. WeinbergerYoav Artzi
2020-06-10
Unsupervised Paraphrase Generation using Pre-trained Language Models
Chaitra HegdeShrikumar Patil
2020-06-09
Few-Shot Generative Conversational Query Rewriting
| Shi YuJiahua LiuJingqin YangChenyan XiongPaul BennettJianfeng GaoZhiyuan Liu
2020-06-09
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines
| Marius MosbachMaksym AndriushchenkoDietrich Klakow
2020-06-08
Pre-training Polish Transformer-based Language Models at Scale
| Sławomir DadasMichał PerełkiewiczRafał Poświata
2020-06-07
Medical Concept Normalization in User Generated Texts by Learning Target Concept Embeddings
Katikapalli Subramanyam KalyanS. Sangeetha
2020-06-07
GMAT: Global Memory Augmentation for Transformers
| Ankit GuptaJonathan Berant
2020-06-05
Accelerating Natural Language Understanding in Task-Oriented Dialog
Ojas AhujaShrey Desai
2020-06-05
UDPipe at EvaLatin 2020: Contextualized Embeddings and Treebank Embeddings
Milan StrakaJana Straková
2020-06-05
DeBERTa: Decoding-enhanced BERT with Disentangled Attention
| Pengcheng HeXiaodong LiuJianfeng GaoWeizhu Chen
2020-06-05
The SOFC-Exp Corpus and Neural Approaches to Information Extraction in the Materials Science Domain
Annemarie FriedrichHeike AdelFederico TomazicJohannes HingerlRenou BenteauAnika MaruscykLukas Lange
2020-06-04
Automatic Text Summarization of COVID-19 Medical Research Articles using BERT and GPT-2
| Virapat KieuvongngamBowen TanYiming Niu
2020-06-03
WikiBERT models: deep transfer learning for many languages
Sampo PyysaloJenna KanervaAntti VirtanenFilip Ginter
2020-06-02
Question Answering on Scholarly Knowledge Graphs
Mohamad Yaser JaradehMarkus StockerSören Auer
2020-06-02
A Pairwise Probe for Understanding BERT Fine-Tuning on Machine Reading Comprehension
Jie CaiZhengzhou ZhuPing NieQian Liu
2020-06-02
BERT Based Multilingual Machine Comprehension in English and Hindi
| Somil GuptaNilesh Khade
2020-06-02
Exploring Cross-sentence Contexts for Named Entity Recognition with BERT
Jouni LuomaSampo Pyysalo
2020-06-02
Position Masking for Language Models
Andy WagnerTiyasa MitraMrinal IyerGodfrey Da CostaMarc Tremblay
2020-06-02
Emergence of Separable Manifolds in Deep Language Representations
Jonathan MamouHang LeMiguel Del RioCory StephensonHanlin TangYoon KimSueYeon Chung
2020-06-01
Conversational Machine Comprehension: a Literature Review
Somil GuptaBhanu Pratap Singh Rawat
2020-06-01
When Bert Forgets How To POS: Amnesic Probing of Linguistic Properties and MLM Predictions
Yanai ElazarShauli RavfogelAlon JacoviYoav Goldberg
2020-06-01
An Effective Contextual Language Modeling Framework for Speech Summarization with Augmented Features
Shi-Yan WengTien-Hong LoBerlin Chen
2020-06-01
BERT-based Ensembles for Modeling Disclosure and Support in Conversational Social Media Text
Tanvi DaduKartikey PantRadhika Mamidi
2020-06-01
Neural Entity Linking: A Survey of Models based on Deep Learning
| Ozge SevgiliArtem ShelmanovMikhail ArkhipovAlexander PanchenkoChris Biemann
2020-05-31
"Judge me by my size (noun), do you?'' YodaLib: A Demographic-Aware Humor Generation Framework
Aparna GarimellaCarmen BaneaNabil HossainRada Mihalcea
2020-05-31
BPGC at SemEval-2020 Task 11: Propaganda Detection in News Articles with Multi-Granularity Knowledge Sharing and Linguistic Features based Ensemble Learning
Rajaswa PatilSomesh SinghSwati Agarwal
2020-05-31
LRG at SemEval-2020 Task 7: Assessing the Ability of BERT and Derivative Models to Perform Short-Edits based Humor Grading
Siddhant MahurkarRajaswa Patil
2020-05-31
Detecting Problem Statements in Peer Assessments
Yunkai XiaoGabriel ZingleQinjin JiaHarsh R. ShahYi ZhangTianyi LiMohsin KarovaliyaWeixiang ZhaoYang SongJie JiAshwin BalasubramaniamHarshit PatelPriyankha BhalasubbramanianVikram PatelEdward F. Gehringer
2020-05-30
First Neural Conjecturing Datasets and Experiments
Josef UrbanJan Jakubův
2020-05-29
Using Large Pretrained Language Models for Answering User Queries from Product Specifications
Kalyani RoySmit ShahNithish PaiJaidam RamtejPrajit Prashant NadkarnJyotirmoy BanerjeePawan GoyalSurender Kumar
2020-05-29
SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions
Mao YeChengyue GongQiang Liu
2020-05-29
A Comparative Study of Lexical Substitution Approaches based on Neural Language Models
Nikolay ArefyevBoris SheludkoAlexander PodolskiyAlexander Panchenko
2020-05-29
Stance Prediction for Contemporary Issues: Data and Experiments
| Marjan HosseiniaEduard DragutArjun Mukherjee
2020-05-29
On Incorporating Structural Information to improve Dialogue Response Generation
| Nikita MoghePriyesh VijayanBalaraman RavindranMitesh M. Khapra
2020-05-28
Language Models are Few-Shot Learners
| Tom B. BrownBenjamin MannNick RyderMelanie SubbiahJared KaplanPrafulla DhariwalArvind NeelakantanPranav ShyamGirish SastryAmanda AskellSandhini AgarwalAriel Herbert-VossGretchen KruegerTom HenighanRewon ChildAditya RameshDaniel M. ZieglerJeffrey WuClemens WinterChristopher HesseMark ChenEric SiglerMateusz LitwinScott GrayBenjamin ChessJack ClarkChristopher BernerSam McCandlishAlec RadfordIlya SutskeverDario Amodei
2020-05-28
Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Adhiguna KuncoroLingpeng KongDaniel FriedDani YogatamaLaura RimellChris DyerPhil Blunsom
2020-05-27
CausaLM: Causal Model Explanation Through Counterfactual Language Models
Amir FederNadav OvedUri ShalitRoi Reichart
2020-05-27
Transition-based Semantic Dependency Parsing with Pointer Networks
Daniel Fernández-GonzálezCarlos Gómez-Rodríguez
2020-05-27
Language Representation Models for Fine-Grained Sentiment Classification
Brian CheangBailey WeiDavid KoganHowey QiuMasud Ahmed
2020-05-27
Network Fusion for Content Creation with Conditional INNs
Robin RombachPatrick EsserBjörn Ommer
2020-05-27
A Data-driven Approach for Noise Reduction in Distantly Supervised Biomedical Relation Extraction
Saadullah AminKatherine Ann DunfieldAnna VechkaevaGünter Neumann
2020-05-26
What Are People Asking About COVID-19? A Question Classification Dataset
| Jerry WeiChengyu HuangSoroush VosoughiJason Wei
2020-05-26
ParsBERT: Transformer-based Model for Persian Language Understanding
| Mehrdad FarahaniMohammad GharachorlooMarzieh FarahaniMohammad Manthouri
2020-05-26
BEEP! Korean Corpus of Online News Comments for Toxic Speech Detection
| Jihyung MoonWon Ik ChoJunbum Lee
2020-05-26
Comparing BERT against traditional machine learning text classification
Santiago González-CarvajalEduardo C. Garrido-Merchán
2020-05-26
BERT-XML: Large Scale Automated ICD Coding Using BERT Pretraining
Zachariah ZhangJingshu LiuNarges Razavian
2020-05-26
Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications
Jiaye LinYuze ZouXiaoru DongShimin GongDinh Thai HoangDusit Niyato
2020-05-25
An Audio-enriched BERT-based Framework for Spoken Multiple-choice Question Answering
Chia-Chih KuoShang-Bao LuoKuan-Yu Chen
2020-05-25
Køpsala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding
| Daniel HershcovichMiryam de LhoneuxArtur KulmizevElham PejhanJoakim Nivre
2020-05-25
Pointwise Paraphrase Appraisal is Potentially Problematic
Hannah ChenYangfeng JiDavid Evans
2020-05-25
Jointly Encoding Word Confusion Network and Dialogue Context with BERT for Spoken Language Understanding
Chen LiuSu ZhuZijian ZhaoRuisheng CaoLu ChenKai Yu
2020-05-24
Comparative Study of Machine Learning Models and BERT on SQuAD
Devshree PatelParam RavalRatnam ParikhYesha Shastri
2020-05-22
L2R2: Leveraging Ranking for Abductive Reasoning
| Yunchang ZhuLiang PangYanyan LanXueqi Cheng
2020-05-22
Living Machines: A study of atypical animacy
Mariona Coll ArdanuyFederico NanniKaspar BeelenKasra HosseiniRuth AhnertJon LawrenceKatherine McDonoughGiorgia TolfoDaniel CS WilsonBarbara McGillivray
2020-05-22
Robust Layout-aware IE for Visually Rich Documents with Pre-trained Language Models
Mengxi WeiYifan HeQiong Zhang
2020-05-22
Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction
Laila RasmyYang XiangZiqian XieCui TaoDegui Zhi
2020-05-22
Supervised Learning in the Presence of Concept Drift: A modelling framework
Michiel StraatFthi AbadiZhuoyun KanChristina GöpfertBarbara HammerMichael Biehl
2020-05-21
BERTweet: A pre-trained language model for English Tweets
| Dat Quoc NguyenThanh VuAnh Tuan Nguyen
2020-05-20
Creative Artificial Intelligence -- Algorithms vs. humans in an incentivized writing competition
Nils KöbisLuca Mossink
2020-05-20
FashionBERT: Text and Image Matching with Adaptive Loss for Cross-modal Retrieval
Dehong GaoLinbo JinBen ChenMinghui QiuPeng LiYi WeiYi HuHao Wang
2020-05-20
Cross-lingual Transfer Learning for Dialogue Act Recognition
Jiří MartínekChristophe CerisaraPavel KrálLadislav Lenc
2020-05-19
Table Search Using a Deep Contextualized Language Model
| Zhiyu ChenMohamed TrabelsiJeff HeflinYinan XuBrian D. Davison
2020-05-19
Experience Augmentation: Boosting and Accelerating Off-Policy Multi-Agent Reinforcement Learning
Zhenhui YeYining ChenGuanghua SongBowei YangShen Fan
2020-05-19
Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestalt
Hangyu LinYanwei FuYu-Gang JiangXiangyang Xue
2020-05-19
Are All Languages Created Equal in Multilingual BERT?
Shijie WuMark Dredze
2020-05-18
Context-Based Quotation Recommendation
Ansel MacLaughlinTao ChenBurcu Karagol AyanDan Roth
2020-05-17
Support-BERT: Predicting Quality of Question-Answer Pairs in MSDN using Deep Bidirectional Transformer
Bhaskar SenNikhil GopalXinwei Xue
2020-05-17
Building a Hebrew Semantic Role Labeling Lexical Resource from Parallel Movie Subtitles
Ben EyalMichael Elhadad
2020-05-17
Cross-Lingual Low-Resource Set-to-Description Retrieval for Global E-Commerce
| Juntao LiChang LiuJian WangLidong BingHongsong LiXiaozhong LiuDongyan ZhaoRui Yan
2020-05-17
Adversarial Training for Commonsense Inference
Lis PereiraXiaodong LiuFei ChengMasayuki AsaharaIchiro Kobayashi
2020-05-17
TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data
| Pengcheng YinGraham NeubigWen-tau YihSebastian Riedel
2020-05-17
CERT: Contrastive Self-supervised Learning for Language Understanding
Hongchao FangSicheng WangMeng ZhouJiayuan DingPengtao Xie
2020-05-16
Leveraging Affective Bidirectional Transformers for Offensive Language Detection
AbdelRahim ElmadanyChiyu ZhangMuhammad Abdul-MageedAzadeh Hashemi
2020-05-16
Spelling Error Correction with Soft-Masked BERT
| Shaohua ZhangHaoran HuangJicong LiuHang Li
2020-05-15
Neural Entity Linking on Technical Service Tickets
Nadja KurzFelix HamannAdrian Ulges
2020-05-15
Challenges in Emotion Style Transfer: An Exploration with a Lexical Substitution Pipeline
David HelbigEnrica TroianoRoman Klinger
2020-05-15
[email protected] at SemEval-2020 Task 12: Identifying Multilingual Offensive Tweets Using Weighted Ensemble and Fine-Tuned BERT
Saja Khaled TawalbehMahmoud HammadMohammad AL-Smadi
2020-05-15
NIT-Agartala-NLP-Team at SemEval-2020 Task 8: Building Multimodal Classifiers to tackle Internet Humor
Steve Durairaj SwamyShubham LaddhaBasil AbdussalamDebayan DattaAnupam Jamatia
2020-05-14
A pre-training technique to localize medical BERT and enhance BioBERT
| Shoya WadaToshihiro TakedaShiro ManabeShozo KonishiJun KamoharaYasushi Matsumura
2020-05-14
Parallel Corpus Filtering via Pre-trained Language Models
Boliang ZhangAjay NageshKevin Knight
2020-05-13
Large Scale Multi-Actor Generative Dialog Modeling
Alex BoydRaul PuriMohammad ShoeybiMostofa PatwaryBryan Catanzaro
2020-05-13
Entity-Enriched Neural Models for Clinical Question Answering
| Bhanu Pratap Singh RawatWei-Hung WengPreethi RaghavanPeter Szolovits
2020-05-13
On the Robustness of Language Encoders against Grammatical Errors
Fan YinQuanyu LongTao MengKai-Wei Chang
2020-05-12
On the Generation of Medical Dialogues for COVID-19
| Wenmian YangGuangtao ZengBowen TanZeqian JuSubrato ChakravortyXuehai HeShu ChenXingyi YangQingyang WuZhou YuEric XingPengtao Xie
2020-05-11
Detecting Adverse Drug Reactions from Twitter through Domain-Specific Preprocessing and BERT Ensembling
Amy BredenLee Moore
2020-05-11
How Context Affects Language Models' Factual Predictions
Fabio PetroniPatrick LewisAleksandra PiktusTim RocktäschelYuxiang WuAlexander H. MillerSebastian Riedel
2020-05-10
Transformer Based Language Models for Similar Text Retrieval and Ranking
Javed Qadrud-DinAshraf Bah RabiouRyan WalkerRavi SoniMartin GajekGabriel PackAkhil Rangaraj
2020-05-10
An FPGA-Based On-Device Reinforcement Learning Approach using Online Sequential Learning
Hirohisa WatanabeMineto TsukadaHiroki Matsutani
2020-05-10
Finding Universal Grammatical Relations in Multilingual BERT
Ethan A. ChiJohn HewittChristopher D. Manning
2020-05-09
It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations
| Samson TanShafiq JotyMin-Yen KanRichard Socher
2020-05-09
LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluation
Gustavo AguilarSudipta KarThamar Solorio
2020-05-09
schuBERT: Optimizing Elements of BERT
Ashish KhetanZohar Karnin
2020-05-09
SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics
| Da YinTao MengKai-Wei Chang
2020-05-08
Distilling Knowledge from Pre-trained Language Models via Text Smoothing
Xing WuYibing LiuXiangyang ZhouDianhai Yu
2020-05-08
GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference
Ali Hadi ZadehAndreas Moshovos
2020-05-08
Temporal Common Sense Acquisition with Minimal Supervision
Ben ZhouQiang NingDaniel KhashabiDan Roth
2020-05-08
Comparative Analysis of Text Classification Approaches in Electronic Health Records
Aurelie MascioZeljko KraljevicDaniel BeanRichard DobsonRobert StewartRebecca BendayanAngus Roberts
2020-05-08
LIIR at SemEval-2020 Task 12: A Cross-Lingual Augmentation Approach for Multilingual Offensive Language Identification
Erfan GhaderyMarie-Francine Moens
2020-05-07
Harvesting and Refining Question-Answer Pairs for Unsupervised QA
| Zhongli LiWenhui WangLi DongFuru WeiKe Xu
2020-05-06
An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining
Yifan PengQingyu ChenZhiyong Lu
2020-05-06
Autoencoding Pixies: Amortised Variational Inference with Graph Convolutions for Functional Distributional Semantics
Guy Emerson
2020-05-06
Categorical Vector Space Semantics for Lambek Calculus with a Relevant Modality
Lachlan McPheatMehrnoosh SadrzadehHadi WazniGijs Wijnholds
2020-05-06
MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models
| Mandy GuoYinfei YangDaniel CerQinlan ShenNoah Constant
2020-05-05
Contextualizing Hate Speech Classifiers with Post-hoc Explanation
Brendan KennedyXisen JinAida Mostafazadeh DavaniMorteza DehghaniXiang Ren
2020-05-05
Establishing Baselines for Text Classification in Low-Resource Languages
| Jan Christian Blaise CruzCharibeth Cheng
2020-05-05
ExpBERT: Representation Engineering with Natural Language Explanations
| Shikhar MurtyPang Wei KohPercy Liang
2020-05-05
ImpactCite: An XLNet-based method for Citation Impact Analysis
Dominique MercierSyed Tahseen Raza RizviVikas RajashekarAndreas DengelSheraz Ahmed
2020-05-05
Distributional Discrepancy: A Metric for Unconditional Text Generation
| Ping CaiXingyuan ChenPeng JinHongjun WangTianrui Li
2020-05-04
Spying on your neighbors: Fine-grained probing of contextual embeddings for information about surrounding words
Josef KlafkaAllyson Ettinger
2020-05-04
Robust Encodings: A Framework for Combating Adversarial Typos
Erik JonesRobin JiaAditi RaghunathanPercy Liang
2020-05-04
Unsupervised Alignment-based Iterative Evidence Retrieval for Multi-hop Question Answering
| Vikas YadavSteven BethardMihai Surdeanu
2020-05-04
Code and Named Entity Recognition in StackOverflow
| Jeniya TabassumMounica MaddelaWei XuAlan Ritter
2020-05-04
Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction
| Masahiro KanekoMasato MitaShun KiyonoJun SuzukiKentaro Inui
2020-05-03
Transformer-based End-to-End Question Generation
| Luis Enrico LopezDiane Kathryn CruzJan Christian Blaise CruzCharibeth Cheng
2020-05-03
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA
Nora KassnerHinrich Schütze
2020-05-02
DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering
| Qingqing CaoHarsh TrivediAruna BalasubramanianNiranjan Balasubramanian
2020-05-02
Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models
Bill Yuchen LinSeyeon LeeRahul KhannaXiang Ren
2020-05-02
Generating Derivational Morphology with BERT
Valentin HofmannJanet B. PierrehumbertHinrich Schütze
2020-05-02
IsoBN: Fine-Tuning BERT with Isotropic Batch Normalization
| Wenxuan ZhouBill Yuchen LinXiang Ren
2020-05-02
A Simple Language Model for Task-Oriented Dialogue
| Ehsan Hosseini-AslBryan McCannChien-Sheng WuSemih YavuzRichard Socher
2020-05-02
Contrastive Self-Supervised Learning for Commonsense Reasoning
| Tassilo KleinMoin Nabi
2020-05-02
A Controllable Model of Grounded Response Generation
Zeqiu WuMichel GalleyChris BrockettYizhe ZhangXiang GaoChris QuirkRik Koncel-KedziorskiJianfeng GaoHannaneh HajishirziMari OstendorfBill Dolan
2020-05-01
HipoRank: Incorporating Hierarchical and Positional Information into Graph-based Unsupervised Long Document Extractive Summarization
Yue DongAndrei RomascanuJackie C. K. Cheung
2020-05-01
Identifying Necessary Elements for BERT's Multilinguality
| Philipp DufterHinrich Schütze
2020-05-01
Hitachi at SemEval-2020 Task 12: Offensive Language Identification with Noisy Labels using Statistical Sampling and Post-Processing
Manikandan RavikiranAmin Ekant MuljibhaiToshinori MiyoshiHiroaki OzakiYuta KoreedaSakata Masayuki
2020-05-01
Cross-Linguistic Syntactic Evaluation of Word Prediction Models
| Aaron MuellerGarrett NicolaiPanayiota Petrou-ZeniouNatalia TalminaTal Linzen
2020-05-01
Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?
Yada PruksachatkunJason PhangHaokun LiuPhu Mon HtutXiaoyi ZhangRichard Yuanzhe PangClara VaniaKatharina KannSamuel R. Bowman
2020-05-01
Clinical Reading Comprehension: A Thorough Analysis of the emrQA Dataset
Xiang YueBernal Jimenez GutierrezHuan Sun
2020-05-01
When BERT Plays the Lottery, All Tickets Are Winning
Sai PrasannaAnna RogersAnna Rumshisky
2020-05-01
POINTER: Constrained Text Generation via Insertion-based Generative Pre-training
| Yizhe ZhangGuoyin WangChunyuan LiZhe GanChris BrockettBill Dolan
2020-05-01
Probing Text Models for Common Ground with Visual Representations
Gabriel IlharcoRowan ZellersAli FarhadiHannaneh Hajishirzi
2020-05-01
Multilingual Corpus Creation for Multilingual Semantic Similarity Task
Mahtab AhmedChahna DixitRobert E. MercerAtif KhanMuhammad Rifayat SameeFelipe Urra
2020-05-01
Text Categorization for Conflict Event Annotation
Fredrik OlssonMagnus SahlgrenFehmi ben AbdesslemAriel EkgrenKristine Eck
2020-05-01
TF-IDF Character N-grams versus Word Embedding-based Models for Fine-grained Event Classification: A Preliminary Study
Jakub PiskorskiGuillaume Jacquet
2020-05-01
TermEval 2020: TALN-LS2N System for Automatic Term Extraction
Amir HazemBouhM{\'e}rieme iFlorian BoudinBeatrice Daille
2020-05-01
FrameNet Annotations Alignment using Attention-based Machine Translation
Gabriel Marzinotto
2020-05-01
Implementation of Supervised Training Approaches for Monolingual Word Sense Alignment: ACDH-CH System Description for the MWSA Shared Task at GlobaLex 2020
Lenka BajceticSeung-bin Yim
2020-05-01
Transfer learning applied to text classification in Spanish radiological reports
Pilar L{\'o}pez {\'U}bedaManuel Carlos D{\'\i}az-GalianoL. Alfonso Urena LopezMaite MartinTeodoro Mart{\'\i}n-NoguerolAntonio Luna
2020-05-01
Aggression Identification in Social Media: a Transfer Learning Based Approach
RamiFaneva risoaJosiane Mothe
2020-05-01
IRIT at TRAC 2020
RamiFaneva risoaJosiane Mothe
2020-05-01
Bagging BERT Models for Robust Aggression Identification
Julian RischRalf Krestel
2020-05-01
Scmhl5 at TRAC-2 Shared Task on Aggression Identification: Bert Based Ensemble Learning Approach
Han LiuPete BurnapWafa AlorainyMatthew Williams
2020-05-01
Aggression Identification in English, Hindi and Bangla Text using BERT, RoBERTa and SVM
| Arup BaruahKaushik DasFerdous BarbhuiyaKuntal Dey
2020-05-01
Aggression and Misogyny Detection using BERT: A Multi-Task Approach
| Niloofar Safi SamghabadiParth PatwaSrinivas PYKLPrerana MukherjeeAmitava DasThamar Solorio
2020-05-01
From Web Crawl to Clean Register-Annotated Corpora
Veronika LaippalaSamuel R{\"o}nnqvistSaara Hellstr{\"o}mJuhani LuotolahtiLiina RepoAnna SalmelaValtteri SkantsiSampo Pyysalo
2020-05-01
Cross-lingual Zero Pronoun Resolution
Abdulrahman AlorainiMassimo Poesio
2020-05-01
Understanding User Utterances in a Dialog System for Caregiving
Yoshihiko AsaoJulien KloetzerJunta MizunoDai SaikiKazuma KadowakiKentaro Torisawa
2020-05-01
Joint Learning of Syntactic Features Helps Discourse Segmentation
Takshak DesaiParag Pravin DakleDan Moldovan
2020-05-01
Adapting BERT to Implicit Discourse Relation Classification with a Focus on Discourse Connectives
Yudai KishimotoYugo MurawakiSadao Kurohashi
2020-05-01
Automated Essay Scoring System for Nonnative Japanese Learners
Reo HiraoMio AraiHiroki ShimanakaSatoru KatsumataMamoru Komachi
2020-05-01
Development and Validation of a Corpus for Machine Humor Comprehension
Yuen-Hsien TsengWun-Syuan WuChia-Yueh ChangHsueh-Chih ChenWei-Lun Hsu
2020-05-01
Abusive language in Spanish children and young teenager's conversations: data preparation and short text classification with contextual word embeddings
Marta R. Costa-juss{\`a}Esther Gonz{\'a}lezAsuncion MorenoEudald Cumalat
2020-05-01
An Evaluation Dataset for Identifying Communicative Functions of Sentences in English Scholarly Papers
Kenichi IwatsukiFlorian BoudinAkiko Aizawa
2020-05-01
Evaluation Metrics for Headline Generation Using Deep Pre-Trained Embeddings
Abdul MoeedYang AnGerhard HagererGeorg Groh
2020-05-01
SiBert: Enhanced Chinese Pre-trained Language Model with Sentence Insertion
Jiahao ChenChenjie CaoXiuyan Jiang
2020-05-01
Adaptation of Deep Bidirectional Transformers for Afrikaans Language
Sello Ralethe
2020-05-01
Massive vs. Curated Embeddings for Low-Resourced Languages: the Case of Yor\`ub\'a and Twi
Jesujoba AlabiKwabena Amponsah-KaakyireDavid AdelaniCristina Espa{\~n}a-Bonet
2020-05-01
Building a Task-oriented Dialog System for Languages with no Training Data: the Case for Basque
Maddalen L{\'o}pez de LacalleXabier SaralegiI{\~n}aki San Vicente
2020-05-01
Introducing a Large-Scale Dataset for Vietnamese POS Tagging on Conversational Texts
Oanh TranTu PhamVu DangBang Nguyen
2020-05-01
DaNE: A Named Entity Resource for Danish
Rasmus HvingelbyAmalie Brogaard PauliMaria BarrettChristina RostedLasse Malm LidegaardAnders S{\o}gaard
2020-05-01
Is Language Modeling Enough? Evaluating Effective Embedding Combinations
Rudolf SchneiderTom OberhauserPaul GrundmannFelix Alex GerserAlex LoesererSteffen Staab
2020-05-01
Parsing as Tagging
Robert VacareanuGeorge Caique Gouveia BarbosaMarco A. Valenzuela-Esc{\'a}rcegaMihai Surdeanu
2020-05-01
AIA-BDE: A Corpus of FAQs in Portuguese and their Variations
Hugo Gon{\c{c}}alo OliveiraJo{\~a}o FerreiraJos{\'e} SantosPedro FialhoRicardo RodriguesLuisa CoheurAna Alves
2020-05-01
Cross-lingual and Cross-domain Evaluation of Machine Reading Comprehension with Squad and CALOR-Quest Corpora
Delphine CharletGeraldine DamnatiFrederic Bechetgabriel marzinottoJohannes Heinecke
2020-05-01
Contextualized Embeddings based Transformer Encoder for Sentence Similarity Modeling in Answer Selection Task
| Md Tahmid Rahman LaskarJimmy Xiangji HuangEnamul Hoque
2020-05-01
One Classifier for All Ambiguous Words: Overcoming Data Sparsity by Utilizing Sense Correlations Across Words
Prafulla Kumar ChoubeyRuihong Huang
2020-05-01
A Summarization Dataset of Slovak News Articles
| Marek SuppaJergus Adamec
2020-05-01
KLEJ: Comprehensive Benchmark for Polish Language Understanding
| Piotr RybakRobert MroczkowskiJanusz TraczIreneusz Gawlik
2020-05-01
Analyzing ELMo and DistilBERT on Socio-political News Classification
Berfu B{\"u}y{\"u}k{\"o}zAli H{\"u}rriyeto{\u{g}}luArzucan {\"O}zg{\"u}r
2020-05-01
SciREX: A Challenge Dataset for Document-Level Information Extraction
| Sarthak JainMadeleine van ZuylenHannaneh HajishirziIz Beltagy
2020-05-01
PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking
Hannah RashkinAsli CelikyilmazYejin ChoiJianfeng Gao
2020-04-30
WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context
Anna BreitArtem RevenkoKiamehr RezaeeMohammad Taher PilehvarJose Camacho-Collados
2020-04-30
On the Evaluation of Contextual Embeddings for Zero-Shot Cross-Lingual Transfer Learning
Phillip KeungYichao LuJulian SalazarVikas Bhardwaj
2020-04-30
A Matter of Framing: The Impact of Linguistic Formalism on Probing Results
Ilia KuznetsovIryna Gurevych
2020-04-30
SegaBERT: Pre-training of Segment-aware BERT for Language Understanding
He BaiPeng ShiJimmy LinLuchen TanKun XiongWen GaoMing Li
2020-04-30
How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking
| Nicola De CaoMichael SchlichtkrullWilker AzizIvan Titov
2020-04-30
Investigating Transferability in Pretrained Language Models
Alex TamkinTrisha SinghDavide GiovanardiNoah Goodman
2020-04-30
Enriched Pre-trained Transformers for Joint Slot Filling and Intent Detection
Momchil HardalovIvan KoychevPreslav Nakov
2020-04-30
Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT
Zhiyong WuYun ChenBen KaoQun Liu
2020-04-30
Robust Question Answering Through Sub-part Alignment
Jifan ChenGreg Durrett
2020-04-30
Modular Representation Underlies Systematic Generalization in Neural Natural Language Inference Models
Atticus GeigerKyle RichardsonChristopher Potts
2020-04-30
Universal Dependencies according to BERT: both more specific and more general
| Tomasz LimisiewiczRudolf RosaDavid Mareček
2020-04-30
Look at the First Sentence: Position Bias in Question Answering
Miyoung KoJinhyuk LeeHyunjae KimGangwoo KimJaewoo Kang
2020-04-30
Exploring Contextualized Neural Language Models for Temporal Dependency Parsing
Hayley RossJonathan CaiBonan Min
2020-04-30
Interpretable Entity Representations through Large-Scale Typing
Yasumasa OnoeGreg Durrett
2020-04-30
MAD-X: An Adapter-based Framework for Multi-task Cross-lingual Transfer
Jonas PfeifferIvan VulićIryna GurevychSebastian Ruder
2020-04-30
GePpeTto Carves Italian into a Language Model
| Lorenzo De MatteiMichele CafagnaFelice Dell'OrlettaMalvina NissimMarco Guerini
2020-04-29
End-to-End Slot Alignment and Recognition for Cross-Lingual NLU
Weijia XuBatool HaiderSaab Mansour
2020-04-29
Detecting Perceived Emotions in Hurricane Disasters
Shrey DesaiCornelia CarageaJunyi Jessy Li
2020-04-29
Training Curricula for Open Domain Answer Re-Ranking
| Sean MacAvaneyFranco Maria NardiniRaffaele PeregoNicola TonellottoNazli GoharianOphir Frieder
2020-04-29
Analysing Lexical Semantic Change with Contextualised Word Representations
Mario GiulianelliMarco Del TrediciRaquel Fernández
2020-04-29
Do Neural Language Models Show Preferences for Syntactic Formalisms?
Artur KulmizevVinit RavishankarMostafa AbdouJoakim Nivre
2020-04-29
BURT: BERT-inspired Universal Representation from Twin Structure
Yian LiHai Zhao
2020-04-29
Revisiting Pre-Trained Models for Chinese Natural Language Processing
| Yiming CuiWanxiang CheTing LiuBing QinShijin WangGuoping Hu
2020-04-29
Bilingual Text Extraction as Reading Comprehension
Katsuki ChousaMasaaki NagataMasaaki Nishino
2020-04-29
What Happens To BERT Embeddings During Fine-tuning?
Amil MerchantElahe RahimtoroghiEllie PavlickIan Tenney
2020-04-29
Distantly-Supervised Neural Relation Extraction with Side Information using BERT
| Johny MoreiraChaina OliveiraDavid MacêdoCleber ZanchettinLuciano Barbosa
2020-04-29
A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERT
Masaaki NagataChousa KatsukiMasaaki Nishino
2020-04-29
Asking without Telling: Exploring Latent Ontologies in Contextual Representations
Julian MichaelJan A. BothaIan Tenney
2020-04-29
TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP
| John X. MorrisEli LiflandJin Yong YooJake GrigsbyDi JinYanjun Qi
2020-04-29
Extending Multilingual BERT to Low-Resource Languages
Zihan WangKarthikeyan KStephen MayhewDan Roth
2020-04-28
Joint Keyphrase Chunking and Salience Ranking with BERT
| Si SunChenyan XiongZhenghao LiuZhiyuan LiuJie Bao
2020-04-28
EARL: Speedup Transformer-based Rankers with Pre-computed Representation
Luyu GaoZhuyun DaiJamie Callan
2020-04-28
VD-BERT: A Unified Vision and Dialog Transformer with BERT
Yue WangShafiq JotyMichael R. LyuIrwin KingCaiming XiongSteven C. H. Hoi
2020-04-28
DomBERT: Domain-oriented Language Model for Aspect-based Sentiment Analysis
Hu XuBing LiuLei ShuPhilip S. Yu
2020-04-28
Kungfupanda at SemEval-2020 Task 12: BERT-Based Multi-Task Learning for Offensive Language Detection
| Wenliang DaiTiezheng YuZihan LiuPascale Fung
2020-04-28
LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning
Kaitao SongHao SunXu TanTao QinJianfeng LuHongzhi LiuTie-Yan Liu
2020-04-27
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference
| Ji XinRaphael TangJaejun LeeYaoliang YuJimmy Lin
2020-04-27
ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Omar KhattabMatei Zaharia
2020-04-27
ColBERT: Using BERT Sentence Embedding for Humor Detection
| Issa Annamoradnejad
2020-04-27
On the Importance of Word and Sentence Representation Learning in Implicit Discourse Relation Classification
| Xin LiuJiefu OuYangqiu SongXin Jiang
2020-04-27
Assessing Discourse Relations in Language Generation from Pre-trained Language Models
Wei-Jen KoJunyi Jessy Li
2020-04-26
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models
Mengjie ZhaoTao LinMartin JaggiHinrich Schütze
2020-04-26
Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Document Matching
Liu YangMingyang ZhangCheng LiMichael BenderskyMarc Najork
2020-04-26
Classification of Cuisines from Sequentially Structured Recipes
Tript SharmaUtkarsh UpadhyayGanesh Bagler
2020-04-26
Challenge Closed-book Science Exam: A Meta-learning Based Question Answering System
Xinyue ZhengPeng WangQigang WangZhongchao Shi
2020-04-26
SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check
| Xingyi ChengWeidi XuKunlong ChenShaohua JiangFeng WangTaifeng WangWei ChuYuan Qi
2020-04-26
Quantifying the Contextualization of Word Representations with Semantic Class Probing
Mengjie ZhaoPhilipp DufterYadollah YaghoobzadehHinrich Schütze
2020-04-25
Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order
Yi LiaoXin JiangQun Liu
2020-04-24
A Tailored Pre-Training Model for Task-Oriented Dialog Generation
Jing GuQingyang WuChongruo WuWeiyan ShiZhou Yu
2020-04-24
Contextualized Representations Using Textual Encyclopedic Knowledge
Mandar JoshiKenton LeeYi LuanKristina Toutanova
2020-04-24
Syntactic Data Augmentation Increases Robustness to Inference Heuristics
Junghyun MinR. Thomas McCoyDipanjan DasEmily PitlerTal Linzen
2020-04-24
The Inception Team at NSURL-2019 Task 8: Semantic Question Similarity in Arabic
Hana Al-TheiabatAisha Al-Sadi
2020-04-24
Cross-lingual Information Retrieval with BERT
Zhuolin JiangAmro El-JaroudiWilliam HartmannDamianos KarakosLingjun Zhao
2020-04-24
Data Annealing for Informal Language Understanding Tasks
Jing GuZhou Yu
2020-04-24
PBCS : Efficient Exploration and Exploitation Using a Synergy between Reinforcement Learning and Motion Planning
Guillaume MatheronNicolas PerrinOlivier Sigaud
2020-04-24
Collecting Entailment Data for Pretraining: New Protocols and Negative Results
| Samuel R. BowmanJennimaria PalomakiLivio Baldini SoaresEmily Pitler
2020-04-24
On Adversarial Examples for Biomedical NLP Tasks
Vladimir AraujoAndres CarvalloCarlos AspillagaDenis Parra
2020-04-23
Same Side Stance Classification Task: Facilitating Argument Stance Classification by Fine-tuning a BERT Model
Stefan OllingerLorik DumaniPremtim SahitajRalph BergmannRalf Schenkel
2020-04-23
Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
Yaru HaoLi DongFuru WeiKe Xu
2020-04-23
UHH-LT at SemEval-2020 Task 12: Fine-Tuning of Pre-Trained Transformer Networks for Offensive Language Detection
Gregor WiedemannSeid Muhie YimamChris Biemann
2020-04-23
YOLOv4: Optimal Speed and Accuracy of Object Detection
| Alexey BochkovskiyChien-Yao WangHong-Yuan Mark Liao
2020-04-23
Keyphrase Prediction With Pre-trained Language Model
Rui LiuZheng LinWeiping Wang
2020-04-22
Learning to Classify Intents and Slot Labels Given a Handful of Examples
Jason KroneYi ZhangMona Diab
2020-04-22
Residual Energy-Based Models for Text Generation
Yuntian DengAnton BakhtinMyle OttArthur SzlamMarc'Aurelio Ranzato
2020-04-22
Mirror Ritual: An Affective Interface for Emotional Self-Reflection
Nina RajcicJon McCormack
2020-04-21
Attention Module is Not Only a Weight: Analyzing Transformers with Vector Norms
Goro KobayashiTatsuki KuribayashiSho YokoiKentaro Inui
2020-04-21
BERT-ATTACK: Adversarial Attack Against BERT Using BERT
Linyang LiRuotian MaQipeng GuoXiangyang XueXipeng Qiu
2020-04-21
DIET: Lightweight Language Understanding for Dialogue Systems
| Tanja BunkDaksh VarshneyaVladimir VlasovAlan Nichol
2020-04-21
Domain-Guided Task Decomposition with Self-Training for Detecting Personal Events in Social Media
Payam KarisaniJoyce C. HoEugene Agichtein
2020-04-21
Investigating the Effectiveness of Representations Based on Pretrained Transformer-based Language Models in Active Learning for Labelling Text Datasets
Jinghui LuBrian MacNamee
2020-04-21
MPNet: Masked and Permuted Pre-training for Language Understanding
| Kaitao SongXu TanTao QinJianfeng LuTie-Yan Liu
2020-04-20
A Study of Cross-Lingual Ability and Language-specific Information in Multilingual BERT
Chi-Liang LiuTsung-Yuan HsuYung-Sung ChuangHung-Yi Lee
2020-04-20
CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT
| Akshay SmitSaahil JainPranav RajpurkarAnuj PareekAndrew Y. NgMatthew P. Lungren
2020-04-20
Adversarial Training for Large Neural Language Models
| Xiaodong LiuHao ChengPengcheng HeWeizhu ChenYu WangHoifung PoonJianfeng Gao
2020-04-20
StereoSet: Measuring stereotypical bias in pretrained language models
| Moin NadeemAnna BethkeSiva Reddy
2020-04-20
ResNeSt: Split-Attention Networks
| Hang ZhangChongruo WuZhongyue ZhangYi ZhuZhi ZhangHaibin LinYue SunTong HeJonas MuellerR. ManmathaMu LiAlexander Smola
2020-04-19
Enhancing Pharmacovigilance with Drug Reviews and Social Media
| Brent BisedaKatie Mo
2020-04-18
Too Many Claims to Fact-Check: Prioritizing Political Claims Based on Check-Worthiness
Yavuz Selim KartalBusra GuvenenMucahid Kutlu
2020-04-17
Fast and Accurate Deep Bidirectional Language Representations for Unsupervised Learning
| Joongbo ShinYoonhyung LeeSeunghyun YoonKyomin Jung
2020-04-17
Learning-to-Rank with BERT in TF-Ranking
Shuguang HanXuanhui WangMike BenderskyMarc Najork
2020-04-17
The Right Tool for the Job: Matching Model and Instance Complexities
| Roy SchwartzGabriel StanovskySwabha SwayamdiptaJesse DodgeNoah A. Smith
2020-04-16
SPECTER: Document-level Representation Learning using Citation-informed Transformers
| Arman CohanSergey FeldmanIz BeltagyDoug DowneyDaniel S. Weld
2020-04-15
lamBERT: Language and Action Learning Using Multimodal BERT
Kazuki MiyazawaTatsuya AokiTakato HoriiTakayuki Nagai
2020-04-15
ToD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogues
| Chien-Sheng WuSteven HoiRichard SocherCaiming Xiong
2020-04-15
Coreferential Reasoning Learning for Language Representation
| Deming YeYankai LinJiaju DuZhenghao LiuMaosong SunZhiyuan Liu
2020-04-15
Training with Quantization Noise for Extreme Model Compression
| Angela FanPierre StockBenjamin GrahamEdouard GraveRemi GribonvalHerve JegouArmand Joulin
2020-04-15
Sentiment Analysis of Yelp Reviews: A Comparison of Techniques and Models
Siqi Liu
2020-04-15
What's so special about BERT's layers? A closer look at the NLP pipeline in monolingual and multilingual models
| Wietse de VriesAndreas van CranenburghMalvina Nissim
2020-04-14
Deep Learning Models for Multilingual Hate Speech Detection
| Sai Saketh AluruBinny MathewPunyajoy SahaAnimesh Mukherjee
2020-04-14
Standardizing and Benchmarking Crisis-related Social Media Datasets for Humanitarian Information Processing
Firoj AlamHassan SajjadMuhammad ImranFerda Ofli
2020-04-14
A Simple Yet Strong Pipeline for HotpotQA
Dirk GroeneveldTushar KhotMausamAshish Sabharwal
2020-04-14
PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation
Bin BiChenliang LiChen WuMing YanWei Wang
2020-04-14
Pretrained Transformers Improve Out-of-Distribution Robustness
Dan HendrycksXiaoyuan LiuEric WallaceAdam DziedzicRishabh KrishnanDawn Song
2020-04-13
Unified Multi-Criteria Chinese Word Segmentation with BERT
Zhen KeLiang ShiErli MengBin WangXipeng QiuXuanjing Huang
2020-04-13
ProFormer: Towards On-Device LSH Projection Based Transformers
Chinnadhurai SankarSujith RaviZornitsa Kozareva
2020-04-13
Cascade Neural Ensemble for Identifying Scientifically Sound Articles
Ashwin Karthik AmbalavananMurthy Devarakonda
2020-04-13
Robustly Pre-trained Neural Model for Direct Temporal Relation Extraction
Hong GuanJianfu LiHua XuMurthy Devarakonda
2020-04-13
Improving Scholarly Knowledge Representation: Evaluating BERT-based Models for Scientific Relation Classification
Ming JiangJennifer D'SouzaSören AuerJ. Stephen Downie
2020-04-13
VGCN-BERT: Augmenting BERT with Graph Embedding for Text Classification
| Zhibin LuPan DuJian-Yun Nie
2020-04-12
Pre-training Text Representations as Meta Learning
Shangwen LvYuechen WangDaya GuoDuyu TangNan DuanFuqing ZhuMing GongLinjun ShouRyan MaDaxin JiangGuihong CaoMing ZhouSonglin Hu
2020-04-12
AMR Parsing via Graph-Sequence Iterative Inference
Deng CaiWai Lam
2020-04-12
LAReQA: Language-agnostic answer retrieval from a multilingual pool
Uma RoyNoah ConstantRami Al-RfouAditya BaruaAaron PhillipsYinfei Yang
2020-04-11
End to End Chinese Lexical Fusion Recognition with Sememe Knowledge
Yijiang LiuMeishan ZhangDonghong Ji
2020-04-11
Longformer: The Long-Document Transformer
| Iz BeltagyMatthew E. PetersArman Cohan
2020-04-10
SimpleTran: Transferring Pre-Trained Sentence Embeddings for Low Resource Text Classification
Siddhant GargRohit Kumar SharmaYingyu Liang
2020-04-10
An In-depth Walkthrough on Evolution of Neural Machine Translation
Rohan JagtapDr. Sudhir N. Dhage
2020-04-10
Telling BERT's full story: from Local Attention to Global Aggregation
Damian PascualGino BrunnerRoger Wattenhofer
2020-04-10
BLEURT: Learning Robust Metrics for Text Generation
Thibault SellamDipanjan DasAnkur P. Parikh
2020-04-09
On the Language Neutrality of Pre-trained Multilingual Representations
Jindřich LibovickýRudolf RosaAlexander Fraser
2020-04-09
Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improve
Oshin AgarwalYinfei YangByron C. WallaceAni Nenkova
2020-04-09
Generating Counter Narratives against Online Hate Speech: Data and Strategies
Serra Sinem TekirogluYi-Ling ChungMarco Guerini
2020-04-08
LadaBERT: Lightweight Adaptation of BERT through Hybrid Model Compression
Yihuan MaoYujing WangChufan WuChen ZhangYang WangYaming YangQuanlu ZhangYunhai TongJing Bai
2020-04-08
DynaBERT: Dynamic BERT with Adaptive Width and Depth
Lu HouLifeng ShangXin JiangQun Liu
2020-04-08
Exploiting Redundancy in Pre-trained Language Models for Efficient Transfer Learning
Fahim DalviHassan SajjadNadir DurraniYonatan Belinkov
2020-04-08
Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence
| Federico BianchiSilvia TerragniDirk Hovy
2020-04-08
Poor Man's BERT: Smaller and Faster Transformer Models
| Hassan SajjadFahim DalviNadir DurraniPreslav Nakov
2020-04-08
Improving BERT with Self-Supervised Attention
Xiaoyu KouYaming YangYujing WangCe ZhangYiren ChenYunhai TongYan ZhangJing Bai
2020-04-08
DialBERT: A Hierarchical Pre-Trained Model for Conversation Disentanglement
Tianda LiJia-Chen GuXiaodan ZhuQuan LiuZhen-Hua LingZhiming SuSi Wei
2020-04-08
Severing the Edge Between Before and After: Neural Architectures for Temporal Ordering of Events
Miguel BallesterosRishita AnubhaiShuai WangNima PourdamghaniYogarshi VyasJie MaParminder BhatiaKathleen McKeownYaser Al-Onaizan
2020-04-08
Error-correction and extraction in request dialogs
Stefan ConstantinAlex Waibel
2020-04-08
SciWING -- A Software Toolkit for Scientific Document Processing
| Abhinav Ramesh KashyapMin-Yen Kan
2020-04-08
Have Your Text and Use It Too! End-to-End Neural Data-to-Text Generation with Semantic Fidelity
Hamza HarkousIsabel GrovesAmir Saffari
2020-04-08
Transformers to Learn Hierarchical Contexts in Multiparty Dialogue for Span-based Question Answering
| Changmao LiJinho D. Choi
2020-04-07
Towards Non-task-specific Distillation of BERT via Sentence Representation Approximation
Bowen WuHuan ZhangMengyuan LiZongsheng WangQihang FengJunhong HuangBaoxun Wang
2020-04-07
Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition
Paloma JereticAlex WarstadtSuvrat BhooshanAdina Williams
2020-04-07
Information-Theoretic Probing for Linguistic Structure
| Tiago PimentelJosef ValvodaRowan Hall MaudslayRan ZmigrodAdina WilliamsRyan Cotterell
2020-04-07
Towards Evaluating the Robustness of Chinese BERT Classifiers
Boxin WangBoyuan PanXin LiBo Li
2020-04-07
The Russian Drug Reaction Corpus and Neural Models for Drug Reactions and Effectiveness Detection in User Reviews
| Elena TutubalinaIlseyar AlimovaZulfat MiftahutdinovAndrey SakhovskiyValentin MalykhSergey Nikolenko
2020-04-07
Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots
| Jia-Chen GuTianda LiQuan LiuZhen-Hua LingZhiming SuSi WeiXiaodan Zhu
2020-04-07
TextGAIL: Generative Adversarial Imitation Learning for Text Generation
Qingyang WuLei LiZhou Yu
2020-04-07
RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databases
DongHyun ChoiMyeong Cheol ShinEungGyun KimDong Ryeol Shin
2020-04-07
DARE: Data Augmented Relation Extraction with GPT-2
Yannis PapanikolaouAndrea Pierleoni
2020-04-06
Leveraging the Inherent Hierarchy of Vacancy Titles for Automated Job Ontology Expansion
Jeroen Van HautteVincent SchelstraeteMikaël Wornoo
2020-04-06
Enhancing Review Comprehension with Domain-Specific Commonsense
Aaron TraylorChen ChenBehzad GolshanXiaolan WangYuliang LiYoshihiko SuharaJinfeng LiCagatay DemiralpWang-Chiew Tan
2020-04-06
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
Zhiqing SunHongkun YuXiaodan SongRenjie LiuYiming YangDenny Zhou
2020-04-06
Sparse Text Generation
Pedro Henrique MartinsZita MarinhoAndré F. T. Martins
2020-04-06
Bootstrapping a Crosslingual Semantic Parser
Tom SherborneYumo XuMirella Lapata
2020-04-06
FastBERT: a Self-distilling BERT with Adaptive Inference Time
| Weijie LiuPeng ZhouZhe ZhaoZhiruo WangHaotang DengQi Ju
2020-04-05
Improved Pretraining for Domain-specific Contextual Embedding Models
Subendhu RongaliAbhyuday JagannathaBhanu Pratap Singh RawatHong Yu
2020-04-05
Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space
| Chunyuan LiXiang GaoYuan LiXiujun LiBaolin PengYizhe ZhangJianfeng Gao
2020-04-05
Generating Rationales in Visual Question Answering
Hammad A. AyyubiMd. Mehrab TanjimJulian J. McAuleyGarrison W. Cottrell
2020-04-04
A Dependency Syntactic Knowledge Augmented Interactive Architecture for End-to-End Aspect-based Sentiment Analysis
| Yunlong LiangFandong MengJinchao ZhangJinan XuYufeng ChenJie Zhou
2020-04-04
CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection
Congying XiaChenwei ZhangHoang NguyenJiawei ZhangPhilip Yu
2020-04-04
Model-based actor-critic: GAN + DRL (actor-critic) => AGI
Aras Dargazany
2020-04-04
Finding Black Cat in a Coal Cellar -- Keyphrase Extraction & Keyphrase-Rubric Relationship Classification from Complex Assignments
| Manikandan Ravikiran
2020-04-03
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
| Yaobo LiangNan DuanYeyun GongNing WuFenfei GuoWeizhen QiMing GongLinjun ShouDaxin JiangGuihong CaoXiaodong FanRuofei ZhangRahul AgrawalEdward CuiSining WeiTaroon BhartiYing QiaoJiun-Hung ChenWinnie WuShuguang LiuFan YangDaniel CamposRangan MajumderMing Zhou
2020-04-03
Testing pre-trained Transformer models for Lithuanian news clustering
Lukas StankevičiusMantas Lukoševičius
2020-04-03
Gestalt: a Stacking Ensemble for SQuAD2.0
Mohamed El-Geish
2020-04-02
Deep Entity Matching with Pre-Trained Language Models
Yuliang LiJinfeng LiYoshihiko SuharaAnHai DoanWang-Chiew Tan
2020-04-01
Towards Productionizing Subjective Search Systems
Aaron FengShuwei ChenYuliang LiHiroshi MatsudaHidekazu TamakiWang-Chiew Tan
2020-03-31
Unification-based Reconstruction of Explanations for Science Questions
| Marco ValentinoMokanarangan ThayaparanAndré Freitas
2020-03-31
Give your Text Representation Models some Love: the Case for Basque
Rodrigo AgerriIñaki San VicenteJon Ander CamposAnder BarrenaXabier SaralegiAitor SoroaEneko Agirre
2020-03-31
InterBERT: An Effective Multi-Modal Pretraining Approach via Vision-and-Language Interaction
Junyang LinAn YangYichang ZhangJie LiuJingren ZhouHongxia Yang
2020-03-30
NukeBERT: A Pre-trained language model for Low Resource Nuclear Domain
Ayush JainMeenachi Ganesamoorty
2020-03-30
TResNet: High Performance GPU-Dedicated Architecture
| Tal RidnikHussam LawenAsaf NoyItamar FriedmanEmanuel Ben BaruchGilad Sharir
2020-03-30
Designing Network Design Spaces
| Ilija RadosavovicRaj Prateek KosarajuRoss GirshickKaiming HePiotr Dollár
2020-03-30
Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement
Alireza MohammadshahiJames Henderson
2020-03-29
Abstractive Text Summarization based on Language Model Conditioning and Locality Modeling
Dmitrii AksenovJulián Moreno-SchneiderPeter BourgonjeRobert SchwarzenbergLeonhard HennigGeorg Rehm
2020-03-29
Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining
Chengyu WangMinghui QiuJun HuangXiaofeng He
2020-03-29
User Generated Data: Achilles' Heel of BERT
Ankit KumarPiyush MakhijaAnuj Gupta
2020-03-29
BERT Fine-tuning For Arabic Text Summarization
| Khalid N. ElmadaniMukhtar ElgezouliAnas Showk
2020-03-29
HIN: Hierarchical Inference Network for Document-Level Relation Extraction
Hengzhu TangYanan CaoZhenyu ZhangJiangxia CaoFang FangShi WangPengfei Yin
2020-03-28
Obstacle Avoidance and Navigation Utilizing Reinforcement Learning with Reward Shaping
Daniel ZhangColleen P. Bailey
2020-03-28
Cycle Text-To-Image GAN with BERT
| Trevor TsueSamir SenJason Li
2020-03-26
Hit-Detector: Hierarchical Trinity Architecture Search for Object Detection
| Jianyuan GuoKai HanYunhe WangChao ZhangZhaohui YangHan WuXinghao ChenChang Xu
2020-03-26
Volumization as a Natural Generalization of Weight Decay
Liu ZiyinZihao WangMakoto YamadaMasahito Ueda
2020-03-25
Circumventing Outliers of AutoAugment with Knowledge Distillation
Longhui WeiAn XiaoLingxi XieXin ChenXiaopeng ZhangQi Tian
2020-03-25
Multi-Agent Reinforcement Learning for Problems with Combined Individual and Team Reward
Hassam Ullah SheikhLadislau Bölöni
2020-03-24
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
| Kevin ClarkMinh-Thang LuongQuoc V. LeChristopher D. Manning
2020-03-23
Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning
| Qian LongZihan ZhouAbhibav GuptaFei FangYi WuXiaolong Wang
2020-03-23
Pairwise Multi-Class Document Classification for Semantic Relations between Wikipedia Articles
| Malte OstendorffTerry RuasMoritz SchubotzGeorg RehmBela Gipp
2020-03-22
Accelerating Deep Reinforcement Learning With the Aid of a Partial Model: Power-Efficient Predictive Video Streaming
Dong LiuJianyu ZhaoChenyang YangLajos Hanzo
2020-03-21
Beheshti-NER: Persian Named Entity Recognition Using BERT
| Ehsan TaherSeyed Abbas HoseiniMehrnoush Shamsfard
2020-03-19
Temporal Embeddings and Transformer Models for Narrative Text Understanding
| Vani KSimone MellaceAlessandro Antonucci
2020-03-19
The value of text for small business default prediction: A deep learning approach
Matthew StevensonChristophe MuesCristián Bravo
2020-03-19
Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations
| Huan ZhangHongge ChenChaowei XiaoBo LiMingyan LiuDuane BoningCho-Jui Hsieh
2020-03-19
Diversity, Density, and Homogeneity: Quantitative Characteristic Metrics for Text Collections
Yi-An LaiXuan ZhuYi ZhangMona Diab
2020-03-19
X-Stance: A Multilingual Multi-Target Dataset for Stance Detection
| Jannis VamvasRico Sennrich
2020-03-18
Calibration of Pre-trained Transformers
| Shrey DesaiGreg Durrett
2020-03-17
Author2Vec: A Framework for Generating User Embedding
Xiaodong WuWeizhe LinZhilin WangElena Rastorgueva
2020-03-17
Revisiting the Sibling Head in Object Detector
| Guanglu SongYu LiuXiaogang Wang
2020-03-17
PO-EMO: Conceptualization, Annotation, and Modeling of Aesthetic Emotions in German and English Poetry
| Thomas HaiderSteffen EgerEvgeny KimRoman KlingerWinfried Menninghaus
2020-03-17
TRANS-BLSTM: Transformer with Bidirectional LSTM for Language Understanding
Zhiheng HuangPeng XuDavis LiangAjay MishraBing Xiang
2020-03-16
Cost-Sensitive BERT for Generalisable Sentence Classification with Imbalanced Data
Harish Tayyar MadabushiElena KochkinaMichael Castelle
2020-03-16
A Survey on Contextual Embeddings
| Qi LiuMatt J. KusnerPhil Blunsom
2020-03-16
Finnish Language Modeling with Deep Transformer Models
Abhilash JainAku RuoheStig-Arne GrönroosMikko Kurimo
2020-03-14
Deep Multi-Agent Reinforcement Learning for Decentralized Continuous Cooperative Control
Christian Schroeder de WittBei PengPierre-Alexandre KamiennyPhilip TorrWendelin BöhmerShimon Whiteson
2020-03-14
Document Ranking with a Pretrained Sequence-to-Sequence Model
Rodrigo NogueiraZhiying JiangJimmy Lin
2020-03-14
Generating Major Types of Chinese Classical Poetry in a Uniformed Framework
Jinyi HuMaosong Sun
2020-03-13
Highly Efficient Salient Object Detection with 100K Parameters
Shang-Hua GaoYong-Qiang TanMing-Ming ChengChengze LuYunpeng ChenShuicheng Yan
2020-03-12
Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking
Samuel Broscheit
2020-03-11
Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings
| Haoran ZhangAmy X. LuMohamed AbdallaMatthew McDermottMarzyeh Ghassemi
2020-03-11
Online Meta-Critic Learning for Off-Policy Actor-Critic Methods
Wei ZhouYiying LiYongxin YangHuaimin WangTimothy M. Hospedales
2020-03-11
Keyword-Attentive Deep Semantic Matching
| Changyu MiaoZhen CaoYik-Cheung Tam
2020-03-11
Efficient Intent Detection with Dual Sentence Encoders
| Iñigo CasanuevaTadas TemčinasDaniela GerzMatthew HendersonIvan Vulić
2020-03-10
Sensitive Data Detection and Classification in Spanish Clinical Text: Experiments with BERT
Aitor García-PablosNaiara PerezMontse Cuadros
2020-03-06
Transfer Learning for Information Extraction with Limited Data
Minh-Tien NguyenViet-Anh PhanLe Thai LinhNguyen Hong SonLe Tien DungMiku HiranoHajime Hotta
2020-03-06
RecipeGPT: Generative Pre-training Based Cooking Recipe Generation and Evaluation System
| Helena H. LeeKe ShuPalakorn AchananuparpPhilips Kokoh PrasetyoYue LiuEe-Peng LimLav R. Varshney
2020-03-05
BERT as a Teacher: Contextual Embeddings for Sequence-Level Reward
Florian SchmidtThomas Hofmann
2020-03-05
What the [MASK]? Making Sense of Language-Specific BERT Models
Debora NozzaFederico BianchiDirk Hovy
2020-03-05
HypoNLI: Exploring the Artificial Patterns of Hypothesis-only Bias in Natural Language Inference
Tianyu LiuXin ZhengBaobao ChangZhifang Sui
2020-03-05
jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models
| Yada PruksachatkunPhil YeresHaokun LiuJason PhangPhu Mon HtutAlex WangIan TenneySamuel R. Bowman
2020-03-04
Data Augmentation using Pre-trained Transformer Models
| Varun KumarAshutosh ChoudharyEunah Cho
2020-03-04
Dynamic Experience Replay
Jieliang LuoHui Li
2020-03-04
Kleister: A novel task for Information Extraction involving Long Documents with Complex Layout
Filip GralińskiTomasz StanisławekAnna WróblewskaDawid LipińskiAgnieszka KaliskaPaulina RosalskaBartosz TopolskiPrzemysław Biecek
2020-03-04
A Study on Efficiency, Accuracy and Document Structure for Answer Sentence Selection
Daniele BonadimanAlessandro Moschitti
2020-03-04
Hybrid Generative-Retrieval Transformers for Dialogue Domain Adaptation
Igor ShalyminovAlessandro SordoniAdam AtkinsonHannes Schulz
2020-03-03
CLUECorpus2020: A Large-scale Chinese Corpus for Pre-training Language Model
| Liang XuXuanwei ZhangQianqian Dong
2020-03-03
Hierarchical Context Enhanced Multi-Domain Dialogue System for Multi-domain Task Completion
Jingyuan YangGuang LiuYuzhao MaoZhiwei ZhaoWeiguo GaoXuan LiHaiqin YangJianping Shen
2020-03-03
Reinforcement co-Learning of Deep and Spiking Neural Networks for Energy-Efficient Mapless Navigation with Neuromorphic Hardware
Guangzhi TangNeelesh KumarKonstantinos P. Michmizos
2020-03-02
Gadam: Combining Adaptivity with Iterate Averaging Gives Greater Generalisation
| Diego GranziolXingchen WanStephen Roberts
2020-03-02
TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing
| Ziqing YangYiming CuiZhipeng ChenWanxiang CheTing LiuShijin WangGuoping Hu
2020-02-28
DC-BERT: Decoupling Question and Document for Efficient Contextual Encoding
Yuyu ZhangPing NieXiubo GengArun RamamurthyLe SongDaxin Jiang
2020-02-28
AraBERT: Transformer-based Model for Arabic Language Understanding
| Wissam AntounFady BalyHazem Hajj
2020-02-28
A Primer in BERTology: What we know about how BERT works
Anna RogersOlga KovalevaAnna Rumshisky
2020-02-27
Compressing Large-Scale Transformer-Based Models: A Case Study on BERT
Prakhar GaneshYao ChenXin LouMohammad Ali KhanYin YangDeming ChenMarianne WinslettHassan SajjadPreslav Nakov
2020-02-27
Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT
Lichao SunKazuma HashimotoWenpeng YinAkari AsaiJia LiPhilip YuCaiming Xiong
2020-02-27
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
| Wenhui WangFuru WeiLi DongHangbo BaoNan YangMing Zhou
2020-02-25
BERT Can See Out of the Box: On the Cross-modal Transferability of Text Representations
Thomas ScialomPatrick BordesPaul-Alexis DrayJacopo StaianoPatrick Gallinari
2020-02-25
Exploring BERT Parameter Efficiency on the Stanford Question Answering Dataset v2.0
Eric Hulburd
2020-02-25
Improving BERT Fine-Tuning via Self-Ensemble and Self-Distillation
| Yige XuXipeng QiuLigao ZhouXuanjing Huang
2020-02-24
Predicting Subjective Features from Questions on QA Websites using BERT
| Issa AnnamoradnejadMohammadamin FazliJafar Habibi
2020-02-24
Predicting Subjective Features from Questions on QA Websites using BERT
| Issa AnnamoradnejadMohammadamin FazliJafar Habibi
2020-02-24
Training Question Answering Models From Synthetic Data
Raul PuriRyan SpringMostofa PatwaryMohammad ShoeybiBryan Catanzaro
2020-02-22
Federated pretraining and fine tuning of BERT using clinical notes from multiple silos
Dianbo LiuTim Miller
2020-02-20
MaxUp: A Simple Way to Improve Generalization of Neural Network Training
Chengyue GongTongzheng RenMao YeQiang Liu
2020-02-20
Improving Generalization by Controlling Label-Noise Information in Neural Network Weights
| Hrayr HarutyunyanKyle ReingGreg Ver SteegAram Galstyan
2020-02-19
Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learning
Mitchell A. GordonKevin DuhNicholas Andrews
2020-02-19
The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding
| Xiaodong LiuYu WangJianshu JiHao ChengXueyun ZhuEmmanuel AwaPengcheng HeWeizhu ChenHoifung PoonGuihong CaoJianfeng Gao
2020-02-19
From English To Foreign Languages: Transferring Pre-trained Language Models
Ke Tran
2020-02-18
Residual Continual Learning
Janghyeon LeeDonggyu JooHyeong Gwon HongJunmo Kim
2020-02-17
Incorporating BERT into Neural Machine Translation
| Jinhua ZhuYingce XiaLijun WuDi HeTao QinWengang ZhouHouqiang LiTie-Yan Liu
2020-02-17
A Financial Service Chatbot based on Deep Bidirectional Transformers
Shi YuYuxin ChenHussain Zaidi
2020-02-17
The Utility of General Domain Transfer Learning for Medical Language Tasks
Daniel RantiKatie HanssShan ZhaoVarun ArvindJoseph TitanoAnthony CostaEric Oermann
2020-02-16
SBERT-WK: A Sentence Embedding Method by Dissecting BERT-based Word Models
| Bin WangC. -C. Jay Kuo
2020-02-16
UniViLM: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation
Huaishao LuoLei JiBotian ShiHaoyang HuangNan DuanTaroon BhartiTianrui LiMing ZhouXilin Chen
2020-02-15
Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping
| Jesse DodgeGabriel IlharcoRoy SchwartzAli FarhadiHannaneh HajishirziNoah Smith
2020-02-15
Transformer on a Diet
| Chenguang WangZihao YeAston ZhangZheng ZhangAlexander J. Smola
2020-02-14
Understanding patient complaint characteristics using contextual clinical BERT embeddings
Budhaditya SahaSanal LisboaShameek Ghosh
2020-02-14
TwinBERT: Distilling Knowledge to Twin-Structured BERT Models for Efficient Retrieval
| Wenhao LuJian JiaoRuofei Zhang
2020-02-14
Stress Test Evaluation of Transformer-based Models in Natural Language Understanding Tasks
Carlos AspillagaAndrés CarvalloVladimir Araujo
2020-02-14
Training Large Neural Networks with Constant Memory using a New Execution Algorithm
Bharadwaj PudipeddiMaral MesmakhosroshahiJinwen XiSujeeth Bharadwaj
2020-02-13
CBAG: Conditional Biomedical Abstract Generation
Justin SybrandtIlya Safro
2020-02-13
A Simple Framework for Contrastive Learning of Visual Representations
| Ting ChenSimon KornblithMohammad NorouziGeoffrey Hinton
2020-02-13
Utilizing BERT Intermediate Layers for Aspect Based Sentiment Analysis and Natural Language Inference
Youwei SongJiahai WangZhiwei LiangZhiyue LiuTao Jiang
2020-02-12
Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Models
Wangchunshu ZhouKe Xu
2020-02-12
Mean-Field Analysis of Two-Layer Neural Networks: Non-Asymptotic Rates and Generalization Bounds
Zixiang ChenYuan CaoQuanquan GuTong Zhang
2020-02-10
Multilingual Alignment of Contextual Word Representations
Steven CaoNikita KitaevDan Klein
2020-02-10
Momentum Improves Normalized SGD
Ashok CutkoskyHarsh Mehta
2020-02-09
Application of Pre-training Models in Named Entity Recognition
Yu WangYining SunZuchang MaLisheng GaoYang XuTing Sun
2020-02-09
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing
| Canwen XuWangchunshu ZhouTao GeFuru WeiMing Zhou
2020-02-07
Introducing Aspects of Creativity in Automatic Poetry Generation
Brendan BenaJugal Kalita
2020-02-06
Rapid Adaptation of BERT for Information Extraction on Domain-Specific Business Documents
Ruixue ZhangWei YangLuyun LinZhengkai TuYuqing XieZihang FuYuhao XieLuchen TanKun XiongJimmy Lin
2020-02-05
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters
Ruize WangDuyu TangNan DuanZhongyu WeiXuanjing HuangJianshu jiGuihong CaoDaxin JiangMing Zhou
2020-02-05
Interpretable & Time-Budget-Constrained Contextualization for Re-Ranking
| Sebastian HofstätterMarkus ZlabingerAllan Hanbury
2020-02-04
Bertrand-DR: Improving Text-to-SQL using a Discriminative Re-ranker
Amol KelkarRohan RelanVaishali BhardwajSaurabh VaichalPeter Relan
2020-02-03
Beat the AI: Investigating Adversarial Human Annotations for Reading Comprehension
Max BartoloAlastair RobertsJohannes WelblSebastian RiedelPontus Stenetorp
2020-02-02
WaveTTS: Tacotron-based TTS with Joint Time-Frequency Domain Loss
Rui LiuBerrak SismanFeilong BaoGuanglai GaoHaizhou Li
2020-02-02
Fine-Tuning BERT for Schema-Guided Zero-Shot Dialogue State Tracking
Yu-Ping RuanZhen-Hua LingJia-Chen GuQuan Liu
2020-02-01
Pretrained Transformers for Simple Question Answering over Knowledge Graphs
D. LukovnikovA. FischerJ. Lehmann
2020-01-31
Adversarial Training for Aspect-Based Sentiment Analysis with BERT
| Akbar KarimiLeonardo RossiAndrea PratiKatharina Full
2020-01-30
On the Importance of Word Order Information in Cross-lingual Sequence Labeling
Zihan LiuGenta Indra WinataSamuel CahyawijayaAndrea MadottoZhaojiang LinPascale Fung
2020-01-30
Joint Contextual Modeling for ASR Correction and Language Understanding
Yue WengSai Sumanth MiryalaChandra KhatriRunze WangHuaixiu ZhengPiero Molino