WordPiece is a subword segmentation algorithm used in natural language processing. The vocabulary is initialized with individual characters in the language, then the most frequent combinations of symbols in the vocabulary are iteratively added to the vocabulary. The process is:

  1. Initialize the word unit inventory with all the characters in the text.
  2. Build a language model on the training data using the inventory from 1.
  3. Generate a new word unit by combining two units out of the current word inventory to increment the word unit inventory by one. Choose the new word unit out of all the possible ones that increases the likelihood on the training data the most when added to the model.
  4. Goto 2 until a predefined limit of word units is reached or the likelihood increase falls below a certain threshold.

Text: Source

Image: WordPiece as used in BERT

Source: Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Latest Papers

PAPER DATE
What does BERT know about books, movies and music? Probing BERT for Conversational Recommendation
| Gustavo PenhaClaudia Hauff
2020-07-30
Depressive, Drug Abusive, or Informative: Knowledge-aware Study of News Exposure during COVID-19 Outbreak
Amanuel AlamboManas GaurKrishnaprasad Thirunarayan
2020-07-30
MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering
Shayne LongpreYi LuJoachim Daiber
2020-07-30
Composer Style Classification of Piano Sheet Music Images Using Language Model Pretraining
TJ TsaiKevin Ji
2020-07-29
Variants of BERT, Random Forests and SVM approach for Multimodal Emotion-Target Sub-challenge
Hoang Manh HungHyung-Jeong YangSoo-Hyung KimGuee-Sang Lee
2020-07-28
Improving Results on Russian Sentiment Datasets
| Anton GolubevNatalia Loukachevitch
2020-07-28
BUT-FIT at SemEval-2020 Task 5: Automatic detection of counterfactual statements with deep pre-trained language representation models
Martin FajcikJosef JonMartin DocekalPavel Smrz
2020-07-28
GUIR at SemEval-2020 Task 12: Domain-Tuned Contextualized Models for Offensive Language Detection
Sajad SotudehTong XiangHao-Ren YaoSean MacAvaneyEugene YangNazli GoharianOphir Frieder
2020-07-28
KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media
| Ali SafayaMoutasem AbdullatifDeniz Yuret
2020-07-26
Reed at SemEval-2020 Task 9: Sentiment Analysis on Code-Mixed Tweets
Vinay GopalanMark Hopkins
2020-07-26
MULTISEM at SemEval-2020 Task 3: Fine-tuning BERT for Lexical Meaning
Aina Garí SolerMarianna Apidianaki
2020-07-24
Product Title Generation for Conversational Systems using BERT
Mansi Ranjit ManeShashank KediaAditya ManthaStephen GuoKannan Achan
2020-07-23
The Lottery Ticket Hypothesis for Pre-trained BERT Networks
| Tianlong ChenJonathan FrankleShiyu ChangSijia LiuYang ZhangZhangyang WangMichael Carbin
2020-07-23
IITK at the FinSim Task: Hypernym Detection in Financial Domain via Context-Free and Contextualized Word Embeddings
Vishal KeswaniSakshi SinghAshutosh Modi
2020-07-22
problemConquero at SemEval-2020 Task 12: Transformer and Soft label-based approaches
Karishma LaudJagriti SinghRandeep Kumar SahuAshutosh Modi
2020-07-21
newsSweeper at SemEval-2020 Task 11: Context-Aware Rich Feature Representations For Propaganda Classification
| Paramansh SinghSiraj SandhuSubham KumarAshutosh Modi
2020-07-21
Word Representation for Rhythms
Tongyu LuLyucheng YanGus Xia
2020-07-21
Understanding BERT Rankers Under Distillation
Luyu GaoZhuyun DaiJamie Callan
2020-07-21
A Comparison of Supervised Learning to Match Methods for Product Search
| Fatemeh SarviNikos VoskaridesLois MooimanSebastian SchelterMaarten de Rijke
2020-07-20
Mono vs Multilingual Transformer-based Models: a Comparison across Several Language Tasks
Diego de Vargas FeijoViviane Pereira Moreira
2020-07-19
Towards Debiasing Sentence Representations
Paul Pu LiangIrene Mengze LiEmily ZhengYao Chong LimRuslan SalakhutdinovLouis-Philippe Morency
2020-07-16
Translate Reverberated Speech to Anechoic Ones: Speech Dereverberation with BERT
Yang Jiao
2020-07-16
AdapterHub: A Framework for Adapting Transformers
| Jonas PfeifferAndreas RückléClifton PothAishwarya KamathIvan VulićSebastian RuderKyunghyun ChoIryna Gurevych
2020-07-15
Multimodal Word Sense Disambiguation in Creative Practice
Manuel Ladron de GuevaraChristopher GeorgeAkshat GuptaDaragh ByrneRamesh Krishnamurti
2020-07-15
Logic Constrained Pointer Networks for Interpretable Textual Similarity
| Subhadeep MajiRohan KumarManish BansalKalyani RoyPawan Goyal
2020-07-15
Predicting Clinical Diagnosis from Patients Electronic Health Records Using BERT-based Neural Networks
Pavel BlinovManvel AvetisianVladimir KokhDmitry UmerenkovAlexander Tuzhilin
2020-07-15
Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media
Alberto Barron-CedenoTamer ElsayedPreslav NakovGiovanni Da San MartinoMaram HasanainReem SuwailehFatima HaouariNikolay BabulkovBayan HamdanAlex NikolovShaden ShaarZien Sheikh Ali
2020-07-15
Deep Reinforced Query Reformulation for Information Retrieval
Xiao WangCraig MacdonaldIadh Ounis
2020-07-15
Fast and Accurate Neural CRF Constituency Parsing
| Yu ZhangHouquan ZhouZhenghua Li
2020-07-14
What's in a Name? Are BERT Named Entity Representations just as Good for any other Name?
Sriram BalasubramanianNaman JainGaurav JindalAbhijeet AwasthiSunita Sarawagi
2020-07-14
An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Models
Lifu TuGarima LalwaniSpandana GellaHe He
2020-07-14
Can neural networks acquire a structural bias from raw linguistic data?
Alex WarstadtSamuel R. Bowman
2020-07-14
Emoji Prediction: Extensions and Benchmarking
Weicheng MaRuibo LiuLili WangSoroush Vosoughi
2020-07-14
Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning
Shauharda KhadkaEstelle AflaloMattias MarderAvrech Ben-DavidSantiago MiretHanlin TangShie MannorTamir HazanSomdeb Majumdar
2020-07-14
Add a SideNet to your MainNet
Adrien Morisot
2020-07-14
An Enhanced Text Classification to Explore Health based Indian Government Policy Tweets
Aarzoo DhimanDurga Toshniwal
2020-07-13
Generative Graph Perturbations for Scene Graph Prediction
Boris KnyazevHarm de VriesCătălina CangeaGraham W. TaylorAaron CourvilleEugene Belilovsky
2020-07-11
To BAN or not to BAN: Bayesian Attention Networks for Reliable Hate Speech Detection
Kristian MiokBlaz SkrljDaniela ZaharieMarko Robnik-Sikonja
2020-07-10
BISON:BM25-weighted Self-Attention Framework for Multi-Fields Document Search
Xuan ShanChuanjie LiuYiqian XiaQi ChenYusi ZhangAngen LuoYuxiang Luo
2020-07-10
Multi-Dialect Arabic BERT for Country-Level Dialect Identification
| Bashar TalafhaMohammad AliMuhy Eddin Za'terHaitham SeelawiIbraheem TuffahaMostafa SamirWael FarhanHussein T. Al-Natsheh
2020-07-10
Contrastive Code Representation Learning
| Paras JainAjay JainTianjun ZhangPieter AbbeelJoseph E. GonzalezIon Stoica
2020-07-09
Fast Transformers with Clustered Attention
| Apoorv VyasAngelos KatharopoulosFrançois Fleuret
2020-07-09
Continual BERT: Continual Learning for Adaptive Extractive Summarization of COVID-19 Literature
Jong Won Park
2020-07-07
Exploring Heterogeneous Information Networks via Pre-Training
Yang FangXiang ZhaoWeidong Xiao
2020-07-07
LMVE at SemEval-2020 Task 4: Commonsense Validation and Explanation using Pretraining Language Model
Shilei LiuYu GuoBochao LiFeiliang Ren
2020-07-06
Deep Contextual Embeddings for Address Classification in E-commerce
Shreyas MangalgiLakshya KumarRavindra Babu Tallamraju
2020-07-06
Text Data Augmentation: Towards better detection of spear-phishing emails
Mehdi ReginaMaxime MeyerSébastien Goutal
2020-07-04
Robust Prediction of Punctuation and Truecasing for Medical ASR
Monica SunkaraSrikanth RonankiKalpit DixitSravan BodapatiKatrin Kirchhoff
2020-07-04
Language-agnostic BERT Sentence Embedding
| Fangxiaoyu FengYinfei YangDaniel CerNaveen ArivazhaganWei Wang
2020-07-03
Pretrained Semantic Speech Embeddings for End-to-End Spoken Language Understanding via Cross-Modal Teacher-Student Learning
Pavel DenisovNgoc Thang Vu
2020-07-03
Reading Comprehension in Czech via Machine Translation and Cross-lingual Transfer
Kateřina MackováMilan Straka
2020-07-03
Playing with Words at the National Library of Sweden -- Making a Swedish BERT
| Martin MalmstenLove BörjesonChris Haffenden
2020-07-03
MIRA: Leveraging Multi-Intention Co-click Information in Web-scale Document Retrieval using Deep Neural Networks
Yusi ZhangChuanjie LiuAngen LuoHui XueXuan ShanYuxiang LuoYiqian XiaYuanchi YanHaidong Wang
2020-07-03
Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey
Shivaji AlaparthiManit Mishra
2020-07-02
The Impact of Explanations on AI Competency Prediction in VQA
Kamran AlipourArijit RayXiao LinJurgen P. SchulzeYi YaoGiedrius T. Burachas
2020-07-02
Improving Event Detection using Contextual Word and Sentence Embeddings
Mariano MaisonnaveFernando DelbiancoFernando TohméAna MaguitmanEvangelos Milios
2020-07-02
Unsupervised FAQ Retrieval with Question Generation and BERT
Yosi MassBoaz CarmeliHaggai RoitmanDavid Konopnicki
2020-07-01
GAN-BERT: Generative Adversarial Learning for Robust Text Classification with a Bunch of Labeled Examples
Danilo CroceGiuseppe CastellucciRoberto Basili
2020-07-01
Integrating Multimodal Information in Large Pretrained Transformers
Wasifur RahmanMd Kamrul HasanSangwu LeeAmirAli Bagher ZadehChengfeng MaoLouis-Philippe MorencyEhsan Hoque
2020-07-01
Modelling Context and Syntactical Features for Aspect-based Sentiment Analysis
Minh Hieu PhanPhilip O. Ogunbona
2020-07-01
Roles and Utilization of Attention Heads in Transformer-based Neural Language Models
Jae-young JoSung-Hyon Myaeng
2020-07-01
Adversarial and Domain-Aware BERT for Cross-Domain Sentiment Analysis
Chunning DuHaifeng SunJingyu WangQi QiJianxin Liao
2020-07-01
How does BERT's attention change when you fine-tune? An analysis methodology and a case study in negation scope
Yiyun ZhaoSteven Bethard
2020-07-01
Intermediate-Task Transfer Learning with Pretrained Language Models: When and Why Does It Work?
Yada PruksachatkunJason PhangHaokun LiuPhu Mon HtutXiaoyi ZhangRichard Yuanzhe PangClara VaniaKatharina KannSamuel R. Bowman
2020-07-01
Would you Rather? A New Benchmark for Learning Machine Alignment with Cultural Values and Social Preferences
Yi TayDonovan OngJie FuAlvin ChanNancy ChenAnh Tuan LuuChris Pal
2020-07-01
Towards Debiasing Sentence Representations
Paul Pu LiangIrene Mengze LiEmily ZhengYao Chong LimRuslan SalakhutdinovLouis-Philippe Morency
2020-07-01
Automatic Generation of Citation Texts in Scholarly Papers: A Pilot Study
Xinyu XingXiaosheng FanXiaojun Wan
2020-07-01
Transition-based Semantic Dependency Parsing with Pointer Networks
Daniel Fern{\'a}ndez-Gonz{\'a}lezCarlos G{\'o}mez-Rodr{\'\i}guez
2020-07-01
tBERT: Topic Models and BERT Joining Forces for Semantic Similarity Detection
Nicole PeineltDong NguyenMaria Liakata
2020-07-01
Understanding Advertisements with BERT
Kanika KalraBhargav KurmaSilpa Vadakkeeveetil SreelathaManasi PatwardhanKarShirish e
2020-07-01
Feature Projection for Improved Text Classification
Qi QinWenpeng HuBing Liu
2020-07-01
A Generate-and-Rank Framework with Semantic Type Regularization for Biomedical Concept Normalization
Dongfang XuZeyu ZhangSteven Bethard
2020-07-01
Revisiting Higher-Order Dependency Parsers
Erick FonsecaAndr{\'e} F. T. Martins
2020-07-01
SUPP.AI: finding evidence for supplement-drug interactions
Lucy WangOyvind TafjordArman CohanSarthak JainSam SkjonsbergCarissa SchoenickNick BotnerWaleed Ammar
2020-07-01
Why is penguin more similar to polar bear than to sea gull? Analyzing conceptual knowledge in distributional models
Pia Sommerauer
2020-07-01
A Simple and Effective Dependency Parser for Telugu
Sneha NallaniManish ShrivastavaDipti Sharma
2020-07-01
Cross-Lingual Disaster-related Multi-label Tweet Classification with Manifold Mixup
Jishnu Ray ChowdhuryCornelia CarageaDoina Caragea
2020-07-01
Should You Fine-Tune BERT for Automated Essay Scoring?
Elijah MayfieldAlan W Black
2020-07-01
A BERT-based One-Pass Multi-Task Model for Clinical Temporal Relation Extraction
Chen LinTimothy MillerDmitriy DligachFarig SadequeSteven BethardGuergana Savova
2020-07-01
Evaluating the Utility of Model Configurations and Data Augmentation on Clinical Semantic Textual Similarity
Yuxia WangFei LiuKarin VerspoorTimothy Baldwin
2020-07-01
Item-based Collaborative Filtering with BERT
Tian WangYuyangzi Fu
2020-07-01
Sarcasm Identification and Detection in Conversion Context using BERT
Kalaivani A.Thenmozhi D.
2020-07-01
Neural Sarcasm Detection using Conversation Context
Nikhil Jaiswal
2020-07-01
Context-Aware Sarcasm Detection Using BERT
Arup BaruahKaushik DasFerdous BarbhuiyaKuntal Dey
2020-07-01
Character aware models with similarity learning for metaphor detection
Tarun KumarYashvardhan Sharma
2020-07-01
IlliniMet: Illinois System for Metaphor Detection with Contextual and Linguistic Information
Hongyu GongKshitij GuptaAkriti JainSuma Bhat
2020-07-01
Go Figure! Multi-task transformer-based architecture for metaphor detection using idioms: ETS team in 2020 metaphor shared task
Xianyang ChenChee Wee (Ben) LeongMichael FlorBeata Beigman Klebanov
2020-07-01
Metaphor Detection Using Contextual Word Embeddings From Transformers
Jerry LiuNathan O{'}HaraAlex RubinerRachel DraelosCynthia Rudin
2020-07-01
A Transformer Approach to Contextual Sarcasm Detection in Twitter
Hunter GregorySteven LiPouya MohammadiNatalie TarnRachel DraelosCynthia Rudin
2020-07-01
Turku Enhanced Parser Pipeline: From Raw Text to Enhanced Graphs in the IWPT 2020 Shared Task
Jenna KanervaFilip GinterSampo Pyysalo
2020-07-01
K\opsala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding
Daniel HershcovichMiryam de LhoneuxArtur KulmizevElham PejhanJoakim Nivre
2020-07-01
RobertNLP at the IWPT 2020 Shared Task: Surprisingly Simple Enhanced UD Parsing for English
Stefan Gr{\"u}newaldAnnemarie Friedrich
2020-07-01
The HW-TSC Video Speech Translation System at IWSLT 2020
Minghan WangHao YangYao DengYing QinLizhi LeiDaimeng WeiHengchao ShangNing XieXiaochun LiJiaxian Guo
2020-07-01
CopyBERT: A Unified Approach to Question Generation with Self-Attention
Stalin VaranasiSaadullah AminGuenter Neumann
2020-07-01
Robust Prediction of Punctuation and Truecasing for Medical ASR
Monica SunkaraSrikanth RonankiKalpit DixitSravan BodapatiKatrin Kirchhoff
2020-07-01
Exploring the Limits of Simple Learners in Knowledge Distillation for Document Classification with DocBERT
Ashutosh AdhikariAchyudh RamRaphael TangWilliam L. HamiltonJimmy Lin
2020-07-01
Joint Training with Semantic Role Labeling for Better Generalization in Natural Language Inference
Cemil CengizDeniz Yuret
2020-07-01
A Metric Learning Approach to Misogyny Categorization
Juan Manuel CoriaSahar GhannaySophie RossetHerv{\'e} Bredin
2020-07-01
Contextual and Non-Contextual Word Embeddings: an in-depth Linguistic Investigation
Alessio MiaschiFelice Dell{'}Orletta
2020-07-01
What's in a Name? Are BERT Named Entity Representations just as Good for any other Name?
Sriram BalasubramanianNaman JainGaurav JindalAbhijeet AwasthiSunita Sarawagi
2020-07-01
Getting the \#\#life out of living: How Adequate Are Word-Pieces for Modelling Complex Morphology?
Stav KleinReut Tsarfaty
2020-07-01
SentiTel: TABSA for Twitter reviews on Uganda Telecoms
David KabiitoJoyce Nakatumba Nabende
2020-07-01
Adversarial Evaluation of BERT for Biomedical Named Entity Recognition
Vladimir AraujoAndr{\'e}s CarvalloDenis Parra
2020-07-01
Improving Multimodal Named Entity Recognition via Entity Span Detection with Unified Multimodal Transformer
Jianfei YuJing JiangLi YangRui Xia
2020-07-01
Investigating the effect of auxiliary objectives for the automated grading of learner English speech transcriptions
Hannah CraigheadAndrew CainesPaula ButteryHelen Yannakoudakis
2020-07-01
SE3M: A Model for Software Effort Estimation Using Pre-trained Embedding Models
Eliane M. De Bortoli FáveroDalcimar CasanovaAndrey Ricardo Pimentel
2020-06-30
Data Movement Is All You Need: A Case Study on Optimizing Transformers
Andrei IvanovNikoli DrydenTal Ben-NunShigang LiTorsten Hoefler
2020-06-30
Segmentation Approach for Coreference Resolution Task
Aref JafariAli Ghodsi
2020-06-30
Want to Identify, Extract and Normalize Adverse Drug Reactions in Tweets? Use RoBERTa
Katikapalli Subramanyam KalyanS. Sangeetha
2020-06-29
Improving Sequence Tagging for Vietnamese Text Using Transformer-based Neural Models
Viet Bui TheOanh Tran ThiPhuong Le-Hong
2020-06-29
Interpreting Hierarchical Linguistic Interactions in DNNs
Die ZhangHuilin ZhouXiaoyi BaoDa HuoRuizhao ChenXu ChengHao ZhangMengyue WuQuanshi Zhang
2020-06-29
Rethinking Positional Encoding in Language Pre-training
| Guolin KeDi HeTie-Yan Liu
2020-06-28
BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision
| Chen LiangYue YuHaoming JiangSiawpeng ErRuijia WangTuo ZhaoChao Zhang
2020-06-28
FastSpec: Scalable Generation and Detection of Spectre Gadgets Using Neural Embeddings
| M. Caner TolKoray YurtsevenBerk GulmezogluBerk Sunar
2020-06-25
Normalizing Text using Language Modelling based on Phonetics and String Similarity
Fenil DoshiJimit GandhiDeep GosaliaSudhir Bagul
2020-06-25
LSBert: A Simple Framework for Lexical Simplification
| Jipeng QiangYun LiYi ZhuYunhao YuanXindong Wu
2020-06-25
Accelerated Large Batch Optimization of BERT Pretraining in 54 minutes
Shuai ZhengHaibin LinSheng ZhaMu Li
2020-06-24
Efficient Constituency Parsing by Pointing
Thanh-Tung NguyenXuan-Phi NguyenShafiq JotyXiaoli Li
2020-06-24
ReCO: A Large Scale Chinese Reading Comprehension Dataset on Opinion
| BingningWangTing YaoQi ZhangJingfang XuXiaochuan Wang
2020-06-22
Students Need More Attention: BERT-based AttentionModel for Small Data with Application to AutomaticPatient Message Triage
Shijing SiRui WangJedrek WosikHao ZhangDavid DovGuoyin WangRicardo HenaoLawrence Carin
2020-06-22
Sarcasm Detection in Tweets with BERT and GloVe Embeddings
Akshay KhatriPranav PDr. Anand Kumar M
2020-06-20
New Vietnamese Corpus for Machine ReadingComprehension of Health News Articles
Kiet Van NguyenDuc-Vu NguyenAnh Gia-Tuan NguyenNgan Luu-Thuy Nguyen
2020-06-19
A Qualitative Evaluation of Language Models on Automatic Question-Answering for COVID-19
| David OnianiYanshan Wang
2020-06-19
SqueezeBERT: What can computer vision teach NLP about efficient neural networks?
Forrest N. IandolaAlbert E. ShawRavi KrishnaKurt W. Keutzer
2020-06-19
Exploring the BERT Cross-Lingual Transferability: a Case Study in Reading Comprehension
Konovalov V. P.Gulyaev P. A.Sorokin A. A.Kuratov Y. M.Burtsev M. S.
2020-06-17
Tagging and parsing of multidomain collections
| Alexey SorokinIvan SmurovDenis Kirianov
2020-06-17
Improving accuracy and speeding up Document Image Classification through parallel systems
Javier FerrandoJuan Luis DominguezJordi TorresRaul GarciaDavid GarciaDaniel GarridoJordi CortadaMateo Valero
2020-06-16
PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models
| Eyal Ben-DavidCarmel RabinovitzRoi Reichart
2020-06-16
The SPPD System for Schema Guided Dialogue State Tracking Challenge
Miao LiHaoqi XiongYunbo Cao
2020-06-16
Scalable Cross Lingual Pivots to Model Pronoun Gender for Translation
Kellie WebsterEmily Pitler
2020-06-16
End-to-End Code Switching Language Models for Automatic Speech Recognition
Ahan M. R.Shreyas Sunil Kulkarni
2020-06-16
FinBERT: A Pretrained Language Model for Financial Communications
| Yi YangMark Christopher Siy UYAllen Huang
2020-06-15
Document Classification for COVID-19 Literature
Bernal Jiménez GutiérrezJuncheng ZengDongdong ZhangPing ZhangYu Su
2020-06-15
Cooking Is All About People: Comment Classification On Cookery Channels Using BERT and Classification Models (Malayalam-English Mix-Code)
Subramaniam KazhuparambilAbhishek Kaushik
2020-06-15
FinEst BERT and CroSloEngual BERT: less is more in multilingual models
Matej UlčarMarko Robnik-Šikonja
2020-06-14
Transferring Monolingual Model to Low-Resource Language: The Case of Tigrinya
Abrhalei TelaAbraham WoubieVille Hautamaki
2020-06-13
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages
Pedro Javier Ortiz SuárezLaurent RomaryBenoît Sagot
2020-06-11
Revisiting Few-sample BERT Fine-tuning
| Tianyi ZhangFelix WuArzoo KatiyarKilian Q. WeinbergerYoav Artzi
2020-06-10
MC-BERT: Efficient Language Pre-Training via a Meta Controller
| Zhenhui XuLinyuan GongGuolin KeDi HeShuxin ZhengLiwei WangJiang BianTie-Yan Liu
2020-06-10
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines
| Marius MosbachMaksym AndriushchenkoDietrich Klakow
2020-06-08
Pre-training Polish Transformer-based Language Models at Scale
| Sławomir DadasMichał PerełkiewiczRafał Poświata
2020-06-07
BERT Loses Patience: Fast and Robust Inference with Early Exit
| Wangchunshu ZhouCanwen XuTao GeJulian McAuleyKe XuFuru Wei
2020-06-07
Medical Concept Normalization in User Generated Texts by Learning Target Concept Embeddings
Katikapalli Subramanyam KalyanS. Sangeetha
2020-06-07
Accelerating Natural Language Understanding in Task-Oriented Dialog
Ojas AhujaShrey Desai
2020-06-05
UDPipe at EvaLatin 2020: Contextualized Embeddings and Treebank Embeddings
Milan StrakaJana Straková
2020-06-05
DeBERTa: Decoding-enhanced BERT with Disentangled Attention
| Pengcheng HeXiaodong LiuJianfeng GaoWeizhu Chen
2020-06-05
The SOFC-Exp Corpus and Neural Approaches to Information Extraction in the Materials Science Domain
Annemarie FriedrichHeike AdelFederico TomazicJohannes HingerlRenou BenteauAnika MaruscykLukas Lange
2020-06-04
Experiments on Paraphrase Identification Using Quora Question Pairs Dataset
Andreas ChandraRuben Stefanus
2020-06-04
Automatic Text Summarization of COVID-19 Medical Research Articles using BERT and GPT-2
| Virapat KieuvongngamBowen TanYiming Niu
2020-06-03
WikiBERT models: deep transfer learning for many languages
Sampo PyysaloJenna KanervaAntti VirtanenFilip Ginter
2020-06-02
Question Answering on Scholarly Knowledge Graphs
Mohamad Yaser JaradehMarkus StockerSören Auer
2020-06-02
A Pairwise Probe for Understanding BERT Fine-Tuning on Machine Reading Comprehension
Jie CaiZhengzhou ZhuPing NieQian Liu
2020-06-02
BERT Based Multilingual Machine Comprehension in English and Hindi
| Somil GuptaNilesh Khade
2020-06-02
Exploring Cross-sentence Contexts for Named Entity Recognition with BERT
Jouni LuomaSampo Pyysalo
2020-06-02
Position Masking for Language Models
Andy WagnerTiyasa MitraMrinal IyerGodfrey Da CostaMarc Tremblay
2020-06-02
Emergence of Separable Manifolds in Deep Language Representations
Jonathan MamouHang LeMiguel Del RioCory StephensonHanlin TangYoon KimSueYeon Chung
2020-06-01
Conversational Machine Comprehension: a Literature Review
Somil GuptaBhanu Pratap Singh Rawat
2020-06-01
When Bert Forgets How To POS: Amnesic Probing of Linguistic Properties and MLM Predictions
Yanai ElazarShauli RavfogelAlon JacoviYoav Goldberg
2020-06-01
An Effective Contextual Language Modeling Framework for Speech Summarization with Augmented Features
Shi-Yan WengTien-Hong LoBerlin Chen
2020-06-01
BERT-based Ensembles for Modeling Disclosure and Support in Conversational Social Media Text
Tanvi DaduKartikey PantRadhika Mamidi
2020-06-01
Neural Entity Linking: A Survey of Models based on Deep Learning
| Ozge SevgiliArtem ShelmanovMikhail ArkhipovAlexander PanchenkoChris Biemann
2020-05-31
"Judge me by my size (noun), do you?'' YodaLib: A Demographic-Aware Humor Generation Framework
Aparna GarimellaCarmen BaneaNabil HossainRada Mihalcea
2020-05-31
BPGC at SemEval-2020 Task 11: Propaganda Detection in News Articles with Multi-Granularity Knowledge Sharing and Linguistic Features based Ensemble Learning
Rajaswa PatilSomesh SinghSwati Agarwal
2020-05-31
LRG at SemEval-2020 Task 7: Assessing the Ability of BERT and Derivative Models to Perform Short-Edits based Humor Grading
Siddhant MahurkarRajaswa Patil
2020-05-31
Detecting Problem Statements in Peer Assessments
Yunkai XiaoGabriel ZingleQinjin JiaHarsh R. ShahYi ZhangTianyi LiMohsin KarovaliyaWeixiang ZhaoYang SongJie JiAshwin BalasubramaniamHarshit PatelPriyankha BhalasubbramanianVikram PatelEdward F. Gehringer
2020-05-30
Using Large Pretrained Language Models for Answering User Queries from Product Specifications
Kalyani RoySmit ShahNithish PaiJaidam RamtejPrajit Prashant NadkarnJyotirmoy BanerjeePawan GoyalSurender Kumar
2020-05-29
SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutions
Mao YeChengyue GongQiang Liu
2020-05-29
A Comparative Study of Lexical Substitution Approaches based on Neural Language Models
Nikolay ArefyevBoris SheludkoAlexander PodolskiyAlexander Panchenko
2020-05-29
Stance Prediction for Contemporary Issues: Data and Experiments
| Marjan HosseiniaEduard DragutArjun Mukherjee
2020-05-29
On Incorporating Structural Information to improve Dialogue Response Generation
| Nikita MoghePriyesh VijayanBalaraman RavindranMitesh M. Khapra
2020-05-28
Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Adhiguna KuncoroLingpeng KongDaniel FriedDani YogatamaLaura RimellChris DyerPhil Blunsom
2020-05-27
CausaLM: Causal Model Explanation Through Counterfactual Language Models
Amir FederNadav OvedUri ShalitRoi Reichart
2020-05-27
Transition-based Semantic Dependency Parsing with Pointer Networks
Daniel Fernández-GonzálezCarlos Gómez-Rodríguez
2020-05-27
Language Representation Models for Fine-Grained Sentiment Classification
Brian CheangBailey WeiDavid KoganHowey QiuMasud Ahmed
2020-05-27
Network Fusion for Content Creation with Conditional INNs
Robin RombachPatrick EsserBjörn Ommer
2020-05-27
A Data-driven Approach for Noise Reduction in Distantly Supervised Biomedical Relation Extraction
Saadullah AminKatherine Ann DunfieldAnna VechkaevaGünter Neumann
2020-05-26
What Are People Asking About COVID-19? A Question Classification Dataset
| Jerry WeiChengyu HuangSoroush VosoughiJason Wei
2020-05-26
ParsBERT: Transformer-based Model for Persian Language Understanding
| Mehrdad FarahaniMohammad GharachorlooMarzieh FarahaniMohammad Manthouri
2020-05-26
BEEP! Korean Corpus of Online News Comments for Toxic Speech Detection
| Jihyung MoonWon Ik ChoJunbum Lee
2020-05-26
Comparing BERT against traditional machine learning text classification
Santiago González-CarvajalEduardo C. Garrido-Merchán
2020-05-26
BERT-XML: Large Scale Automated ICD Coding Using BERT Pretraining
Zachariah ZhangJingshu LiuNarges Razavian
2020-05-26
An Audio-enriched BERT-based Framework for Spoken Multiple-choice Question Answering
Chia-Chih KuoShang-Bao LuoKuan-Yu Chen
2020-05-25
Køpsala: Transition-Based Graph Parsing via Efficient Training and Effective Encoding
| Daniel HershcovichMiryam de LhoneuxArtur KulmizevElham PejhanJoakim Nivre
2020-05-25
Pointwise Paraphrase Appraisal is Potentially Problematic
Hannah ChenYangfeng JiDavid Evans
2020-05-25
Jointly Encoding Word Confusion Network and Dialogue Context with BERT for Spoken Language Understanding
Chen LiuSu ZhuZijian ZhaoRuisheng CaoLu ChenKai Yu
2020-05-24
Comparative Study of Machine Learning Models and BERT on SQuAD
Devshree PatelParam RavalRatnam ParikhYesha Shastri
2020-05-22
L2R2: Leveraging Ranking for Abductive Reasoning
| Yunchang ZhuLiang PangYanyan LanXueqi Cheng
2020-05-22
Living Machines: A study of atypical animacy
Mariona Coll ArdanuyFederico NanniKaspar BeelenKasra HosseiniRuth AhnertJon LawrenceKatherine McDonoughGiorgia TolfoDaniel CS WilsonBarbara McGillivray
2020-05-22
Robust Layout-aware IE for Visually Rich Documents with Pre-trained Language Models
Mengxi WeiYifan HeQiong Zhang
2020-05-22
Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction
Laila RasmyYang XiangZiqian XieCui TaoDegui Zhi
2020-05-22
BERTweet: A pre-trained language model for English Tweets
| Dat Quoc NguyenThanh VuAnh Tuan Nguyen
2020-05-20
FashionBERT: Text and Image Matching with Adaptive Loss for Cross-modal Retrieval
Dehong GaoLinbo JinBen ChenMinghui QiuPeng LiYi WeiYi HuHao Wang
2020-05-20
Cross-lingual Transfer Learning for Dialogue Act Recognition
Jiří MartínekChristophe CerisaraPavel KrálLadislav Lenc
2020-05-19
Table Search Using a Deep Contextualized Language Model
| Zhiyu ChenMohamed TrabelsiJeff HeflinYinan XuBrian D. Davison
2020-05-19
Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch Gestalt
Hangyu LinYanwei FuYu-Gang JiangXiangyang Xue
2020-05-19
Audio ALBERT: A Lite BERT for Self-supervised Learning of Audio Representation
Po-Han ChiPei-Hung ChungTsung-Han WuChun-Cheng HsiehShang-Wen LiHung-yi Lee
2020-05-18
Are All Languages Created Equal in Multilingual BERT?
Shijie WuMark Dredze
2020-05-18
Context-Based Quotation Recommendation
Ansel MacLaughlinTao ChenBurcu Karagol AyanDan Roth
2020-05-17
Support-BERT: Predicting Quality of Question-Answer Pairs in MSDN using Deep Bidirectional Transformer
Bhaskar SenNikhil GopalXinwei Xue
2020-05-17
Building a Hebrew Semantic Role Labeling Lexical Resource from Parallel Movie Subtitles
Ben EyalMichael Elhadad
2020-05-17
Cross-Lingual Low-Resource Set-to-Description Retrieval for Global E-Commerce
| Juntao LiChang LiuJian WangLidong BingHongsong LiXiaozhong LiuDongyan ZhaoRui Yan
2020-05-17
Adversarial Training for Commonsense Inference
Lis PereiraXiaodong LiuFei ChengMasayuki AsaharaIchiro Kobayashi
2020-05-17
TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data
| Pengcheng YinGraham NeubigWen-tau YihSebastian Riedel
2020-05-17
CERT: Contrastive Self-supervised Learning for Language Understanding
Hongchao FangSicheng WangMeng ZhouJiayuan DingPengtao Xie
2020-05-16
Leveraging Affective Bidirectional Transformers for Offensive Language Detection
AbdelRahim ElmadanyChiyu ZhangMuhammad Abdul-MageedAzadeh Hashemi
2020-05-16
Spelling Error Correction with Soft-Masked BERT
| Shaohua ZhangHaoran HuangJicong LiuHang Li
2020-05-15
Neural Entity Linking on Technical Service Tickets
Nadja KurzFelix HamannAdrian Ulges
2020-05-15
Challenges in Emotion Style Transfer: An Exploration with a Lexical Substitution Pipeline
David HelbigEnrica TroianoRoman Klinger
2020-05-15
[email protected] at SemEval-2020 Task 12: Identifying Multilingual Offensive Tweets Using Weighted Ensemble and Fine-Tuned BERT
Saja Khaled TawalbehMahmoud HammadMohammad AL-Smadi
2020-05-15
NIT-Agartala-NLP-Team at SemEval-2020 Task 8: Building Multimodal Classifiers to tackle Internet Humor
Steve Durairaj SwamyShubham LaddhaBasil AbdussalamDebayan DattaAnupam Jamatia
2020-05-14
A pre-training technique to localize medical BERT and enhance BioBERT
| Shoya WadaToshihiro TakedaShiro ManabeShozo KonishiJun KamoharaYasushi Matsumura
2020-05-14
Parallel Corpus Filtering via Pre-trained Language Models
Boliang ZhangAjay NageshKevin Knight
2020-05-13
Entity-Enriched Neural Models for Clinical Question Answering
| Bhanu Pratap Singh RawatWei-Hung WengPreethi RaghavanPeter Szolovits
2020-05-13
On the Robustness of Language Encoders against Grammatical Errors
Fan YinQuanyu LongTao MengKai-Wei Chang
2020-05-12
Detecting Adverse Drug Reactions from Twitter through Domain-Specific Preprocessing and BERT Ensembling
Amy BredenLee Moore
2020-05-11
How Context Affects Language Models' Factual Predictions
Fabio PetroniPatrick LewisAleksandra PiktusTim RocktäschelYuxiang WuAlexander H. MillerSebastian Riedel
2020-05-10
Transformer Based Language Models for Similar Text Retrieval and Ranking
Javed Qadrud-DinAshraf Bah RabiouRyan WalkerRavi SoniMartin GajekGabriel PackAkhil Rangaraj
2020-05-10
Finding Universal Grammatical Relations in Multilingual BERT
Ethan A. ChiJohn HewittChristopher D. Manning
2020-05-09
It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations
| Samson TanShafiq JotyMin-Yen KanRichard Socher
2020-05-09
LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluation
Gustavo AguilarSudipta KarThamar Solorio
2020-05-09
schuBERT: Optimizing Elements of BERT
Ashish KhetanZohar Karnin
2020-05-09
SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics
| Da YinTao MengKai-Wei Chang
2020-05-08
Distilling Knowledge from Pre-trained Language Models via Text Smoothing
Xing WuYibing LiuXiangyang ZhouDianhai Yu
2020-05-08
GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference
Ali Hadi ZadehAndreas Moshovos
2020-05-08
Temporal Common Sense Acquisition with Minimal Supervision
Ben ZhouQiang NingDaniel KhashabiDan Roth
2020-05-08
Comparative Analysis of Text Classification Approaches in Electronic Health Records
Aurelie MascioZeljko KraljevicDaniel BeanRichard DobsonRobert StewartRebecca BendayanAngus Roberts
2020-05-08
LIIR at SemEval-2020 Task 12: A Cross-Lingual Augmentation Approach for Multilingual Offensive Language Identification
Erfan GhaderyMarie-Francine Moens
2020-05-07
Harvesting and Refining Question-Answer Pairs for Unsupervised QA
| Zhongli LiWenhui WangLi DongFuru WeiKe Xu
2020-05-06
An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining
Yifan PengQingyu ChenZhiyong Lu
2020-05-06
Autoencoding Pixies: Amortised Variational Inference with Graph Convolutions for Functional Distributional Semantics
Guy Emerson
2020-05-06
Categorical Vector Space Semantics for Lambek Calculus with a Relevant Modality
Lachlan McPheatMehrnoosh SadrzadehHadi WazniGijs Wijnholds
2020-05-06
MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models
| Mandy GuoYinfei YangDaniel CerQinlan ShenNoah Constant
2020-05-05
Contextualizing Hate Speech Classifiers with Post-hoc Explanation
Brendan KennedyXisen JinAida Mostafazadeh DavaniMorteza DehghaniXiang Ren
2020-05-05
Establishing Baselines for Text Classification in Low-Resource Languages
| Jan Christian Blaise CruzCharibeth Cheng
2020-05-05
ExpBERT: Representation Engineering with Natural Language Explanations
| Shikhar MurtyPang Wei KohPercy Liang
2020-05-05
ImpactCite: An XLNet-based method for Citation Impact Analysis
Dominique MercierSyed Tahseen Raza RizviVikas RajashekarAndreas DengelSheraz Ahmed
2020-05-05
Robust Encodings: A Framework for Combating Adversarial Typos
Erik JonesRobin JiaAditi RaghunathanPercy Liang
2020-05-04
Unsupervised Alignment-based Iterative Evidence Retrieval for Multi-hop Question Answering
| Vikas YadavSteven BethardMihai Surdeanu
2020-05-04
Spying on your neighbors: Fine-grained probing of contextual embeddings for information about surrounding words
Josef KlafkaAllyson Ettinger
2020-05-04
Code and Named Entity Recognition in StackOverflow
| Jeniya TabassumMounica MaddelaWei XuAlan Ritter
2020-05-04
Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction
| Masahiro KanekoMasato MitaShun KiyonoJun SuzukiKentaro Inui
2020-05-03
BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA
Nora KassnerHinrich Schütze
2020-05-02
DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering
| Qingqing CaoHarsh TrivediAruna BalasubramanianNiranjan Balasubramanian
2020-05-02
Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Models
Bill Yuchen LinSeyeon LeeRahul KhannaXiang Ren
2020-05-02
Generating Derivational Morphology with BERT
Valentin HofmannJanet B. PierrehumbertHinrich Schütze
2020-05-02
IsoBN: Fine-Tuning BERT with Isotropic Batch Normalization
| Wenxuan ZhouBill Yuchen LinXiang Ren
2020-05-02
Contrastive Self-Supervised Learning for Commonsense Reasoning
| Tassilo KleinMoin Nabi
2020-05-02
HipoRank: Incorporating Hierarchical and Positional Information into Graph-based Unsupervised Long Document Extractive Summarization
Yue DongAndrei RomascanuJackie C. K. Cheung
2020-05-01
Identifying Necessary Elements for BERT's Multilinguality
| Philipp DufterHinrich Schütze
2020-05-01
Hitachi at SemEval-2020 Task 12: Offensive Language Identification with Noisy Labels using Statistical Sampling and Post-Processing
Manikandan RavikiranAmin Ekant MuljibhaiToshinori MiyoshiHiroaki OzakiYuta KoreedaSakata Masayuki
2020-05-01
Cross-Linguistic Syntactic Evaluation of Word Prediction Models
| Aaron MuellerGarrett NicolaiPanayiota Petrou-ZeniouNatalia TalminaTal Linzen
2020-05-01
Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?
Yada PruksachatkunJason PhangHaokun LiuPhu Mon HtutXiaoyi ZhangRichard Yuanzhe PangClara VaniaKatharina KannSamuel R. Bowman
2020-05-01
Clinical Reading Comprehension: A Thorough Analysis of the emrQA Dataset
Xiang YueBernal Jimenez GutierrezHuan Sun
2020-05-01
When BERT Plays the Lottery, All Tickets Are Winning
Sai PrasannaAnna RogersAnna Rumshisky
2020-05-01
POINTER: Constrained Text Generation via Insertion-based Generative Pre-training
| Yizhe ZhangGuoyin WangChunyuan LiZhe GanChris BrockettBill Dolan
2020-05-01
Probing Text Models for Common Ground with Visual Representations
Gabriel IlharcoRowan ZellersAli FarhadiHannaneh Hajishirzi
2020-05-01
Text Categorization for Conflict Event Annotation
Fredrik OlssonMagnus SahlgrenFehmi ben AbdesslemAriel EkgrenKristine Eck
2020-05-01
TF-IDF Character N-grams versus Word Embedding-based Models for Fine-grained Event Classification: A Preliminary Study
Jakub PiskorskiGuillaume Jacquet
2020-05-01
TermEval 2020: TALN-LS2N System for Automatic Term Extraction
Amir HazemBouhM{\'e}rieme iFlorian BoudinBeatrice Daille
2020-05-01
FrameNet Annotations Alignment using Attention-based Machine Translation
Gabriel Marzinotto
2020-05-01
Implementation of Supervised Training Approaches for Monolingual Word Sense Alignment: ACDH-CH System Description for the MWSA Shared Task at GlobaLex 2020
Lenka BajceticSeung-bin Yim
2020-05-01
Transfer learning applied to text classification in Spanish radiological reports
Pilar L{\'o}pez {\'U}bedaManuel Carlos D{\'\i}az-GalianoL. Alfonso Urena LopezMaite MartinTeodoro Mart{\'\i}n-NoguerolAntonio Luna
2020-05-01
Aggression Identification in Social Media: a Transfer Learning Based Approach
RamiFaneva risoaJosiane Mothe
2020-05-01
IRIT at TRAC 2020
RamiFaneva risoaJosiane Mothe
2020-05-01
Bagging BERT Models for Robust Aggression Identification
Julian RischRalf Krestel
2020-05-01
Scmhl5 at TRAC-2 Shared Task on Aggression Identification: Bert Based Ensemble Learning Approach
Han LiuPete BurnapWafa AlorainyMatthew Williams
2020-05-01
Aggression Identification in English, Hindi and Bangla Text using BERT, RoBERTa and SVM
| Arup BaruahKaushik DasFerdous BarbhuiyaKuntal Dey
2020-05-01
Aggression and Misogyny Detection using BERT: A Multi-Task Approach
| Niloofar Safi SamghabadiParth PatwaSrinivas PYKLPrerana MukherjeeAmitava DasThamar Solorio
2020-05-01
From Web Crawl to Clean Register-Annotated Corpora
Veronika LaippalaSamuel R{\"o}nnqvistSaara Hellstr{\"o}mJuhani LuotolahtiLiina RepoAnna SalmelaValtteri SkantsiSampo Pyysalo
2020-05-01
Cross-lingual Zero Pronoun Resolution
Abdulrahman AlorainiMassimo Poesio
2020-05-01
Understanding User Utterances in a Dialog System for Caregiving
Yoshihiko AsaoJulien KloetzerJunta MizunoDai SaikiKazuma KadowakiKentaro Torisawa
2020-05-01
Joint Learning of Syntactic Features Helps Discourse Segmentation
Takshak DesaiParag Pravin DakleDan Moldovan
2020-05-01
Adapting BERT to Implicit Discourse Relation Classification with a Focus on Discourse Connectives
Yudai KishimotoYugo MurawakiSadao Kurohashi
2020-05-01
Automated Essay Scoring System for Nonnative Japanese Learners
Reo HiraoMio AraiHiroki ShimanakaSatoru KatsumataMamoru Komachi
2020-05-01
Development and Validation of a Corpus for Machine Humor Comprehension
Yuen-Hsien TsengWun-Syuan WuChia-Yueh ChangHsueh-Chih ChenWei-Lun Hsu
2020-05-01
Abusive language in Spanish children and young teenager's conversations: data preparation and short text classification with contextual word embeddings
Marta R. Costa-juss{\`a}Esther Gonz{\'a}lezAsuncion MorenoEudald Cumalat
2020-05-01
An Evaluation Dataset for Identifying Communicative Functions of Sentences in English Scholarly Papers
Kenichi IwatsukiFlorian BoudinAkiko Aizawa
2020-05-01
SiBert: Enhanced Chinese Pre-trained Language Model with Sentence Insertion
Jiahao ChenChenjie CaoXiuyan Jiang
2020-05-01
Adaptation of Deep Bidirectional Transformers for Afrikaans Language
Sello Ralethe
2020-05-01
Massive vs. Curated Embeddings for Low-Resourced Languages: the Case of Yor\`ub\'a and Twi
Jesujoba AlabiKwabena Amponsah-KaakyireDavid AdelaniCristina Espa{\~n}a-Bonet
2020-05-01
Building a Task-oriented Dialog System for Languages with no Training Data: the Case for Basque
Maddalen L{\'o}pez de LacalleXabier SaralegiI{\~n}aki San Vicente
2020-05-01
Introducing a Large-Scale Dataset for Vietnamese POS Tagging on Conversational Texts
Oanh TranTu PhamVu DangBang Nguyen
2020-05-01
DaNE: A Named Entity Resource for Danish
Rasmus HvingelbyAmalie Brogaard PauliMaria BarrettChristina RostedLasse Malm LidegaardAnders S{\o}gaard
2020-05-01
Is Language Modeling Enough? Evaluating Effective Embedding Combinations
Rudolf SchneiderTom OberhauserPaul GrundmannFelix Alex GerserAlex LoesererSteffen Staab
2020-05-01
Parsing as Tagging
Robert VacareanuGeorge Caique Gouveia BarbosaMarco A. Valenzuela-Esc{\'a}rcegaMihai Surdeanu
2020-05-01
AIA-BDE: A Corpus of FAQs in Portuguese and their Variations
Hugo Gon{\c{c}}alo OliveiraJo{\~a}o FerreiraJos{\'e} SantosPedro FialhoRicardo RodriguesLuisa CoheurAna Alves
2020-05-01
Cross-lingual and Cross-domain Evaluation of Machine Reading Comprehension with Squad and CALOR-Quest Corpora
Delphine CharletGeraldine DamnatiFrederic Bechetgabriel marzinottoJohannes Heinecke
2020-05-01
Contextualized Embeddings based Transformer Encoder for Sentence Similarity Modeling in Answer Selection Task
| Md Tahmid Rahman LaskarJimmy Xiangji HuangEnamul Hoque
2020-05-01
One Classifier for All Ambiguous Words: Overcoming Data Sparsity by Utilizing Sense Correlations Across Words
Prafulla Kumar ChoubeyRuihong Huang
2020-05-01
A Summarization Dataset of Slovak News Articles
| Marek SuppaJergus Adamec
2020-05-01
KLEJ: Comprehensive Benchmark for Polish Language Understanding
| Piotr RybakRobert MroczkowskiJanusz TraczIreneusz Gawlik
2020-05-01
Analyzing ELMo and DistilBERT on Socio-political News Classification
Berfu B{\"u}y{\"u}k{\"o}zAli H{\"u}rriyeto{\u{g}}luArzucan {\"O}zg{\"u}r
2020-05-01
SciREX: A Challenge Dataset for Document-Level Information Extraction
| Sarthak JainMadeleine van ZuylenHannaneh HajishirziIz Beltagy
2020-05-01
WiC-TSV: An Evaluation Benchmark for Target Sense Verification of Words in Context
Anna BreitArtem RevenkoKiamehr RezaeeMohammad Taher PilehvarJose Camacho-Collados
2020-04-30
On the Evaluation of Contextual Embeddings for Zero-Shot Cross-Lingual Transfer Learning
Phillip KeungYichao LuJulian SalazarVikas Bhardwaj
2020-04-30
A Matter of Framing: The Impact of Linguistic Formalism on Probing Results
Ilia KuznetsovIryna Gurevych
2020-04-30
SegaBERT: Pre-training of Segment-aware BERT for Language Understanding
He BaiPeng ShiJimmy LinLuchen TanKun XiongWen GaoMing Li
2020-04-30
How do Decisions Emerge across Layers in Neural Models? Interpretation with Differentiable Masking
| Nicola De CaoMichael SchlichtkrullWilker AzizIvan Titov
2020-04-30
Investigating Transferability in Pretrained Language Models
Alex TamkinTrisha SinghDavide GiovanardiNoah Goodman
2020-04-30
Enriched Pre-trained Transformers for Joint Slot Filling and Intent Detection
Momchil HardalovIvan KoychevPreslav Nakov
2020-04-30
Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT
Zhiyong WuYun ChenBen KaoQun Liu
2020-04-30
Robust Question Answering Through Sub-part Alignment
Jifan ChenGreg Durrett
2020-04-30
Modular Representation Underlies Systematic Generalization in Neural Natural Language Inference Models
Atticus GeigerKyle RichardsonChristopher Potts
2020-04-30
Universal Dependencies according to BERT: both more specific and more general
| Tomasz LimisiewiczRudolf RosaDavid Mareček
2020-04-30
Look at the First Sentence: Position Bias in Question Answering
Miyoung KoJinhyuk LeeHyunjae KimGangwoo KimJaewoo Kang
2020-04-30
Exploring Contextualized Neural Language Models for Temporal Dependency Parsing
Hayley RossJonathan CaiBonan Min
2020-04-30
Interpretable Entity Representations through Large-Scale Typing
Yasumasa OnoeGreg Durrett
2020-04-30
MAD-X: An Adapter-based Framework for Multi-task Cross-lingual Transfer
Jonas PfeifferIvan VulićIryna GurevychSebastian Ruder
2020-04-30
End-to-End Slot Alignment and Recognition for Cross-Lingual NLU
Weijia XuBatool HaiderSaab Mansour
2020-04-29
Detecting Perceived Emotions in Hurricane Disasters
Shrey DesaiCornelia CarageaJunyi Jessy Li
2020-04-29
Training Curricula for Open Domain Answer Re-Ranking
| Sean MacAvaneyFranco Maria NardiniRaffaele PeregoNicola TonellottoNazli GoharianOphir Frieder
2020-04-29
Analysing Lexical Semantic Change with Contextualised Word Representations
Mario GiulianelliMarco Del TrediciRaquel Fernández
2020-04-29
Do Neural Language Models Show Preferences for Syntactic Formalisms?
Artur KulmizevVinit RavishankarMostafa AbdouJoakim Nivre
2020-04-29
Learning Better Universal Representations from Pre-trained Contextualized Language Models
Yian LiHai Zhao
2020-04-29
Bilingual Text Extraction as Reading Comprehension
Katsuki ChousaMasaaki NagataMasaaki Nishino
2020-04-29
What Happens To BERT Embeddings During Fine-tuning?
Amil MerchantElahe RahimtoroghiEllie PavlickIan Tenney
2020-04-29
Distantly-Supervised Neural Relation Extraction with Side Information using BERT
| Johny MoreiraChaina OliveiraDavid MacêdoCleber ZanchettinLuciano Barbosa
2020-04-29
Revisiting Pre-Trained Models for Chinese Natural Language Processing
| Yiming CuiWanxiang CheTing LiuBing QinShijin WangGuoping Hu
2020-04-29
A Supervised Word Alignment Method based on Cross-Language Span Prediction using Multilingual BERT
Masaaki NagataChousa KatsukiMasaaki Nishino
2020-04-29
Asking without Telling: Exploring Latent Ontologies in Contextual Representations
Julian MichaelJan A. BothaIan Tenney
2020-04-29
TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP
| John X. MorrisEli LiflandJin Yong YooJake GrigsbyDi JinYanjun Qi
2020-04-29
Extending Multilingual BERT to Low-Resource Languages
Zihan WangKarthikeyan KStephen MayhewDan Roth
2020-04-28
Joint Keyphrase Chunking and Salience Ranking with BERT
| Si SunChenyan XiongZhenghao LiuZhiyuan LiuJie Bao
2020-04-28
EARL: Speedup Transformer-based Rankers with Pre-computed Representation
Luyu GaoZhuyun DaiJamie Callan
2020-04-28
VD-BERT: A Unified Vision and Dialog Transformer with BERT
Yue WangShafiq JotyMichael R. LyuIrwin KingCaiming XiongSteven C. H. Hoi
2020-04-28
DomBERT: Domain-oriented Language Model for Aspect-based Sentiment Analysis
Hu XuBing LiuLei ShuPhilip S. Yu
2020-04-28
Kungfupanda at SemEval-2020 Task 12: BERT-Based Multi-Task Learning for Offensive Language Detection
| Wenliang DaiTiezheng YuZihan LiuPascale Fung
2020-04-28
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference
| Ji XinRaphael TangJaejun LeeYaoliang YuJimmy Lin
2020-04-27
ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
Omar KhattabMatei Zaharia
2020-04-27
ColBERT: Using BERT Sentence Embedding for Humor Detection
| Issa Annamoradnejad
2020-04-27
On the Importance of Word and Sentence Representation Learning in Implicit Discourse Relation Classification
| Xin LiuJiefu OuYangqiu SongXin Jiang
2020-04-27
LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning
Kaitao SongHao SunXu TanTao QinJianfeng LuHongzhi LiuTie-Yan Liu
2020-04-27
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models
Mengjie ZhaoTao LinMartin JaggiHinrich Schütze
2020-04-26
Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Document Matching
Liu YangMingyang ZhangCheng LiMichael BenderskyMarc Najork
2020-04-26
Classification of Cuisines from Sequentially Structured Recipes
Tript SharmaUtkarsh UpadhyayGanesh Bagler
2020-04-26
Challenge Closed-book Science Exam: A Meta-learning Based Question Answering System
Xinyue ZhengPeng WangQigang WangZhongchao Shi
2020-04-26
SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check
| Xingyi ChengWeidi XuKunlong ChenShaohua JiangFeng WangTaifeng WangWei ChuYuan Qi
2020-04-26
Quantifying the Contextualization of Word Representations with Semantic Class Probing
Mengjie ZhaoPhilipp DufterYadollah YaghoobzadehHinrich Schütze
2020-04-25
Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order
Yi LiaoXin JiangQun Liu
2020-04-24
Syntactic Data Augmentation Increases Robustness to Inference Heuristics
Junghyun MinR. Thomas McCoyDipanjan DasEmily PitlerTal Linzen
2020-04-24
The Inception Team at NSURL-2019 Task 8: Semantic Question Similarity in Arabic
Hana Al-TheiabatAisha Al-Sadi
2020-04-24
Cross-lingual Information Retrieval with BERT
Zhuolin JiangAmro El-JaroudiWilliam HartmannDamianos KarakosLingjun Zhao
2020-04-24
A Tailored Pre-Training Model for Task-Oriented Dialog Generation
Jing GuQingyang WuChongruo WuWeiyan ShiZhou Yu
2020-04-24
Data Annealing for Informal Language Understanding Tasks
Jing GuZhou Yu
2020-04-24
Contextualized Representations Using Textual Encyclopedic Knowledge
Mandar JoshiKenton LeeYi LuanKristina Toutanova
2020-04-24
Collecting Entailment Data for Pretraining: New Protocols and Negative Results
| Samuel R. BowmanJennimaria PalomakiLivio Baldini SoaresEmily Pitler
2020-04-24
On Adversarial Examples for Biomedical NLP Tasks
Vladimir AraujoAndres CarvalloCarlos AspillagaDenis Parra
2020-04-23
Same Side Stance Classification Task: Facilitating Argument Stance Classification by Fine-tuning a BERT Model
Stefan OllingerLorik DumaniPremtim SahitajRalph BergmannRalf Schenkel
2020-04-23
Self-Attention Attribution: Interpreting Information Interactions Inside Transformer
Yaru HaoLi DongFuru WeiKe Xu
2020-04-23
UHH-LT at SemEval-2020 Task 12: Fine-Tuning of Pre-Trained Transformer Networks for Offensive Language Detection
Gregor WiedemannSeid Muhie YimamChris Biemann
2020-04-23
Keyphrase Prediction With Pre-trained Language Model
Rui LiuZheng LinWeiping Wang
2020-04-22
Learning to Classify Intents and Slot Labels Given a Handful of Examples
Jason KroneYi ZhangMona Diab
2020-04-22
Residual Energy-Based Models for Text Generation
Yuntian DengAnton BakhtinMyle OttArthur SzlamMarc'Aurelio Ranzato
2020-04-22
Attention Module is Not Only a Weight: Analyzing Transformers with Vector Norms
Goro KobayashiTatsuki KuribayashiSho YokoiKentaro Inui
2020-04-21
BERT-ATTACK: Adversarial Attack Against BERT Using BERT
Linyang LiRuotian MaQipeng GuoXiangyang XueXipeng Qiu
2020-04-21
DIET: Lightweight Language Understanding for Dialogue Systems
| Tanja BunkDaksh VarshneyaVladimir VlasovAlan Nichol
2020-04-21
Domain-Guided Task Decomposition with Self-Training for Detecting Personal Events in Social Media
Payam KarisaniJoyce C. HoEugene Agichtein
2020-04-21
Investigating the Effectiveness of Representations Based on Pretrained Transformer-based Language Models in Active Learning for Labelling Text Datasets
Jinghui LuBrian MacNamee
2020-04-21
MPNet: Masked and Permuted Pre-training for Language Understanding
| Kaitao SongXu TanTao QinJianfeng LuTie-Yan Liu
2020-04-20
A Study of Cross-Lingual Ability and Language-specific Information in Multilingual BERT
Chi-Liang LiuTsung-Yuan HsuYung-Sung ChuangHung-Yi Lee
2020-04-20
CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT
| Akshay SmitSaahil JainPranav RajpurkarAnuj PareekAndrew Y. NgMatthew P. Lungren
2020-04-20
Adversarial Training for Large Neural Language Models
| Xiaodong LiuHao ChengPengcheng HeWeizhu ChenYu WangHoifung PoonJianfeng Gao
2020-04-20
StereoSet: Measuring stereotypical bias in pretrained language models
| Moin NadeemAnna BethkeSiva Reddy
2020-04-20
Enhancing Pharmacovigilance with Drug Reviews and Social Media
| Brent BisedaKatie Mo
2020-04-18
Too Many Claims to Fact-Check: Prioritizing Political Claims Based on Check-Worthiness
Yavuz Selim KartalBusra GuvenenMucahid Kutlu
2020-04-17
Fast and Accurate Deep Bidirectional Language Representations for Unsupervised Learning
| Joongbo ShinYoonhyung LeeSeunghyun YoonKyomin Jung
2020-04-17
Learning-to-Rank with BERT in TF-Ranking
Shuguang HanXuanhui WangMike BenderskyMarc Najork
2020-04-17
The Right Tool for the Job: Matching Model and Instance Complexities
| Roy SchwartzGabriel StanovskySwabha SwayamdiptaJesse DodgeNoah A. Smith
2020-04-16
SPECTER: Document-level Representation Learning using Citation-informed Transformers
| Arman CohanSergey FeldmanIz BeltagyDoug DowneyDaniel S. Weld
2020-04-15
lamBERT: Language and Action Learning Using Multimodal BERT
Kazuki MiyazawaTatsuya AokiTakato HoriiTakayuki Nagai
2020-04-15
ToD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogues
| Chien-Sheng WuSteven HoiRichard SocherCaiming Xiong
2020-04-15
Coreferential Reasoning Learning for Language Representation
| Deming YeYankai LinJiaju DuZhenghao LiuMaosong SunZhiyuan Liu
2020-04-15
Training with Quantization Noise for Extreme Model Compression
| Angela FanPierre StockBenjamin GrahamEdouard GraveRemi GribonvalHerve JegouArmand Joulin
2020-04-15
Sentiment Analysis of Yelp Reviews: A Comparison of Techniques and Models
Siqi Liu
2020-04-15
What's so special about BERT's layers? A closer look at the NLP pipeline in monolingual and multilingual models
| Wietse de VriesAndreas van CranenburghMalvina Nissim
2020-04-14
Deep Learning Models for Multilingual Hate Speech Detection
| Sai Saketh AluruBinny MathewPunyajoy SahaAnimesh Mukherjee
2020-04-14
Standardizing and Benchmarking Crisis-related Social Media Datasets for Humanitarian Information Processing
Firoj AlamHassan SajjadMuhammad ImranFerda Ofli
2020-04-14
A Simple Yet Strong Pipeline for HotpotQA
Dirk GroeneveldTushar KhotMausamAshish Sabharwal
2020-04-14
PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation
Bin BiChenliang LiChen WuMing YanWei Wang
2020-04-14
Pretrained Transformers Improve Out-of-Distribution Robustness
Dan HendrycksXiaoyuan LiuEric WallaceAdam DziedzicRishabh KrishnanDawn Song
2020-04-13
Unified Multi-Criteria Chinese Word Segmentation with BERT
Zhen KeLiang ShiErli MengBin WangXipeng QiuXuanjing Huang
2020-04-13
ProFormer: Towards On-Device LSH Projection Based Transformers
Chinnadhurai SankarSujith RaviZornitsa Kozareva
2020-04-13
Cascade Neural Ensemble for Identifying Scientifically Sound Articles
Ashwin Karthik AmbalavananMurthy Devarakonda
2020-04-13
Robustly Pre-trained Neural Model for Direct Temporal Relation Extraction
Hong GuanJianfu LiHua XuMurthy Devarakonda
2020-04-13
Improving Scholarly Knowledge Representation: Evaluating BERT-based Models for Scientific Relation Classification
Ming JiangJennifer D'SouzaSören AuerJ. Stephen Downie
2020-04-13
VGCN-BERT: Augmenting BERT with Graph Embedding for Text Classification
| Zhibin LuPan DuJian-Yun Nie
2020-04-12
Pre-training Text Representations as Meta Learning
Shangwen LvYuechen WangDaya GuoDuyu TangNan DuanFuqing ZhuMing GongLinjun ShouRyan MaDaxin JiangGuihong CaoMing ZhouSonglin Hu
2020-04-12
AMR Parsing via Graph-Sequence Iterative Inference
Deng CaiWai Lam
2020-04-12
LAReQA: Language-agnostic answer retrieval from a multilingual pool
Uma RoyNoah ConstantRami Al-RfouAditya BaruaAaron PhillipsYinfei Yang
2020-04-11
End to End Chinese Lexical Fusion Recognition with Sememe Knowledge
Yijiang LiuMeishan ZhangDonghong Ji
2020-04-11
SimpleTran: Transferring Pre-Trained Sentence Embeddings for Low Resource Text Classification
Siddhant GargRohit Kumar SharmaYingyu Liang
2020-04-10
An In-depth Walkthrough on Evolution of Neural Machine Translation
Rohan JagtapDr. Sudhir N. Dhage
2020-04-10
Telling BERT's full story: from Local Attention to Global Aggregation
Damian PascualGino BrunnerRoger Wattenhofer
2020-04-10
Longformer: The Long-Document Transformer
| Iz BeltagyMatthew E. PetersArman Cohan
2020-04-10
BLEURT: Learning Robust Metrics for Text Generation
Thibault SellamDipanjan DasAnkur P. Parikh
2020-04-09
On the Language Neutrality of Pre-trained Multilingual Representations
Jindřich LibovickýRudolf RosaAlexander Fraser
2020-04-09
Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improve
Oshin AgarwalYinfei YangByron C. WallaceAni Nenkova
2020-04-09
LadaBERT: Lightweight Adaptation of BERT through Hybrid Model Compression
Yihuan MaoYujing WangChufan WuChen ZhangYang WangYaming YangQuanlu ZhangYunhai TongJing Bai
2020-04-08
DynaBERT: Dynamic BERT with Adaptive Width and Depth
Lu HouLifeng ShangXin JiangQun Liu
2020-04-08
Exploiting Redundancy in Pre-trained Language Models for Efficient Transfer Learning
Fahim DalviHassan SajjadNadir DurraniYonatan Belinkov
2020-04-08
Pre-training is a Hot Topic: Contextualized Document Embeddings Improve Topic Coherence
| Federico BianchiSilvia TerragniDirk Hovy
2020-04-08
Poor Man's BERT: Smaller and Faster Transformer Models
| Hassan SajjadFahim DalviNadir DurraniPreslav Nakov
2020-04-08
Improving BERT with Self-Supervised Attention
Xiaoyu KouYaming YangYujing WangCe ZhangYiren ChenYunhai TongYan ZhangJing Bai
2020-04-08
DialBERT: A Hierarchical Pre-Trained Model for Conversation Disentanglement
Tianda LiJia-Chen GuXiaodan ZhuQuan LiuZhen-Hua LingZhiming SuSi Wei
2020-04-08
Severing the Edge Between Before and After: Neural Architectures for Temporal Ordering of Events
Miguel BallesterosRishita AnubhaiShuai WangNima PourdamghaniYogarshi VyasJie MaParminder BhatiaKathleen McKeownYaser Al-Onaizan
2020-04-08
Error-correction and extraction in request dialogs
Stefan ConstantinAlex Waibel
2020-04-08
SciWING -- A Software Toolkit for Scientific Document Processing
| Abhinav Ramesh KashyapMin-Yen Kan
2020-04-08
Inexpensive Domain Adaptation of Pretrained Language Models: Case Studies on Biomedical NER and Covid-19 QA
Nina PoernerUlli WaltingerHinrich Schütze
2020-04-07
Byte Pair Encoding is Suboptimal for Language Model Pretraining
Kaj BostromGreg Durrett
2020-04-07
Transformers to Learn Hierarchical Contexts in Multiparty Dialogue for Span-based Question Answering
| Changmao LiJinho D. Choi
2020-04-07
Towards Non-task-specific Distillation of BERT via Sentence Representation Approximation
Bowen WuHuan ZhangMengyuan LiZongsheng WangQihang FengJunhong HuangBaoxun Wang
2020-04-07
Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition
Paloma JereticAlex WarstadtSuvrat BhooshanAdina Williams
2020-04-07
Information-Theoretic Probing for Linguistic Structure
| Tiago PimentelJosef ValvodaRowan Hall MaudslayRan ZmigrodAdina WilliamsRyan Cotterell
2020-04-07
Towards Evaluating the Robustness of Chinese BERT Classifiers
Boxin WangBoyuan PanXin LiBo Li
2020-04-07
The Russian Drug Reaction Corpus and Neural Models for Drug Reactions and Effectiveness Detection in User Reviews
| Elena TutubalinaIlseyar AlimovaZulfat MiftahutdinovAndrey SakhovskiyValentin MalykhSergey Nikolenko
2020-04-07
Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots
| Jia-Chen GuTianda LiQuan LiuZhen-Hua LingZhiming SuSi WeiXiaodan Zhu
2020-04-07
TextGAIL: Generative Adversarial Imitation Learning for Text Generation
Qingyang WuLei LiZhou Yu
2020-04-07
RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databases
DongHyun ChoiMyeong Cheol ShinEungGyun KimDong Ryeol Shin
2020-04-07
Leveraging the Inherent Hierarchy of Vacancy Titles for Automated Job Ontology Expansion
Jeroen Van HautteVincent SchelstraeteMikaël Wornoo
2020-04-06
Enhancing Review Comprehension with Domain-Specific Commonsense
Aaron TraylorChen ChenBehzad GolshanXiaolan WangYuliang LiYoshihiko SuharaJinfeng LiCagatay DemiralpWang-Chiew Tan
2020-04-06
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
Zhiqing SunHongkun YuXiaodan SongRenjie LiuYiming YangDenny Zhou
2020-04-06
Bootstrapping a Crosslingual Semantic Parser
Tom SherborneYumo XuMirella Lapata
2020-04-06
FastBERT: a Self-distilling BERT with Adaptive Inference Time
| Weijie LiuPeng ZhouZhe ZhaoZhiruo WangHaotang DengQi Ju
2020-04-05
Improved Pretraining for Domain-specific Contextual Embedding Models
Subendhu RongaliAbhyuday JagannathaBhanu Pratap Singh RawatHong Yu
2020-04-05
Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space
| Chunyuan LiXiang GaoYuan LiXiujun LiBaolin PengYizhe ZhangJianfeng Gao
2020-04-05
A Dependency Syntactic Knowledge Augmented Interactive Architecture for End-to-End Aspect-based Sentiment Analysis
| Yunlong LiangFandong MengJinchao ZhangJinan XuYufeng ChenJie Zhou
2020-04-04
CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection
Congying XiaChenwei ZhangHoang NguyenJiawei ZhangPhilip Yu
2020-04-04
Finding Black Cat in a Coal Cellar -- Keyphrase Extraction & Keyphrase-Rubric Relationship Classification from Complex Assignments
| Manikandan Ravikiran
2020-04-03
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
| Yaobo LiangNan DuanYeyun GongNing WuFenfei GuoWeizhen QiMing GongLinjun ShouDaxin JiangGuihong CaoXiaodong FanRuofei ZhangRahul AgrawalEdward CuiSining WeiTaroon BhartiYing QiaoJiun-Hung ChenWinnie WuShuguang LiuFan YangDaniel CamposRangan MajumderMing Zhou
2020-04-03
Testing pre-trained Transformer models for Lithuanian news clustering
Lukas StankevičiusMantas Lukoševičius
2020-04-03
Gestalt: a Stacking Ensemble for SQuAD2.0
Mohamed El-Geish
2020-04-02
Deep Entity Matching with Pre-Trained Language Models
Yuliang LiJinfeng LiYoshihiko SuharaAnHai DoanWang-Chiew Tan
2020-04-01
Towards Productionizing Subjective Search Systems
Aaron FengShuwei ChenYuliang LiHiroshi MatsudaHidekazu TamakiWang-Chiew Tan
2020-03-31
Unification-based Reconstruction of Explanations for Science Questions
| Marco ValentinoMokanarangan ThayaparanAndré Freitas
2020-03-31
Give your Text Representation Models some Love: the Case for Basque
Rodrigo AgerriIñaki San VicenteJon Ander CamposAnder BarrenaXabier SaralegiAitor SoroaEneko Agirre
2020-03-31
InterBERT: An Effective Multi-Modal Pretraining Approach via Vision-and-Language Interaction
Junyang LinAn YangYichang ZhangJie LiuJingren ZhouHongxia Yang
2020-03-30
NukeBERT: A Pre-trained language model for Low Resource Nuclear Domain
Ayush JainMeenachi Ganesamoorty
2020-03-30
Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement
Alireza MohammadshahiJames Henderson
2020-03-29
Abstractive Text Summarization based on Language Model Conditioning and Locality Modeling
Dmitrii AksenovJulián Moreno-SchneiderPeter BourgonjeRobert SchwarzenbergLeonhard HennigGeorg Rehm
2020-03-29
Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining
Chengyu WangMinghui QiuJun HuangXiaofeng He
2020-03-29
User Generated Data: Achilles' Heel of BERT
Ankit KumarPiyush MakhijaAnuj Gupta
2020-03-29
BERT Fine-tuning For Arabic Text Summarization
| Khalid N. ElmadaniMukhtar ElgezouliAnas Showk
2020-03-29
HIN: Hierarchical Inference Network for Document-Level Relation Extraction
Hengzhu TangYanan CaoZhenyu ZhangJiangxia CaoFang FangShi WangPengfei Yin
2020-03-28
Cycle Text-To-Image GAN with BERT
| Trevor TsueSamir SenJason Li
2020-03-26
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
| Kevin ClarkMinh-Thang LuongQuoc V. LeChristopher D. Manning
2020-03-23
Pairwise Multi-Class Document Classification for Semantic Relations between Wikipedia Articles
| Malte OstendorffTerry RuasMoritz SchubotzGeorg RehmBela Gipp
2020-03-22
Beheshti-NER: Persian Named Entity Recognition Using BERT
| Ehsan TaherSeyed Abbas HoseiniMehrnoush Shamsfard
2020-03-19
Temporal Embeddings and Transformer Models for Narrative Text Understanding
Vani KSimone MellaceAlessandro Antonucci
2020-03-19
The value of text for small business default prediction: A deep learning approach
Matthew StevensonChristophe MuesCristián Bravo
2020-03-19
Diversity, Density, and Homogeneity: Quantitative Characteristic Metrics for Text Collections
Yi-An LaiXuan ZhuYi ZhangMona Diab
2020-03-19
X-Stance: A Multilingual Multi-Target Dataset for Stance Detection
| Jannis VamvasRico Sennrich
2020-03-18
Calibration of Pre-trained Transformers
Shrey DesaiGreg Durrett
2020-03-17
Author2Vec: A Framework for Generating User Embedding
Xiaodong WuWeizhe LinZhilin WangElena Rastorgueva
2020-03-17
PO-EMO: Conceptualization, Annotation, and Modeling of Aesthetic Emotions in German and English Poetry
| Thomas HaiderSteffen EgerEvgeny KimRoman KlingerWinfried Menninghaus
2020-03-17
TRANS-BLSTM: Transformer with Bidirectional LSTM for Language Understanding
Zhiheng HuangPeng XuDavis LiangAjay MishraBing Xiang
2020-03-16
Cost-Sensitive BERT for Generalisable Sentence Classification with Imbalanced Data
Harish Tayyar MadabushiElena KochkinaMichael Castelle
2020-03-16
A Survey on Contextual Embeddings
| Qi LiuMatt J. KusnerPhil Blunsom
2020-03-16
Finnish Language Modeling with Deep Transformer Models
Abhilash JainAku RuoheStig-Arne GrönroosMikko Kurimo
2020-03-14
Document Ranking with a Pretrained Sequence-to-Sequence Model
Rodrigo NogueiraZhiying JiangJimmy Lin
2020-03-14
Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking
Samuel Broscheit
2020-03-11
Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings
| Haoran ZhangAmy X. LuMohamed AbdallaMatthew McDermottMarzyeh Ghassemi
2020-03-11
Keyword-Attentive Deep Semantic Matching
| Changyu MiaoZhen CaoYik-Cheung Tam
2020-03-11
Efficient Intent Detection with Dual Sentence Encoders
| Iñigo CasanuevaTadas TemčinasDaniela GerzMatthew HendersonIvan Vulić
2020-03-10
Sensitive Data Detection and Classification in Spanish Clinical Text: Experiments with BERT
Aitor García-PablosNaiara PerezMontse Cuadros
2020-03-06
Transfer Learning for Information Extraction with Limited Data
Minh-Tien NguyenViet-Anh PhanLe Thai LinhNguyen Hong SonLe Tien DungMiku HiranoHajime Hotta
2020-03-06
BERT as a Teacher: Contextual Embeddings for Sequence-Level Reward
Florian SchmidtThomas Hofmann
2020-03-05
What the [MASK]? Making Sense of Language-Specific BERT Models
Debora NozzaFederico BianchiDirk Hovy
2020-03-05
HypoNLI: Exploring the Artificial Patterns of Hypothesis-only Bias in Natural Language Inference
Tianyu LiuXin ZhengBaobao ChangZhifang Sui
2020-03-05
jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models
| Yada PruksachatkunPhil YeresHaokun LiuJason PhangPhu Mon HtutAlex WangIan TenneySamuel R. Bowman
2020-03-04
Data Augmentation using Pre-trained Transformer Models
| Varun KumarAshutosh ChoudharyEunah Cho
2020-03-04
Kleister: A novel task for Information Extraction involving Long Documents with Complex Layout
Filip GralińskiTomasz StanisławekAnna WróblewskaDawid LipińskiAgnieszka KaliskaPaulina RosalskaBartosz TopolskiPrzemysław Biecek
2020-03-04
A Study on Efficiency, Accuracy and Document Structure for Answer Sentence Selection
Daniele BonadimanAlessandro Moschitti
2020-03-04
CLUECorpus2020: A Large-scale Chinese Corpus for Pre-training Language Model
| Liang XuXuanwei ZhangQianqian Dong
2020-03-03
Hierarchical Context Enhanced Multi-Domain Dialogue System for Multi-domain Task Completion
Jingyuan YangGuang LiuYuzhao MaoZhiwei ZhaoWeiguo GaoXuan LiHaiqin YangJianping Shen
2020-03-03
TextBrewer: An Open-Source Knowledge Distillation Toolkit for Natural Language Processing
| Ziqing YangYiming CuiZhipeng ChenWanxiang CheTing LiuShijin WangGuoping Hu
2020-02-28
DC-BERT: Decoupling Question and Document for Efficient Contextual Encoding
Yuyu ZhangPing NieXiubo GengArun RamamurthyLe SongDaxin Jiang
2020-02-28
AraBERT: Transformer-based Model for Arabic Language Understanding
| Wissam AntounFady BalyHazem Hajj
2020-02-28
A Primer in BERTology: What we know about how BERT works
Anna RogersOlga KovalevaAnna Rumshisky
2020-02-27
Compressing Large-Scale Transformer-Based Models: A Case Study on BERT
Prakhar GaneshYao ChenXin LouMohammad Ali KhanYin YangDeming ChenMarianne WinslettHassan SajjadPreslav Nakov
2020-02-27
Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT
Lichao SunKazuma HashimotoWenpeng YinAkari AsaiJia LiPhilip YuCaiming Xiong
2020-02-27
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
| Wenhui WangFuru WeiLi DongHangbo BaoNan YangMing Zhou
2020-02-25
BERT Can See Out of the Box: On the Cross-modal Transferability of Text Representations
Thomas ScialomPatrick BordesPaul-Alexis DrayJacopo StaianoPatrick Gallinari
2020-02-25
Exploring BERT Parameter Efficiency on the Stanford Question Answering Dataset v2.0
Eric Hulburd
2020-02-25
Improving BERT Fine-Tuning via Self-Ensemble and Self-Distillation
| Yige XuXipeng QiuLigao ZhouXuanjing Huang
2020-02-24
Predicting Subjective Features from Questions on QA Websites using BERT
| Issa AnnamoradnejadMohammadamin FazliJafar Habibi
2020-02-24
Predicting Subjective Features from Questions on QA Websites using BERT
| Issa AnnamoradnejadMohammadamin FazliJafar Habibi
2020-02-24
Federated pretraining and fine tuning of BERT using clinical notes from multiple silos
Dianbo LiuTim Miller
2020-02-20
Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learning
Mitchell A. GordonKevin DuhNicholas Andrews
2020-02-19
The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding
| Xiaodong LiuYu WangJianshu JiHao ChengXueyun ZhuEmmanuel AwaPengcheng HeWeizhu ChenHoifung PoonGuihong CaoJianfeng Gao
2020-02-19
From English To Foreign Languages: Transferring Pre-trained Language Models
Ke Tran
2020-02-18
Incorporating BERT into Neural Machine Translation
| Jinhua ZhuYingce XiaLijun WuDi HeTao QinWengang ZhouHouqiang LiTie-Yan Liu
2020-02-17
A Financial Service Chatbot based on Deep Bidirectional Transformers
Shi YuYuxin ChenHussain Zaidi
2020-02-17
The Utility of General Domain Transfer Learning for Medical Language Tasks
Daniel RantiKatie HanssShan ZhaoVarun ArvindJoseph TitanoAnthony CostaEric Oermann
2020-02-16
SBERT-WK: A Sentence Embedding Method by Dissecting BERT-based Word Models
| Bin WangC. -C. Jay Kuo
2020-02-16
UniViLM: A Unified Video and Language Pre-Training Model for Multimodal Understanding and Generation
Huaishao LuoLei JiBotian ShiHaoyang HuangNan DuanTianrui LiXilin ChenMing Zhou
2020-02-15
Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping
| Jesse DodgeGabriel IlharcoRoy SchwartzAli FarhadiHannaneh HajishirziNoah Smith
2020-02-15
Transformer on a Diet
| Chenguang WangZihao YeAston ZhangZheng ZhangAlexander J. Smola
2020-02-14
Understanding patient complaint characteristics using contextual clinical BERT embeddings
Budhaditya SahaSanal LisboaShameek Ghosh
2020-02-14
TwinBERT: Distilling Knowledge to Twin-Structured BERT Models for Efficient Retrieval
| Wenhao LuJian JiaoRuofei Zhang
2020-02-14
Stress Test Evaluation of Transformer-based Models in Natural Language Understanding Tasks
Carlos AspillagaAndrés CarvalloVladimir Araujo
2020-02-14
Training Large Neural Networks with Constant Memory using a New Execution Algorithm
Bharadwaj PudipeddiMaral MesmakhosroshahiJinwen XiSujeeth Bharadwaj
2020-02-13
Utilizing BERT Intermediate Layers for Aspect Based Sentiment Analysis and Natural Language Inference
Youwei SongJiahai WangZhiwei LiangZhiyue LiuTao Jiang
2020-02-12
Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Models
Wangchunshu ZhouKe Xu
2020-02-12
Multilingual Alignment of Contextual Word Representations
Steven CaoNikita KitaevDan Klein
2020-02-10
Momentum Improves Normalized SGD
Ashok CutkoskyHarsh Mehta
2020-02-09
Application of Pre-training Models in Named Entity Recognition
Yu WangYining SunZuchang MaLisheng GaoYang XuTing Sun
2020-02-09
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing
| Canwen XuWangchunshu ZhouTao GeFuru WeiMing Zhou
2020-02-07
Rapid Adaptation of BERT for Information Extraction on Domain-Specific Business Documents
Ruixue ZhangWei YangLuyun LinZhengkai TuYuqing XieZihang FuYuhao XieLuchen TanKun XiongJimmy Lin
2020-02-05
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters
Ruize WangDuyu TangNan DuanZhongyu WeiXuanjing HuangJianshu jiGuihong CaoDaxin JiangMing Zhou
2020-02-05
Interpretable & Time-Budget-Constrained Contextualization for Re-Ranking
| Sebastian HofstätterMarkus ZlabingerAllan Hanbury
2020-02-04
Bertrand-DR: Improving Text-to-SQL using a Discriminative Re-ranker
Amol KelkarRohan RelanVaishali BhardwajSaurabh VaichalPeter Relan
2020-02-03
Beat the AI: Investigating Adversarial Human Annotations for Reading Comprehension
Max BartoloAlastair RobertsJohannes WelblSebastian RiedelPontus Stenetorp
2020-02-02
Fine-Tuning BERT for Schema-Guided Zero-Shot Dialogue State Tracking
Yu-Ping RuanZhen-Hua LingJia-Chen GuQuan Liu
2020-02-01
Pretrained Transformers for Simple Question Answering over Knowledge Graphs
D. LukovnikovA. FischerJ. Lehmann
2020-01-31
Adversarial Training for Aspect-Based Sentiment Analysis with BERT
| Akbar KarimiLeonardo RossiAndrea PratiKatharina Full
2020-01-30
On the Importance of Word Order Information in Cross-lingual Sequence Labeling
Zihan LiuGenta Indra WinataSamuel CahyawijayaAndrea MadottoZhaojiang LinPascale Fung
2020-01-30
PEL-BERT: A Joint Model for Protocol Entity Linking
Shoubin LiWenzao CuiYujiang LiuXuran MingJun HuYuanzheHuQing Wang
2020-01-28
Retrospective Reader for Machine Reading Comprehension
| Zhuosheng ZhangJunjie YangHai Zhao
2020-01-27
Further Boosting BERT-based Models by Duplicating Existing Layers: Some Intriguing Phenomena inside BERT
Wei-Tsung KaoTsung-Han WuPo-Han ChiChun-Cheng HsiehHung-Yi Lee
2020-01-25
Generation-Distillation for Efficient Natural Language Understanding in Low-Data Settings
Luke Melas-KyriaziGeorge HanCeline Liang
2020-01-25
PoWER-BERT: Accelerating BERT Inference via Progressive Word-vector Elimination
| Saurabh GoyalAnamitra R. ChoudhurySaurabh M. RajeVenkatesan T. ChakaravarthyYogish SabharwalAshish Verma
2020-01-24
Navigation-Based Candidate Expansion and Pretrained Language Models for Citation Recommendation
Rodrigo NogueiraZhiying JiangKyunghyun ChoJimmy Lin
2020-01-23
A multimodal deep learning approach for named entity recognition from social media
Meysam Asgari-ChenaghluM. Reza Feizi-DerakhshiLeili FarzinvashM. A. BalafarCina Motamed
2020-01-19
Deep Learning for Hindi Text Classification: A Comparison
Ramchandra JoshiPurvi GoelRaviraj Joshi
2020-01-19
Capturing Evolution in Word Usage: Just Add More Clusters?
Matej MartincSyrielle MontariolElaine ZosaLidia Pivovarova
2020-01-18
RobBERT: a Dutch RoBERTa-based Language Model
| Pieter DelobelleThomas WintersBettina Berendt
2020-01-17
Schema2QA: Answering Complex Queries on the Structured Web with a Neural Model
| Silei XuGiovanni CampagnaJian LiMonica S. Lam
2020-01-16
FGN: Fusion Glyph Network for Chinese Named Entity Recognition
| Zhenyu XuanRui BaoChuyu MaShengyi Jiang
2020-01-15
A BERT based Sentiment Analysis and Key Entity Detection Approach for Online Financial Texts
Lingyun ZhaoLin LiXinhao Zheng
2020-01-14
AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search
Daoyuan ChenYaliang LiMinghui QiuZhen WangBofang LiBolin DingHongbo DengJun HuangWei LinJingren Zhou
2020-01-13
Représentations lexicales pour la détection non supervisée d'événements dans un flux de tweets : étude sur des corpus français et anglais
| Béatrice MazoyerNicolas HervéCéline HudelotJulia Cage
2020-01-13
Exploring and Improving Robustness of Multi Task Deep Neural Networks via Domain Agnostic Defenses
| Kashyap Coimbatore Murali
2020-01-11
Resolving the Scope of Speculation and Negation using Transformer-Based Architectures
Benita Kathleen BrittoAditya Khandelwal
2020-01-09
To Transfer or Not to Transfer: Misclassification Attacks Against Transfer Learned Text Classifiers
Bijeeta PalShruti Tople
2020-01-08
Improving Entity Linking by Modeling Latent Entity Type Information
Shuang ChenJinpeng WangFeng JiangChin-Yew Lin
2020-01-06
Multi-Layer Content Interaction Through Quaternion Product For Visual Question Answering
Lei ShiShijie GengKai ShuangChiori HoriSongxiang LiuPeng GaoSen Su
2020-01-03
Fooling Pre-trained Language Models: An Evolutionary Approach to Generate Wrong Sentences with High Acceptability Score
Anonymous
2020-01-01
BERT for Sequence-to-Sequence Milti-Label Text Classification
Anonymous
2020-01-01
Resolving Lexical Ambiguity in English–Japanese Neural Machine Translation
Anonymous
2020-01-01
Faster and Just As Accurate: A Simple Decomposition for Transformer Models
Anonymous
2020-01-01
Language-independent Cross-lingual Contextual Representations
Anonymous
2020-01-01
Data Annealing Transfer learning Procedure for Informal Language Understanding Tasks
Anonymous
2020-01-01
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators
| Anonymous
2020-01-01
Towards Effective and Efficient Zero-shot Learning by Fine-tuning with Task Descriptions
Anonymous
2020-01-01
Alternating Recurrent Dialog Model with Large-Scale Pre-Trained Language Models
Anonymous
2020-01-01
Improving Neural Language Generation with Spectrum Control
Anonymous
2020-01-01
Robust Instruction-Following in a Situated Agent via Transfer-Learning from Text
Anonymous
2020-01-01
BERT-AL: BERT for Arbitrarily Long Document Understanding
Ruixuan ZhangZhuoyu WeiYu ShiYining Chen
2020-01-01
Building Hierarchical Interpretations in Natural Language via Feature Interaction Detection
Anonymous
2020-01-01
Unsupervised Distillation of Syntactic Information from Contextualized Word Representations
Anonymous
2020-01-01
Generating Biased Datasets for Neural Natural Language Processing
Anonymous
2020-01-01
AutoLR: A Method for Automatic Tuning of Learning Rate
Anonymous
2020-01-01
Neural Symbolic Reader: Scalable Integration of Distributed and Symbolic Representations for Reading Comprehension
Anonymous
2020-01-01
Stacked DeBERT: All Attention in Incomplete Data for Text Classification
| Gwenaelle Cunha SergioMinho Lee
2020-01-01
oLMpics -- On what Language Model Pre-training Captures
Alon TalmorYanai ElazarYoav GoldbergJonathan Berant
2019-12-31
AutoDiscern: Rating the Quality of Online Health Information with Hierarchical Encoder Attention-based Neural Networks
Laura KinkeadAhmed AllamMichael Krauthammer
2019-12-30
Probing the phonetic and phonological knowledge of tones in Mandarin TTS models
| Jian Zhu
2019-12-23
Harnessing Evolution of Multi-Turn Conversations for Effective Answer Retrieval
| Mohammad AliannejadiManajit ChakrabortyEsteban Andrés RíssolaFabio Crestani
2019-12-22
Learning and Evaluating Contextual Embedding of Source Code
| Aditya KanadePetros ManiatisGogul BalakrishnanKensen Shi
2019-12-21
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model
Wenhan XiongJingfei DuWilliam Yang WangVeselin Stoyanov
2019-12-20
Shareable Representations for Search Query Understanding
Mukul KumarYouna HuWill HeaddenRahul GoutamHeran LinBing Yin
2019-12-20
CJRC: A Reliable Human-Annotated Benchmark DataSet for Chinese Judicial Reading Comprehension
Xingyi DuanBaoxin WangZiyue WangWentao MaYiming CuiDayong WuShijin WangTing LiuTianxiang HuoZhen HuHeng WangZhiyuan Liu
2019-12-19
BERTje: A Dutch BERT Model
| Wietse de VriesAndreas van CranenburghArianna BisazzaTommaso CaselliGertjan van NoordMalvina Nissim
2019-12-19
Neural Simile Recognition with Cyclic Multitask Learning and Local Attention
| Jiali ZengLinfeng SongJinsong SuJun XieWei SongJiebo Luo
2019-12-19
A Multi-task Learning Model for Chinese-oriented Aspect Polarity Classification and Aspect Term Extraction
| Heng YangBiqing ZengJianHao YangYouwei SongRuyang Xu
2019-12-17
Cross-Lingual Ability of Multilingual BERT: An Empirical Study
Karthikeyan KZihan WangStephen MayhewDan Roth
2019-12-17
The performance evaluation of Multi-representation in the Deep Learning models for Relation Extraction Task
Jefferson A. Peña TorresRaul Ernesto GutierrezVictor A. BucheliFabio A. Gonzalez O
2019-12-17
Learning Malware Representation based on Execution Sequences
Yi-Ting HuangTing-Yi ChenYeali S. SunMeng Chang Chen
2019-12-16
Robust Named Entity Recognition with Truecasing Pretraining
Stephen MayhewNitish GuptaDan Roth
2019-12-15
Multilingual is not enough: BERT for Finnish
| Antti VirtanenJenna KanervaRami IloJouni LuomaJuhani LuotolahtiTapio SalakoskiFilip GinterSampo Pyysalo
2019-12-15
BERTQA -- Attention on Steroids
Ankit ChadhaRewa Sood
2019-12-14
Towards Robust Toxic Content Classification
Keita KuritaAnna BelovaAntonios Anastasopoulos
2019-12-14
WaLDORf: Wasteless Language-model Distillation On Reading-comprehension
James Yi TianAlexander P. KreuzerPai-Hung ChenHans-Martin Will
2019-12-13
BERT has a Moral Compass: Improvements of ethical and moral values of machines
Patrick SchramowskiCigdem TuranSophie JentzschConstantin RothkopfKristian Kersting
2019-12-11
Unsupervised Transfer Learning via BERT Neuron Selection
Mehrdad ValipourEn-Shiun Annie LeeJaime R. JamacaroCarolina Bessega
2019-12-10
Personalized Patent Claim Generation and Measurement
Jieh-Sheng Lee
2019-12-07
Adversarial Analysis of Natural Language Inference Systems
Tiffany ChienJugal Kalita
2019-12-07
Why ADAM Beats SGD for Attention Models
Jingzhao ZhangSai Praneeth KarimireddyAndreas VeitSeungyeon KimSashank J ReddiSanjiv KumarSuvrit Sra
2019-12-06
Semantic Mask for Transformer based End-to-End Speech Recognition
Chengyi WangYu WuYujiao DuJinyu LiShujie LiuLiang LuShuo RenGuoli YeSheng ZhaoMing Zhou
2019-12-06
Self-Supervised Contextual Language Representation of Radiology Reports to Improve the Identification of Communication Urgency
Xing MengCraig H. GanoeRyan T. SiebergYvonne Y. CheungSaeed Hassanpour
2019-12-05
Acquiring Knowledge from Pre-trained Model to Neural Machine Translation
Rongxiang WengHeng YuShujian HuangShanbo ChengWeihua Luo
2019-12-04
A Comparative Study of Pretrained Language Models on Thai Social Text Categorization
Thanapapas HorsuwanKasidis KanwatcharaPeerapon VateekulBoonserm Kijsirikul
2019-12-03
BERT for Large-scale Video Segment Classification with Test-time Augmentation
Tianqi LiuQizhan Shao
2019-12-02
Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift
Matej MartincPetra Kralj NovakSenja Pollak
2019-12-02
Perceiving the arrow of time in autoregressive motion
Kristof MedingDominik JanzingBernhard SchölkopfFelix A. Wichmann
2019-12-01
Fast and Accurate Stochastic Gradient Estimation
| Beidi ChenYingchen XuAnshumali Shrivastava
2019-12-01
Inducing Relational Knowledge from BERT
Zied BouraouiJose Camacho-ColladosSteven Schockaert
2019-11-28
Do Attention Heads in BERT Track Syntactic Dependencies?
Phu Mon HtutJason PhangShikha BordiaSamuel R. Bowman
2019-11-27
Taking a Stance on Fake News: Towards Automatic Disinformation Assessment via Deep Bidirectional Transformer Language Models for Stance Detection
Chris DulhantyJason L. DeglintIbrahim Ben DayaAlexander Wong
2019-11-27
Evaluating Commonsense in Pre-trained Language Models
| Xuhui ZhouYue ZhangLeyang CuiDandan Huang
2019-11-27
Single Headed Attention RNN: Stop Thinking With Your Head
| Stephen Merity
2019-11-26
Who did They Respond to? Conversation Structure Modeling using Masked Hierarchical Transformer
| Henghui ZhuFeng NanZhiguo WangRamesh NallapatiBing Xiang
2019-11-25
Chemical-protein Interaction Extraction via Gaussian Probability Distribution and External Biomedical Knowledge
| Cong SunZhihao YangLeilei SuLei WangYin ZhangHongfei LinJian Wang
2019-11-21
Automatically Neutralizing Subjective Bias in Text
| Reid PryzantRichard Diehl MartinezNathan DassSadao KurohashiDan JurafskyDiyi Yang
2019-11-21
Joint Emotion Label Space Modelling for Affect Lexica
Luna De BruynePepa AtanasovaIsabelle Augenstein
2019-11-20
Towards non-toxic landscapes: Automatic toxic comment detection using DNN
Ashwin Geet D'SaIrina IllinaDominique Fohr
2019-11-19
Towards Lingua Franca Named Entity Recognition with BERT
Taesun MoonParul AwasthyJian NiRadu Florian
2019-11-19
Improving Relation Classification by Entity Pair Graph
Yi ZhaoHuaiyu WanJianwei GaoYoufang Lin
2019-11-17
Unsupervised Visual Representation Learning with Increasing Object Shape Bias
Zhibo WangShen YanXiaoyu ZhangNiels Lobo
2019-11-17
Robust Reading Comprehension with Linguistic Constraints via Posterior Regularization
Mantong ZhouMinlie HuangXiaoyan Zhu
2019-11-16
Evaluating robustness of language models for chief complaint extraction from patient-generated text
Ilya ValmianskiCaleb GoodwinIan M. FinnNaqi KhanDaniel S. Zisook
2019-11-15
Adapting and evaluating a deep learning language model for clinical why-question answering
Andrew WenMohamed Y. ElwazirSungrim MoonJungwei Fan
2019-11-13
What do you mean, BERT? Assessing BERT as a Distributional Semantics Model
Timothee MickusDenis PapernoMathieu ConstantKees van Deemter
2019-11-13
Unsupervised Domain Adaptation on Reading Comprehension
| Yu CaoMeng FangBaosheng YuJoey Tianyi Zhou
2019-11-13
A Syntax-aware Multi-task Learning Framework for Chinese Semantic Role Labeling
| Qingrong XiaZhenghua LiMin Zhang
2019-11-12
Understanding BERT performance in propaganda analysis
Yiqing Hua
2019-11-11
Attending to Entities for Better Text Understanding
Pengxiang ChengKatrin Erk
2019-11-11
NegBERT: A Transfer Learning Approach for Negation Detection and Scope Resolution
Aditya KhandelwalSuraj Sawant
2019-11-11
Meta Answering for Machine Reading
Benjamin BorschingerJordan Boyd-GraberChristian BuckJannis BulianMassimiliano CiaramitaMichelle Chen HuebscherWojciech GajewskiYannic KilcherRodrigo NogueiraLierni Sestorain Saralegu
2019-11-11
Improving BERT Fine-tuning with Embedding Normalization
Wenxuan ZhouJunyi DuXiang Ren
2019-11-10
Effectiveness of self-supervised pre-training for speech recognition
Alexei BaevskiMichael AuliAbdelrahman Mohamed
2019-11-10
INSET: Sentence Infilling with INter-SEntential Transformer
Yichen HuangYizhe ZhangOussama ElachqarYu Cheng
2019-11-10
Robust Natural Language Inference Models with Example Forgetting
Yadollah YaghoobzadehRemi TachetT. J. HazenAlessandro Sordoni
2019-11-10
YELM: End-to-End Contextualized Entity Linking
Haotian ChenSahil WadhwaXi David LiAndrej Zukov-Gregoric
2019-11-10
Distilling Knowledge Learned in BERT for Text Generation
| Yen-Chun ChenZhe GanYu ChengJingzhou LiuJingjing Liu
2019-11-10
Zero-shot Entity Linking with Dense Entity Retrieval
| Ledell WuFabio PetroniMartin JosifoskiSebastian RiedelLuke Zettlemoyer
2019-11-10
Syntax-Infused Transformer and BERT models for Machine Translation and Natural Language Understanding
Dhanasekar SundararamanVivek SubramanianGuoyin WangShijing SiDinghan ShenDong WangLawrence Carin
2019-11-10
RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers
| Bailin WangRichard ShinXiaodong LiuOleksandr PolozovMatthew Richardson
2019-11-10
E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT
Nina PoernerUlli WaltingerHinrich Schütze
2019-11-09
The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents
Kurt ShusterDa JuStephen RollerEmily DinanY-Lan BoureauJason Weston
2019-11-09
ConveRT: Efficient and Accurate Conversational Representations from Transformers
| Matthew HendersonIñigo CasanuevaNikola MrkšićPei-Hao SuTsung-Hsien WenIvan Vulić
2019-11-09
MKD: a Multi-Task Knowledge Distillation Approach for Pretrained Language Models
Linqing LiuHuan WangJimmy LinRichard SocherCaiming Xiong
2019-11-09
Multi-Perspective Inferrer: Reasoning Sentences Relationship from Holistic Perspective
Zhen ChengZaixiang ZhengXin-Yu DaiShujian HuangJiajun Chen
2019-11-09
Transforming Wikipedia into Augmented Data for Query-Focused Summarization
Haichao ZhuLi DongFuru WeiBing QinTing Liu
2019-11-08
How Language-Neutral is Multilingual BERT?
Jindřich LibovickýRudolf RosaAlexander Fraser
2019-11-08
What Would Elsa Do? Freezing Layers During Transformer Fine-Tuning
Jaejun LeeRaphael TangJimmy Lin
2019-11-08
Cross-Lingual Relevance Transfer for Document Retrieval
Peng ShiJimmy Lin
2019-11-08
Graph-to-Graph Transformer for Transition-based Dependency Parsing
Alireza MohammadshahiJames Henderson
2019-11-08
Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Models
Xisen JinZhongyu WeiJunyi DuXiangyang XueXiang Ren