Linguistic Acceptability
47 papers with code • 5 benchmarks • 5 datasets
Linguistic Acceptability is the task of determining whether a sentence is grammatical or ungrammatical.
Image Source: Warstadt et al
Benchmarks
These leaderboards are used to track progress in Linguistic Acceptability
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Linguistic Acceptability models and implementations
GLUE
CoLA
ItaCoLA
RuCoLA
Latest papers
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
huggingface/transformers-bloom-inference
•
•
We develop a procedure for Int8 matrix multiplication for feed-forward and attention projection layers in transformers, which cut the memory needed for inference by half while retaining full precision performance.
Acceptability Judgements via Examining the Topology of Attention Maps
The role of the attention mechanism in encoding linguistic knowledge has received special interest in NLP.
VALUE: Understanding Dialect Disparity in NLU
To understand disparities in current models and to facilitate more dialect-competent NLU systems, we introduce the VernAcular Language Understanding Evaluation (VALUE) benchmark, a challenging variant of GLUE that we created with a set of lexical and morphosyntactic transformation rules.
data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language
huggingface/transformers
•
•
Preprint 2022
While the general idea of self-supervised learning is identical across modalities, the actual algorithms and objectives differ widely because they were developed with a single modality in mind.
Monolingual and Cross-Lingual Acceptability Judgments with the Italian CoLA corpus
The development of automated approaches to linguistic acceptability has been greatly fostered by the availability of the English CoLA corpus, which has also been included in the widely used GLUE benchmark.
General Cross-Architecture Distillation of Pretrained Language Models into Matrix Embeddings
lgalke/cross-architecture-distillation
•
•
We match or exceed the scores of ELMo for all tasks of the GLUE benchmark except for the sentiment analysis task SST-2 and the linguistic acceptability task CoLA.
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization
google-research/google-research
•
•
ICLR 2022
In this paper, we propose a new model inductive bias that learns a subword tokenization end-to-end as part of the model.
Language Models Use Monotonicity to Assess NPI Licensing
jumelet/monotonicity-npi-lm
•
•
Findings (ACL) 2021
We investigate the semantic knowledge of language models (LMs), focusing on (1) whether these LMs create categories of linguistic environments based on their semantic monotonicity properties, and (2) whether these categories play a similar role in LMs as in human language understanding, using negative polarity item licensing as a case study.
FNet: Mixing Tokens with Fourier Transforms
labmlai/annotated_deep_learning_paper_implementations
•
•
NAACL 2022
At longer input lengths, our FNet model is significantly faster: when compared to the "efficient" Transformers on the Long Range Arena benchmark, FNet matches the accuracy of the most accurate models, while outpacing the fastest models across all sequence lengths on GPUs (and across relatively shorter lengths on TPUs).
Entailment as Few-Shot Learner
Large pre-trained language models (LMs) have demonstrated remarkable ability as few-shot learners.
