Knowledge Probing
21 papers with code • 6 benchmarks • 3 datasets
Benchmarks
These leaderboards are used to track progress in Knowledge Probing
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
PopQA
BioLAMA
BEAR-probe
Most implemented papers
Calibrating Factual Knowledge in Pretrained Language Models
dqxiu/calinet
•
•
However, we find that facts stored in the PLMs are not always correct.
COPEN: Probing Conceptual Knowledge in Pre-trained Language Models
thu-keg/copen
•
•
We believe this is a critical bottleneck for realizing human-like cognition in PLMs.
Galactica: A Large Language Model for Science
We believe these results demonstrate the potential for language models as a new interface for science.
Injecting Domain Knowledge in Language Models for Task-Oriented Dialogue Systems
amazon-research/domain-knowledge-injection
•
•
Pre-trained language models (PLM) have advanced the state-of-the-art across NLP applications, but lack domain-specific knowledge that does not naturally occur in pre-training data.
When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
alextmallen/adaptive-retrieval
•
•
Despite their impressive performance on diverse tasks, large language models (LMs) still struggle with tasks requiring rich world knowledge, implying the limitations of relying solely on their parameters to encode a wealth of world knowledge.
Is BERT Blind? Exploring the Effect of Vision-and-Language Pretraining on Visual Language Understanding
TAU-VAILab/isbertblind
•
•
CVPR 2023
We show that SOTA multimodally trained text encoders outperform unimodally trained text encoders on the VLU tasks while being underperformed by them on the NLU tasks, lending new context to previously mixed results regarding the NLU capabilities of multimodal models.
LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Development
To this end, we release a multinational English legal corpus (LeXFiles) and a legal knowledge probing benchmark (LegalLAMA) to facilitate training and detailed analysis of legal-oriented PLMs.
Using Large Language Models for Knowledge Engineering (LLMKE): A Case Study on Wikidata
In this work, we explore the use of Large Language Models (LLMs) for knowledge engineering tasks in the context of the ISWC 2023 LM-KBC Challenge.
Assessing the Reliability of Large Language Model Knowledge
Large language models (LLMs) have been treated as knowledge bases due to their strong performance in knowledge probing tasks.
PromptCBLUE: A Chinese Prompt Tuning Benchmark for the Medical Domain
Biomedical language understanding benchmarks are the driving forces for artificial intelligence applications with large language model (LLM) back-ends.
