Search Results for author:
Found 7 papers, 4 papers with code
Standardizing the Measurement of Text Diversity: A Tool and a Comparative Analysis of Scores
The applicability of scores extends beyond analysis of generative models; for example, we highlight applications on instruction-tuning datasets and human-produced texts.
Future Lens: Anticipating Subsequent Tokens from a Single Hidden State
More concretely, in this paper we ask: Given a hidden (internal) representation of a single token at position $t$ in an input, can we reliably anticipate the tokens that will appear at positions $\geq t + 2$?
Evaluating the Zero-shot Robustness of Instruction-tuned Language Models
To answer the former, we collect a set of 319 instructions manually written by NLP practitioners for over 80 unique tasks included in widely used benchmarks, and we evaluate the variance and average performance of these instructions as compared to instruction phrasings observed during instruction fine-tuning.
Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction
In this paper, we present the first benchmark that simulates the evaluation of open information extraction models in the real world, where the syntactic and expressive distributions under the same knowledge meaning may drift variously.
EDUKG: a Heterogeneous Sustainable K-12 Educational Knowledge Graph
To tackle these issues, we propose EDUKG, a heterogeneous sustainable K-12 Educational Knowledge Graph.
Unveiling the Black Box of PLMs with Semantic Anchors: Towards Interpretable Neural Semantic Parsing
The recent prevalence of pretrained language models (PLMs) has dramatically shifted the paradigm of semantic parsing, where the mapping from natural language utterances to structured logical forms is now formulated as a Seq2Seq task.
GraphQ IR: Unifying the Semantic Parsing of Graph Query Languages with One Intermediate Representation
Subject to the huge semantic gap between natural and formal languages, neural semantic parsing is typically bottlenecked by its complexity of dealing with both input semantics and output syntax.
