Search Results for author:
Found 6 papers, 5 papers with code
Cross-Architecture Transfer Learning for Linear-Cost Inference Transformers
Motivated by this approach, we propose Cross-Architecture Transfer Learning (XATL), in which the weights of the shared components between LCI and self-attention-based transformers, such as layernorms, MLPs, input/output embeddings, are directly transferred to the new architecture from already pre-trained model parameters.
AbsInstruct: Eliciting Abstraction Ability from LLMs through Explanation Tuning with Plausibility Estimation
Abstraction ability is crucial in human intelligence, which can also benefit various tasks in NLP study.
AbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graph
Cognitive research indicates that abstraction ability is essential in human intelligence, which remains under-explored in language models.
KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detection
Large Language Models (LLMs) have demonstrated remarkable human-level natural language generation capabilities.
CKBP v2: An Expert-Annotated Evaluation Set for Commonsense Knowledge Base Population
Populating Commonsense Knowledge Bases (CSKB) is an important yet hard task in NLP, as it tackles knowledge from external sources with unseen events and entities.
Benchmarking Commonsense Knowledge Base Population with an Effective Evaluation Dataset
Experimental results show that generalizing commonsense reasoning on unseen assertions is inherently a hard task.
