Search Results for author:
Found 41 papers, 31 papers with code
{\mathcal{P}^2}: A Plan-and-Pretrain Approach for Knowledge Graph-to-Text Generation
Text verbalization of knowledge graphs is an important problem with wide application to natural language generation (NLG) systems.
InternLM2 Technical Report
The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI).
Ranked #5 on
Long-Context Understanding
on Ada-LEval (BestAnswer)
A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond
Building on our examination of the developmental trajectories, we further investigate the emerging synergies between code intelligence and broader machine intelligence, uncovering new cross-domain opportunities and illustrating the substantial influence of code intelligence across various domains.
NovelQA: A Benchmark for Long-Range Novel Question Answering
The rapid advancement of Large Language Models (LLMs) has introduced a new frontier in natural language processing, particularly in understanding and processing long-context information.
Benchmarking Hallucination in Large Language Models based on Unanswerable Math Word Problem
This paper presents a new method for evaluating LLM hallucination in Question Answering (QA) based on the unanswerable math word problem (MWP).
Data-freeWeight Compress and Denoise for Large Language Models
Significantly, our method is characterized by without necessitating additional involvement of any corpus, while simultaneously preserving orthogonality in conjunction with pruning and quantization methods.
LongWanjuan: Towards Systematic Measurement for Long Text Quality
The quality of training data are crucial for enhancing the long-text capabilities of foundation models.
Identifying Semantic Induction Heads to Understand In-Context Learning
Although large language models (LLMs) have demonstrated remarkable performance, the lack of transparency in their inference logic raises concerns about their trustworthiness.
Code Needs Comments: Enhancing Code LLMs with Comment Augmentation
The programming skill is one crucial ability for Large Language Models (LLMs), necessitating a deep understanding of programming languages (PLs) and their correlation with natural languages (NLs).
Turn Waste into Worth: Rectifying Top-$k$ Router of MoE
To address the dropped tokens and padding, we propose the Rectify-Router, comprising the Intra-GPU Rectification and the Fill-in Rectification.
F-Eval: Asssessing Fundamental Abilities with Refined Evaluation Methods
However, these evaluation benchmarks are limited to assessing the instruction-following capabilities, overlooking the fundamental abilities that emerge during the pre-training stage.
Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication
Large Language Models (LLMs) have recently made significant strides in complex reasoning tasks through the Chain-of-Thought technique.
CoLLiE: Collaborative Training of Large Language Models in an Efficient Way
This paper introduces CoLLiE, an efficient library that facilitates collaborative training of large language models using 3D parallelism, parameter-efficient fine-tuning (PEFT) methods, and optimizers such as Lion, Adan, Sophia, LOMO and AdaLomo.
Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus
Large Language Models (LLMs) have gained significant popularity for their impressive performance across diverse fields.
Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughts
As large language models (LLMs) have shown effectiveness with different prompting methods, such as Chain of Thought, Program of Thought, we find that these methods have formed a great complementarity to each other on math reasoning tasks.
StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding
Analogy-making between narratives is crucial for human reasoning.
AdaLomo: Low-memory Optimization with Adaptive Learning Rate
Building on this insight, we introduce the low-memory optimization with adaptive learning rate (AdaLomo), which offers an adaptive learning rate for each parameter.
Full Parameter Fine-tuning for Large Language Models with Limited Resources
Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP) but demand massive GPU resources for training.
An AMR-based Link Prediction Approach for Document-level Event Argument Extraction
Motivated by the fact that all event structures can be inferred from AMR, this work reformulates EAE as a link prediction problem on AMR graphs.
Do Large Language Models Know What They Don't Know?
Large language models (LLMs) have a wealth of knowledge that allows them to excel in various Natural Language Processing (NLP) tasks.
Exploiting Abstract Meaning Representation for Open-Domain Question Answering
The Open-Domain Question Answering (ODQA) task involves retrieving and subsequently generating answers from fine-grained relevant passages within a database.
All Roads Lead to Rome? Exploring the Invariance of Transformers' Representations
Transformer models bring propelling advances in various NLP tasks, thus inducing lots of interpretability research on the learned representations of the models.
Evaluating Open-QA Evaluation
This study focuses on the evaluation of the Open Question Answering (Open-QA) task, which can directly estimate the factuality of large language models (LLMs).
Word-Level Representation From Bytes For Language Modeling
Based on this observation, we rethink the existing character-aware method that takes character-level inputs but makes word-level sequence modeling and prediction.
RLET: A Reinforcement Learning Based Approach for Explainable QA with Entailment Trees
RLET iteratively performs single step reasoning with sentence selection and deduction generation modules, from which the training signal is accumulated across the tree with elaborately designed aligned reward function that is consistent with the evaluation.
DORE: Document Ordered Relation Extraction based on Generative Framework
In this paper, we investigate the root cause of the underwhelming performance of the existing generative DocRE models and discover that the culprit is the inadequacy of the training paradigm, instead of the capacities of the models.
What Dense Graph Do You Need for Self-Attention?
Transformers have made progress in miscellaneous tasks, but suffer from quadratic computational and memory complexities.
Dialogue Meaning Representation for Task-Oriented Dialogue Systems
We propose Dialogue Meaning Representation (DMR), a pliable and easily extendable representation for task-oriented dialogue.
Towards Collaborative Question Answering: A Preliminary Study
Knowledge and expertise in the real-world can be disjointedly owned.
A Unified Generative Framework for Various NER Subtasks
To that end, we propose to formulate the NER subtasks as an entity span sequence generation task, which can be solved by a unified sequence-to-sequence (Seq2Seq) framework.
Ranked #10 on
Nested Named Entity Recognition
on GENIA
Fork or Fail: Cycle-Consistent Training with Many-to-One Mappings
Cycle-consistent training is widely used for jointly learning a forward and inverse mapping between two domains of interest without the cumbersome requirement of collecting matched pairs within each domain.
GenWiki: A Dataset of 1.3 Million Content-Sharing Text and Graphs for Unsupervised Graph-to-Text Generation
With a human-annotated test set, we provide this new benchmark dataset for future research on unsupervised text generation from knowledge graphs.
Ranked #1 on
Unsupervised KG-to-Text Generation
on GenWiki (Fine)
CoLAKE: Contextualized Language and Knowledge Embedding
With the emerging branch of incorporating factual knowledge into pre-trained language models such as BERT, most existing models consider shallow, static, and separately pre-trained entity embeddings, which limits the performance gains of these models.
CycleGT: Unsupervised Graph-to-Text and Text-to-Graph Generation via Cycle Training
Due to the difficulty and high cost of data collection, the supervised data available in the two fields are usually on the magnitude of tens of thousands, for example, 18K in the WebNLG~2017 dataset after preprocessing, which is far fewer than the millions of data for other tasks such as machine translation.
BERT-ATTACK: Adversarial Attack Against BERT Using BERT
Adversarial attacks for discrete data (such as texts) have been proved significantly more challenging than continuous data (such as images) since it is difficult to generate adversarial samples with gradient-based methods.
Multi-Scale Self-Attention for Text Classification
In this paper, we introduce the prior knowledge, multi-scale structure, into self-attention modules.
Joint Parsing and Generation for Abstractive Summarization
If generating a word can introduce an erroneous relation to the summary, the behavior must be discouraged.
Ranked #27 on
Text Summarization
on GigaWord
BP-Transformer: Modelling Long-Range Context via Binary Partitioning
The Transformer model is widely successful on many natural language processing tasks.
Ranked #1 on
Machine Translation
on IWSLT2015 Chinese-English
Star-Transformer
Although Transformer has achieved great successes on many NLP tasks, its heavy structure with fully-connected attention connections leads to dependencies on large training data.
Ranked #13 on
Sentiment Analysis
on SST-5 Fine-grained classification
Named Entity Recognition (NER)
Natural Language Inference
+2
Top-Down Tree Structured Text Generation
Text generation is a fundamental building block in natural language processing tasks.
First Step toward Model-Free, Anonymous Object Tracking with Recurrent Neural Networks
In this paper, we propose and study a novel visual object tracking approach based on convolutional networks and recurrent networks.
