Search Results for author:
Found 16 papers, 10 papers with code
Recurrent Event Network: Autoregressive Structure Inferenceover Temporal Knowledge Graphs
The task becomes more challenging on temporal knowledge graphs, where each fact is associated with a timestamp.
What Will My Model Forget? Forecasting Forgotten Examples in Language Model Refinement
We propose a partially interpretable forecasting model based on the observation that changes in pre-softmax logit scores of pretraining examples resemble that of online learned examples, which performs decently on BART but fails on T5 models.
Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning
Real-life multilingual systems should be able to efficiently incorporate new languages as data distributions fed to the system evolve and shift over time.
Dataless Knowledge Fusion by Merging Weights of Language Models
In this paper, we study the problem of merging individual models built on different training data sets to obtain a single model that performs well both across all data set domains and can generalize on out-of-domain data.
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora
We evaluate PTLM's ability to adapt to new corpora while retaining learned knowledge in earlier corpora.
Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning
The ability to continuously expand knowledge over time and utilize it to rapidly generalize to new tasks is a key feature of human linguistic intelligence.
Refining Language Models with Compositional Explanations
However, such a regularization technique lacks flexibility and coverage, since only importance scores towards a pre-defined list of features are adjusted, while more complex human knowledge such as feature interaction and pattern generalization can hardly be incorporated.
Efficient Learning of Less Biased Models with Transfer Learning
Prediction bias in machine learning models, referring to undesirable model behaviors that discriminates inputs mentioning or produced by certain group, has drawn increasing attention from the research community given its societal impact.
On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning
Fine-tuned language models have been shown to exhibit biases against protected groups in a host of modeling tasks such as text classification and coreference resolution.
Gradient-based Editing of Memory Examples for Online Task-free Continual Learning
We explore task-free continual learning (CL), in which a model is trained to avoid catastrophic forgetting in the absence of explicit task boundaries or identities.
Contextualizing Hate Speech Classifiers with Post-hoc Explanation
Hate speech classifiers trained on imbalanced datasets struggle to determine if group identifiers like "gay" or "black" are used in offensive or prejudiced ways.
Visually Grounded Continual Learning of Compositional Phrases
To study this human-like language acquisition ability, we present VisCOLL, a visually grounded language learning task, which simulates the continual acquisition of compositional phrases from streaming visual scenes.
Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Models
Human and metrics evaluation on both LSTM models and BERT Transformer models on multiple datasets show that our algorithms outperform prior hierarchical explanation algorithms.
Recurrent Event Network: Autoregressive Structure Inference over Temporal Knowledge Graphs
The task becomes more challenging on temporal knowledge graphs, where each fact is associated with a timestamp.
Explicit State Tracking with Semi-Supervision for Neural Dialogue Generation
However, the \emph{expensive nature of state labeling} and the \emph{weak interpretability} make the dialogue state tracking a challenging problem for both task-oriented and non-task-oriented dialogue generation: For generating responses in task-oriented dialogues, state tracking is usually learned from manually annotated corpora, where the human annotation is expensive for training; for generating responses in non-task-oriented dialogues, most of existing work neglects the explicit state tracking due to the unlimited number of dialogue states.
Sequicity: Simplifying Task-oriented Dialogue Systems with Single Sequence-to-Sequence Architectures
Existing solutions to task-oriented dialogue systems follow pipeline designs which introduces architectural complexity and fragility.
