Search Results for author:
Found 4 papers, 4 papers with code
Why are hyperbolic neural networks effective? A study on hierarchical representation capability
Inspired by the analysis results, we propose several pre-training strategies to enhance HRC and improve the performance of downstream tasks, further validating the reliability of the analysis.
GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model
Currently, the reduction in the parameter scale of large-scale pre-trained language models (PLMs) through knowledge distillation has greatly facilitated their widespread deployment on various devices.
Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method
To address these problems, we propose a general language model distillation (GLMD) method that performs two-stage word prediction distillation and vocabulary compression, which is simple and surprisingly shows extremely strong performance.
Coherence-Based Distributed Document Representation Learning for Scientific Documents
In this paper, we propose a coupled text pair embedding (CTPE) model to learn the representation of scientific documents, which maintains the coherence of the document with coupled text pairs formed by segmenting the document.
