Search Results for author:
Found 9 papers, 6 papers with code
The Information of Large Language Model Geometry
This paper investigates the information encoded in the embeddings of large language models (LLMs).
Large Language Model Evaluation via Matrix Entropy
Large language models (LLMs) have revolutionized the field of natural language processing, extending their strong capabilities into multi-modal domains.
Understanding Grokking Through A Robustness Viewpoint
Recently, an interesting phenomenon called grokking has gained much attention, where generalization occurs long after the models have initially overfitted the training data.
OTMatch: Improving Semi-Supervised Learning with Optimal Transport
In this paper, we present a new approach called OTMatch, which leverages semantic relationships among classes by employing an optimal transport loss function.
Information Flow in Self-Supervised Learning
In this paper, we provide a comprehensive toolbox for understanding and enhancing self-supervised learning (SSL) methods through the lens of matrix information theory.
Matrix Information Theory for Self-Supervised Learning
Inspired by this framework, we introduce Matrix-SSL, a novel approach that leverages matrix information theory to interpret the maximum entropy encoding loss as matrix uniformity loss.
Ranked #1 on
Contrastive Learning
on imagenet-1k
RelationMatch: Matching In-batch Relationships for Semi-supervised Learning
Semi-supervised learning has achieved notable success by leveraging very few labeled data and exploiting the wealth of information derived from unlabeled data.
SEAL: Simultaneous Label Hierarchy Exploration And Learning
Label hierarchy is an important source of external knowledge that can enhance classification performance.
Contrastive Learning Is Spectral Clustering On Similarity Graph
Contrastive learning is a powerful self-supervised learning method, but we have a limited theoretical understanding of how it works and why it works.
