Search Results for author:
Found 39 papers, 17 papers with code
CatCode: A Comprehensive Evaluation Framework for LLMs On the Mixture of Code and Text
Large language models (LLMs) such as ChatGPT are increasingly proficient in understanding and generating a mixture of code and text.
Autonomous Data Selection with Language Models for Mathematical Texts
Our method showcases a 2 times increase in pretraining token efficiency compared to state-of-the-art baselines, underscoring the potential of our approach in enhancing models' mathematical reasoning capabilities.
Meta Prompting for AI Systems
In this work, we present a comprehensive study of Meta Prompting (MP), an innovative technique reshaping the utilization of language models (LMs) and AI systems in problem-solving and data interaction.
MatChat: A Large Language Model and Application Service Platform for Materials Science
The prediction of chemical synthesis pathways plays a pivotal role in materials science research.
Information Flow in Self-Supervised Learning
In this paper, we provide a comprehensive toolbox for understanding and enhancing self-supervised learning (SSL) methods through the lens of matrix information theory.
Cumulative Reasoning with Large Language Models
We demonstrate CR's superiority through several complex reasoning tasks: it outperforms existing methods in logical inference tasks with up to a 9. 3% improvement, achieving 98. 04% accuracy on the curated FOLIO wiki dataset.
Ranked #3 on
Math Word Problem Solving
on MATH
Matrix Information Theory for Self-Supervised Learning
Inspired by this framework, we introduce Matrix-SSL, a novel approach that leverages matrix information theory to interpret the maximum entropy encoding loss as matrix uniformity loss.
Ranked #1 on
Contrastive Learning
on imagenet-1k
RelationMatch: Matching In-batch Relationships for Semi-supervised Learning
Semi-supervised learning has achieved notable success by leveraging very few labeled data and exploiting the wealth of information derived from unlabeled data.
On Uni-Modal Feature Learning in Supervised Multi-Modal Learning
We abstract the features (i. e. learned representations) of multi-modal data into 1) uni-modal features, which can be learned from uni-modal training, and 2) paired features, which can only be learned from cross-modal interactions.
Contrastive Learning Is Spectral Clustering On Similarity Graph
Contrastive learning is a powerful self-supervised learning method, but we have a limited theoretical understanding of how it works and why it works.
Succinct Representations for Concepts
Foundation models like chatGPT have demonstrated remarkable performance on various tasks.
On the Power of Foundation Models
The second one says fine tuning does not have this limit, as a foundation model with the minimum required power (up to symmetry) can theoretically solve downstream tasks for the category defined by pretext task, with fine tuning and enough resources.
Predictive Inference with Feature Conformal Prediction
Conformal prediction is a distribution-free technique for establishing valid prediction intervals.
Anomaly Detection with Test Time Augmentation and Consistency Evaluation
Deep neural networks are known to be vulnerable to unseen data: they may wrongly assign high confidence stcores to out-distribuion samples.
Modality Laziness: Everybody's Business is Nobody's Business
We name this problem of multi-modal training, \emph{Modality Laziness}.
Towards Understanding Generalization via Decomposing Excess Risk Dynamics
Generalization is one of the fundamental issues in machine learning.
T-SCI: A Two-Stage Conformal Inference Algorithm with Guaranteed Coverage for Cox-MLP
It is challenging to deal with censored data, where we only have access to the incomplete information of survival time instead of its exact value.
Imbalance Robust Softmax for Deep Embeeding Learning
IR-Softmax can generalise to any softmax and its variants (which are discriminative for open-set problem) by directly setting the weights as their class centers, naturally solving the data imbalance problem.
Secure Data Sharing With Flow Model
In the classical multi-party computation setting, multiple parties jointly compute a function without revealing their own input data.
Inject Machine Learning into Significance Test for Misspecified Linear Models
First, we apply a machine learning method to fit the ground truth function on the training set and calculate its linear approximation.
Adversarial Data Encryption
In the big data era, many organizations face the dilemma of data sharing.
Learning-Based Low-Rank Approximations
Our experiments show that, for multiple types of data sets, a learned sketch matrix can substantially reduce the approximation loss compared to a random matrix $S$, sometimes by one order of magnitude.
Neural Embeddings for Nearest Neighbor Search Under Edit Distance
The edit distance between two sequences is an important metric with many applications.
$\ell_1$ Adversarial Robustness Certificates: a Randomized Smoothing Approach
Robustness is an important property to guarantee the security of machine learning models.
Tight Certificates of Adversarial Robustness for Randomly Smoothed Classifiers
Specifically, an $\ell_2$ bounded adversary cannot alter the ensemble prediction generated by an additive isotropic Gaussian noise, where the radius for the adversary depends on both the variance of the distribution as well as the ensemble margin at the point of interest.
Asymmetric Valleys: Beyond Sharp and Flat Local Minima
Specifically, at a local minimum there exist many asymmetric directions such that the loss increases abruptly along one side, and slowly along the opposite side--we formally define such minima as asymmetric valleys.
Expanding Holographic Embeddings for Knowledge Completion
Neural models operating over structured spaces such as knowledge graphs require a continuous embedding of the discrete elements of this space (such as entities) as well as the relationships between them.
Deep Multi-Task Learning to Recognise Subtle Facial Expressions of Mental States
To advance subtle expression recognition, we contribute a Large-scale Subtle Emotions and Mental States in the Wild database (LSEMSW).
An empirical study on evaluation metrics of generative adversarial networks
Evaluating generative adversarial networks (GANs) is inherently challenging.
An Alternative View: When Does SGD Escape Local Minima?
Stochastic gradient descent (SGD) is widely used in machine learning.
Attribute-Enhanced Face Recognition With Neural Tensor Fusion Networks
To solve this problem, we establish a theoretical equivalence between tensor optimisation and a two-stream gated neural network.
Hyperparameter Optimization: A Spectral Approach
In particular, we obtain the first quasi-polynomial time algorithm for learning noisy decision trees with polynomial sample complexity.
Convergence Analysis of Two-layer Neural Networks with ReLU Activation
We also show that the identity mapping is necessary for convergence, as it moves the initial point to a better place for optimization.
Exploiting the Structure: Stochastic Gradient Methods Using Raw Clusters
The amount of data available in the world is growing faster than our ability to deal with it.
Even Faster Accelerated Coordinate Descent Using Non-Uniform Sampling
Accelerated coordinate descent is widely used in optimization due to its cheap per-iteration cost and scalability to large-scale problems.
Improved SVRG for Non-Strongly-Convex or Sum-of-Non-Convex Objectives
Many classical algorithms are found until several years later to outlive the confines in which they were conceived, and continue to be relevant in unforeseen settings.
Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition
To the best of our knowledge this is the first work that gives global convergence guarantees for stochastic gradient descent on non-convex functions with exponentially many local minima and saddle points.
Combinatorial Multi-Armed Bandit and Its Extension to Probabilistically Triggered Arms
The objective of an online learning algorithm for CMAB is to minimize (\alpha,\beta)-approximation regret, which is the difference between the \alpha{\beta} fraction of the expected reward when always playing the optimal super arm, and the expected reward of playing super arms according to the algorithm.
