Search Results for author:
Found 19 papers, 5 papers with code
Wave Physics-informed Matrix Factorizations
With the recent success of representation learning methods, which includes deep learning as a special case, there has been considerable interest in developing techniques that incorporate known physical constraints into the learned representation.
White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is?
This leads to a family of white-box transformer-like deep network architectures, named CRATE, which are mathematically fully interpretable.
White-Box Transformers via Sparse Rate Reduction
Particularly, we show that the standard transformer block can be derived from alternating optimization on complementary parts of this objective: the multi-head self-attention operator can be viewed as a gradient descent step to compress the token sets by minimizing their lossy coding rate, and the subsequent multi-layer perceptron can be viewed as attempting to sparsify the representation of the tokens.
Variational Information Pursuit for Interpretable Predictions
We then demonstrate that the IP strategy is the optimal solution to this problem.
Unsupervised Manifold Linearizing and Clustering
We consider the problem of simultaneously clustering and learning a linear representation of data lying close to a union of low-dimensional manifolds, a fundamental task in machine learning and computer vision.
Learning Globally Smooth Functions on Manifolds
To do so, it shows that under typical conditions the problem of learning a Lipschitz continuous function on a manifold is equivalent to a dynamically weighted manifold regularization problem.
Interpretable by Design: Learning Predictors by Composing Interpretable Queries
There is a growing concern about typically opaque decision-making with high-performance machine learning algorithms.
Efficient Maximal Coding Rate Reduction by Variational Forms
The principle of Maximal Coding Rate Reduction (MCR$^2$) has recently been proposed as a training objective for learning discriminative low-dimensional structures intrinsic to high-dimensional data to allow for more robust training than standard approaches, such as cross-entropy minimization.
Implicit Bias of Projected Subgradient Method Gives Provable Robust Recovery of Subspaces of Unknown Codimension
Robust subspace recovery (RSR) is a fundamental problem in robust representation learning.
Wave-Informed Matrix Factorization with Global Optimality Guarantees
With the recent success of representation learning methods, which includes deep learning as a special case, there has been considerable interest in developing representation learning techniques that can incorporate known physical constraints into the learned representation.
Doubly Stochastic Subspace Clustering
Many state-of-the-art subspace clustering methods follow a two-step process by first constructing an affinity matrix between data points and then applying spectral clustering to this affinity.
Ranked #1 on
Image Clustering
on UMist
A Critique of Self-Expressive Deep Subspace Clustering
To extend this approach to data supported on a union of non-linear manifolds, numerous studies have proposed learning an embedding of the original data using a neural network which is regularized by a self-expressive loss function on the data in the embedded space to encourage a union of linear subspaces prior on the data in the embedded space.
On the Regularization Properties of Structured Dropout
We also show that the global minimizer for DropBlock can be computed in closed form, and that DropConnect is equivalent to Dropout.
Global Optimality in Separable Dictionary Learning with Applications to the Analysis of Diffusion MRI
In the classical setting, signals are represented as vectors and the dictionary learning problem is posed as a matrix factorization problem where the data matrix is approximately factorized into a dictionary matrix and a sparse matrix of coefficients.
Multi-Cell Detection and Classification Using a Generative Convolutional Model
This paper proposes a new approach to detecting, counting and classifying white blood cell populations in holographic images, which capitalizes on the fact that the variability in a mixture of blood cells is constrained by physiology.
An Analysis of Dropout for Matrix Factorization
While the resulting regularizer is closely related to a variational form of the nuclear norm, suggesting that dropout may limit the size of the factorization, we show that it is possible to trivially lower the objective value by doubling the size of the factorization.
Structured Low-Rank Matrix Factorization: Global Optimality, Algorithms, and Applications
Recently, convex formulations of low-rank matrix factorization problems have received considerable attention in machine learning.
Global Optimality in Neural Network Training
The past few years have seen a dramatic increase in the performance of recognition systems thanks to the introduction of deep networks for representation learning.
Global Optimality in Tensor Factorization, Deep Learning, and Beyond
Techniques involving factorization are found in a wide range of applications and have enjoyed significant empirical success in many fields.
