Search Results for author:
Found 9 papers, 7 papers with code
Nonparametric Modern Hopfield Models
We present a nonparametric construction for deep learning compatible modern Hopfield models and utilize this framework to debut an efficient variant.
BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model
We introduce the \textbf{B}i-Directional \textbf{S}parse \textbf{Hop}field Network (\textbf{BiSHop}), a novel end-to-end framework for deep tabular learning.
Outlier-Efficient Hopfield Layers for Large Transformer-Based Models
Interestingly, this memory model manifests a model-based interpretation of an outlier-efficient attention mechanism ($\text{Softmax}_1$): it is an approximation of the memory retrieval process of $\mathtt{OutEffHop}$.
Ranked #1 on
Quantization
on Wiki-40B
Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models
Specifically, we accomplish this by constructing a separation loss $\mathcal{L}_\Phi$ that separates the local minima of kernelized energy by separating stored memory patterns in kernel space.
On Computational Limits of Modern Hopfield Models: A Fine-Grained Complexity Analysis
Specifically, we establish an upper bound criterion for the norm of input query patterns and memory patterns.
Beyond PID Controllers: PPO with Neuralized PID Policy for Proton Beam Intensity Control in Mu2e
We introduce a novel Proximal Policy Optimization (PPO) algorithm aimed at addressing the challenge of maintaining a uniform proton beam intensity delivery in the Muon to Electron Conversion Experiment (Mu2e) at Fermi National Accelerator Laboratory (Fermilab).
STanHop: Sparse Tandem Hopfield Model for Memory-Enhanced Time Series Prediction
We present STanHop-Net (Sparse Tandem Hopfield Network) for multivariate time series prediction with memory-enhanced capabilities.
On Sparse Modern Hopfield Model
Building upon this, we derive the sparse memory retrieval dynamics from the sparse energy function and show its one-step approximation is equivalent to the sparse-structured attention.
Feature Programming for Multivariate Time Series Prediction
We introduce the concept of programmable feature engineering for time series modeling and propose a feature programming framework.
