Search Results for author:
Found 7 papers, 1 papers with code
PIDformer: Transformer Meets Control Theory
Motivated by this control framework, we derive a novel class of transformers, PID-controlled Transformer (PIDformer), aimed at improving robustness and mitigating the rank-collapse issue inherent in softmax transformers.
Mitigating Over-smoothing in Transformers via Regularized Nonlocal Functionals
Transformers have achieved remarkable success in a wide range of natural language processing and computer vision applications.
p-Laplacian Transformer
$p$-Laplacian regularization, rooted in graph and image signal processing, introduces a parameter $p$ to control the regularization effect on these data.
Transformer with Fourier Integral Attentions
Multi-head attention empowers the recent success of transformers, the state-of-the-art models that have achieved remarkable success in sequence modeling and beyond.
Improving Transformers with Probabilistic Attention Keys
Inspired by this observation, we propose Transformer with a Mixture of Gaussian Keys (Transformer-MGK), a novel transformer architecture that replaces redundant heads in transformers with a mixture of keys at each head.
Active Learning in Incomplete Label Multiple Instance Multiple Label Learning
Due to the partial availability of bag-level labels, we focus on the incomplete-label MIML setting for the proposed active learning approach.
DeepUSPS: Deep Robust Unsupervised Saliency Prediction via Self-supervision
Alternative unsupervised approaches rely on careful selection of multiple handcrafted saliency methods to generate noisy pseudo-ground-truth labels.
