Search Results for author:
Found 6 papers, 1 papers with code
Towards Understanding the Word Sensitivity of Attention Layers: A Study via Random Features
Unveiling the reasons behind the exceptional success of transformers requires a better understanding of why attention layers are suitable for NLP tasks.
Stability, Generalization and Privacy: Precise Analysis for Random and NTK Features
Deep learning models can be vulnerable to recovery attacks, raising privacy concerns to users, and widespread algorithms such as empirical risk minimization (ERM) often do not directly enforce safety guarantees.
Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernels
However, this "universal" law provides only a necessary condition for robustness, and it is unable to discriminate between models.
Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization
The Neural Tangent Kernel (NTK) has emerged as a powerful tool to provide memorization, optimization and generalization guarantees in deep neural networks.
Sharp asymptotics on the compression of two-layer neural networks
In this paper, we study the compression of a target two-layer neural network with N nodes into a compressed network with M<N nodes.
Towards Differential Relational Privacy and its use in Question Answering
While bounding general memorization can have detrimental effects on the performance of a trained model, bounding RM does not prevent effective learning.
