Search Results for author:
Found 10 papers, 7 papers with code
Do Efficient Transformers Really Save Computation?
Our results show that while these models are expressive enough to solve general DP tasks, contrary to expectations, they require a model size that scales with the problem size.
Beyond Weisfeiler-Lehman: A Quantitative Framework for GNN Expressiveness
Specifically, we identify a fundamental expressivity measure termed homomorphism expressivity, which quantifies the ability of GNN models to count graphs under homomorphism.
Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective
By using circuit complexity theory, we first give impossibility results showing that bounded-depth Transformers are unable to directly produce correct answers for basic arithmetic/equation tasks unless the model size grows super-polynomially with respect to the input length.
A Complete Expressiveness Hierarchy for Subgraph GNNs via Subgraph Weisfeiler-Lehman Tests
Recently, subgraph GNNs have emerged as an important direction for developing expressive graph neural networks (GNNs).
Ranked #1 on
Subgraph Counting - C6
on Synthetic Graph
Rethinking the Expressive Power of GNNs via Graph Biconnectivity
In this paper, we take a fundamentally different perspective to study the expressive power of GNNs beyond the WL test.
Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspective
Designing neural networks with bounded Lipschitz constant is a promising way to obtain certifiably robust classifiers against adversarial examples.
Non-convex Distributionally Robust Optimization: Non-asymptotic Analysis
Distributionally robust optimization (DRO) is a widely-used approach to learn models that are robust against distribution shift.
Boosting the Certified Robustness of L-infinity Distance Nets
Recently, Zhang et al. (2021) developed a new neural network architecture based on $\ell_\infty$-distance functions, which naturally possesses certified $\ell_\infty$ robustness by its construction.
Towards Certifying L-infinity Robustness using Neural Networks with L-inf-dist Neurons
This directly provides a rigorous guarantee of the certified robustness based on the margin of prediction outputs.
Improved Analysis of Clipping Algorithms for Non-convex Optimization
Gradient clipping is commonly used in training deep neural networks partly due to its practicability in relieving the exploding gradient problem.
