Search Results for author:
Found 7 papers, 3 papers with code
Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning
We propose end-to-end (e2e) sparse dictionary learning, a method for training SAEs that ensures the features learned are functionally important by minimizing the KL divergence between the output distributions of the original model and the model with SAE activations inserted.
Black-Box Access is Insufficient for Rigorous AI Audits
External audits of AI systems are increasingly recognized as a key mechanism for AI governance.
Sparse Autoencoders Find Highly Interpretable Features in Language Models
One hypothesised cause of polysemanticity is \textit{superposition}, where neural networks represent more features than they have neurons by assigning features to an overcomplete set of directions in activation space, rather than to individual neurons.
A technical note on bilinear layers for interpretability
The ability of neural networks to represent more features than neurons makes interpreting them challenging.
Circumventing interpretability: How to defeat mind-readers
The increasing capabilities of artificial intelligence (AI) systems make it ever more important that we interpret their internals to ensure that their intentions are aligned with human values.
Interpreting Neural Networks through the Polytope Lens
Previous mechanistic descriptions have used individual neurons or their linear combinations to understand the representations a network has learned.
Goal Misgeneralization in Deep Reinforcement Learning
We study goal misgeneralization, a type of out-of-distribution generalization failure in reinforcement learning (RL).
