Search Results for author:
Found 18 papers, 10 papers with code
Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions
In this paper, we investigate the interplay between attention heads and specialized "next-token" neurons in the Multilayer Perceptron that predict specific tokens.
Increasing Trust in Language Models through the Reuse of Verified Circuits
To exhibit the reusability of verified modules, we insert the trained integer addition model into an untrained model and train the combined model to perform both addition and subtraction.
Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it).
Large Language Models Relearn Removed Concepts
This demonstrates that models exhibit polysemantic capacities and can blend old and new concepts in individual neurons.
Measuring Value Alignment
As artificial intelligence (AI) systems become increasingly integrated into various domains, ensuring that they align with human values becomes critical.
Interpreting Shared Circuits for Ordered Sequence Prediction in a Large Language Model
While transformer models exhibit strong capabilities on linguistic tasks, their complex architectures make them difficult to interpret.
Understanding Addition in Transformers
Understanding the inner workings of machine learning models like Transformers is vital for their safe and ethical use.
Beyond Training Objectives: Interpreting Reward Model Divergence in Large Language Models
Large language models (LLMs) fine-tuned by reinforcement learning from human feedback (RLHF) are becoming more widely deployed.
AI Systems of Concern
Concerns around future dangers from advanced AI often centre on systems hypothesised to have intrinsic characteristics such as agent-like behaviour, strategic awareness, and long-range planning.
DeepDecipher: Accessing and Investigating Neuron Activation in Large Language Models
By granting access to state-of-the-art interpretability methods, DeepDecipher makes LLMs more transparent, trustworthy, and safe.
Neuron to Graph: Interpreting Language Model Neurons at Scale
Conventional methods require examination of examples with strong neuron activation and manual identification of patterns to decipher the concepts a neuron responds to.
Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark
We use this improved benchmark to evaluate recent model editing techniques and find that they suffer from low specificity.
The Larger They Are, the Harder They Fail: Language Models do not Recognize Identifier Swaps in Python
Large Language Models (LLMs) have successfully been applied to code generation tasks, raising the question of how well these models understand programming.
System III: Learning with Domain Knowledge for Safety Constraints
We evaluate the satisfaction of these constraints via p-norms in state vector space.
N2G: A Scalable Approach for Quantifying Interpretable Neuron Representations in Large Language Models
Understanding the function of individual neurons within language models is essential for mechanistic interpretability research.
Fairness in AI and Its Long-Term Implications on Society
Successful deployment of artificial intelligence (AI) in various settings has led to numerous positive outcomes for individuals and society.
Exploring the Advantages of Transformers for High-Frequency Trading
This paper explores the novel deep learning Transformers architectures for high-frequency Bitcoin-USDT log-return forecasting and compares them to the traditional Long Short-Term Memory models.
PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaboration
However, we reveal sub-optimal collaborative behaviors also emerge with strong correlations, and simply maximizing the MI can, surprisingly, hinder the learning towards better collaboration.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
