Search Results for author:
Found 25 papers, 16 papers with code
Aligning language models with human preferences
In Chapter 3, I investigate the relation between two approaches to finetuning pretrained LMs using feedback given by a scoring function: reinforcement learning from human feedback (RLHF) and distribution matching.
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs).
Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
The proliferation of generative models, combined with pretraining on web-scale data, raises a timely question: what happens when these models are trained on their own generated outputs?
Towards Understanding Sycophancy in Language Models
Overall, our results indicate that sycophancy is a general behavior of state-of-the-art AI assistants, likely driven in part by human preference judgments favoring sycophantic responses.
Compositional preference models for aligning LMs
As language models (LMs) become more capable, it is increasingly important to align them with human preferences.
The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"
If a model is trained on a sentence of the form "A is B", it will not automatically generalize to the reverse direction "B is A".
Taken out of context: On measuring situational awareness in LLMs
At test time, we assess whether the model can pass the test.
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals.
Inverse Scaling: When Bigger Isn't Better
Here, we present evidence for the claim that LMs may show inverse scaling, or worse task performance with increased scale, e. g., due to flaws in the training objective and data.
Improving Code Generation by Training with Natural Language Feedback
The potential for pre-trained large language models (LLMs) to use natural language feedback at inference time has been an exciting recent development.
Training Language Models with Language Feedback at Scale
Third, finetuning the language model to maximize the likelihood of the chosen refinement given the input.
Models of symbol emergence in communication: a conceptual review and a guide for avoiding local minima
Computational simulations are a popular method for testing hypotheses about the emergence of communication.
Aligning Language Models with Preferences through f-divergence Minimization
We show that Jensen-Shannon divergence strikes a good balance between these objectives, and frequently outperforms forward KL divergence by a wide margin, leading to significant improvements over prior work.
Pretraining Language Models with Human Preferences
Language models (LMs) are pretrained to imitate internet text, including content that would violate human preferences if generated by an LM: falsehoods, offensive comments, personally identifiable information, low-quality or buggy code, and more.
On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgetting
Here we explore the theoretical connections between the two paradigms, and show that methods such as KL-control developed for RM can also be construed as belonging to DM.
RL with KL penalties is better viewed as Bayesian inference
We show that KL-regularised RL is equivalent to variational inference: approximating a Bayesian posterior which specifies how to update a prior LM to conform with evidence provided by the reward function.
A continuity of Markov blanket interpretations under the Free Energy Principle
Bruineberg and colleagues helpfully distinguish between instrumental and ontological interpretations of Markov blankets, exposing the dangers of using the former to make claims about the latter.
Controlling Conditional Language Models without Catastrophic Forgetting
Machine learning is shifting towards general-purpose pretrained generative models, trained in a self-supervised manner on large amounts of data, which can then be applied to solve a large number of tasks.
Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication
Communication is compositional if complex signals can be represented as a combination of simpler subparts.
On Reward Maximization and Distribution Matching for Fine-Tuning Language Models
The availability of large pre-trained models is changing the landscape of Machine Learning research and practice, moving from a "training from scratch" to a "fine-tuning'' paradigm.
Energy-Based Models for Code Generation under Compilability Constraints
Neural language models can be successfully trained on source code, leading to applications such as code completion.
Measuring non-trivial compositionality in emergent communication
Compositionality is an important explanatory target in emergent communication and language evolution.
Developmentally motivated emergence of compositional communication via template transfer
This paper explores a novel approach to achieving emergent compositional communication in multi-agent systems.
Exploiting Unsupervised Pre-training and Automated Feature Engineering for Low-resource Hate Speech Detection in Polish
This paper presents our contribution to PolEval 2019 Task 6: Hate speech and bullying detection.
Fine-tuning Tree-LSTM for phrase-level sentiment classification on a Polish dependency treebank. Submission to PolEval task 2
We describe a variant of Child-Sum Tree-LSTM deep neural network (Tai et al, 2015) fine-tuned for working with dependency trees and morphologically rich languages using the example of Polish.
