Search Results for author: Xiaoyan Bai

Found 3 papers, 2 papers with code

Learn To be Efficient: Build Structured Sparsity in Large Language Models

no code implementations9 Feb 2024 Haizhong Zheng, Xiaoyan Bai, Beidi Chen, Fan Lai, Atul Prakash

The emergence of activation sparsity in LLMs provides a natural approach to reduce this cost by involving only parts of the parameters for inference.

Llama Text Generation

A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity

1 code implementation3 Jan 2024 Andrew Lee, Xiaoyan Bai, Itamar Pres, Martin Wattenberg, Jonathan K. Kummerfeld, Rada Mihalcea

While alignment algorithms are now commonly used to tune pre-trained language models towards a user's preferences, we lack explanations for the underlying mechanisms in which models become ``aligned'', thus making it difficult to explain phenomena like jailbreaks.

Language Modelling

Cannot find the paper you are looking for? You can Submit a new open access paper.