Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models

1 code implementation • 23 Oct 2023 • Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosselut, Mrinmaya Sachan

We show that MechanisticProbe is able to detect the information of the reasoning tree from the model's attentions for most examples, suggesting that the LM indeed is going through a process of multi-step reasoning within its architecture in many cases.

Mitigating Label Biases for In-context Learning

1 code implementation • 28 May 2023 • Yu Fei, Yifan Hou, Zeming Chen, Antoine Bosselut

In this work, we define a typology for three types of label biases in ICL for text classification: vanilla-label bias, context-label bias, and domain-label bias (which we conceptualize and detect for the first time).

In-Context Learning text-classification +1

Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations

1 code implementation • 29 Oct 2022 • Yu Fei, Ping Nie, Zhao Meng, Roger Wattenhofer, Mrinmaya Sachan

We further explore the applicability of our clustering approach by evaluating it on 14 datasets with more diverse topics, text lengths, and numbers of classes.

Clustering Sentence +7

Network-based recommendation algorithms: A review

no code implementations • 19 Nov 2015 • Yu Fei, Zeng An, Gillard Sebastien, Medo Matus

Recommender systems are a vital tool that helps us to overcome the information overload problem.

Recommendation Systems