Large-Scale Quantitative Evaluation of Dialogue Agents’ Response Strategies against Offensive Users

1 code implementation • SIGDIAL (ACL) 2021 • Haojun Li, Dilara Soylu, Christopher Manning

As voice assistants and dialogue agents grow in popularity, so does the abuse they receive.

Building Efficient and Effective OpenQA Systems for Low-Resource Languages

1 code implementation • 7 Jan 2024 • Emrah Budur, Rıza Özçelik, Dilara Soylu, Omar Khattab, Tunga Güngör, Christopher Potts

We present SQuAD-TR, a machine translation of SQuAD2. 0, and we build our OpenQA system by adapting ColBERT-QA for Turkish.

Machine Translation Question Answering

Ecosystem Graphs: The Social Footprint of Foundation Models

no code implementations • 28 Mar 2023 • Rishi Bommasani, Dilara Soylu, Thomas I. Liao, Kathleen A. Creel, Percy Liang

Foundation models (e. g. ChatGPT, StableDiffusion) pervasively influence society, warranting immediate social attention.

Holistic Evaluation of Language Models

1 code implementation • 16 Nov 2022 • Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, Yuta Koreeda

We present Holistic Evaluation of Language Models (HELM) to improve the transparency of language models.

Fairness Question Answering

Neural Generation Meets Real People: Building a Social, Informative Open-Domain Dialogue Agent

1 code implementation • SIGDIAL (ACL) 2022 • Ethan A. Chi, Ashwin Paranjape, Abigail See, Caleb Chiam, Trenton Chang, Kathleen Kenealy, Swee Kiat Lim, Amelia Hardy, Chetanya Rastogi, Haojun Li, Alexander Iyabor, Yutong He, Hari Sowrirajan, Peng Qi, Kaushik Ram Sadagopan, Nguyet Minh Phu, Dilara Soylu, Jillian Tang, Avanika Narayan, Giovanni Campagna, Christopher D. Manning

We present Chirpy Cardinal, an open-domain social chatbot.

Chatbot

Neural Generation Meets Real People: Towards Emotionally Engaging Mixed-Initiative Conversations

no code implementations • 27 Aug 2020 • Ashwin Paranjape, Abigail See, Kathleen Kenealy, Haojun Li, Amelia Hardy, Peng Qi, Kaushik Ram Sadagopan, Nguyet Minh Phu, Dilara Soylu, Christopher D. Manning

At the end of the competition, Chirpy Cardinal progressed to the finals with an average rating of 3. 6/5. 0, a median conversation duration of 2 minutes 16 seconds, and a 90th percentile duration of over 12 minutes.

World Knowledge