Detecting Hallucination and Coverage Errors in Retrieval Augmented Generation for Controversial Topics

no code implementations • 13 Mar 2024 • Tyler A. Chang, Katrin Tomanek, Jessica Hoffmann, Nithum Thain, Erin Van Liemt, Kathleen Meier-Hellstern, Lucas Dixon

We explore a strategy to handle controversial topics in LLM-based chatbots based on Wikipedia's Neutral Point of View (NPOV) principle: acknowledge the absence of a single true answer and surface multiple perspectives.

Hallucination Retrieval +1

Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generation

no code implementations • 14 Feb 2024 • Jessica Quaye, Alicia Parrish, Oana Inel, Charvi Rastogi, Hannah Rose Kirk, Minsuk Kahng, Erin Van Liemt, Max Bartolo, Jess Tsang, Justin White, Nathan Clement, Rafael Mosquera, Juan Ciro, Vijay Janapa Reddi, Lora Aroyo

By focusing on ``implicitly adversarial'' prompts (those that trigger T2I models to generate unsafe images for non-obvious reasons), we isolate a set of difficult safety issues that human creativity is well-suited to uncover.

Text-to-Image Generation

Discipline and Label: A WEIRD Genealogy and Social Theory of Data Annotation

no code implementations • 9 Feb 2024 • Andrew Smart, Ding Wang, Ellis Monk, Mark Díaz, Atoosa Kasirzadeh, Erin Van Liemt, Sonja Schmer-Galunder

Data annotation remains the sine qua non of machine learning and AI.

Fairness