Search Results for author: Sharon Levy

Found 20 papers, 11 papers with code

Lost in Translation? Translation Errors and Challenges for Fair Assessment of Text-to-Image Models on Multilingual Concepts

no code implementations17 Mar 2024 Michael Saxon, Yiran Luo, Sharon Levy, Chitta Baral, Yezhou Yang, William Yang Wang

Benchmarks of the multilingual capabilities of text-to-image (T2I) models compare generated images prompted in a test language to an expected image distribution over a concept set.

Translation

Evaluating Biases in Context-Dependent Health Questions

1 code implementation7 Mar 2024 Sharon Levy, Tahilin Sanchez Karver, William D. Adler, Michelle R. Kaufman, Mark Dredze

We study how large language model biases are exhibited through these contextual questions in the healthcare domain.

Language Modelling Large Language Model

ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Models

1 code implementation14 Oct 2023 Alex Mei, Sharon Levy, William Yang Wang

As large language models are integrated into society, robustness toward a suite of prompts is increasingly important to maintain reliability in a high-variance environment. Robustness evaluations must comprehensively encapsulate the various settings in which a user may invoke an intelligent system.

Foveate, Attribute, and Rationalize: Towards Physically Safe and Trustworthy AI

1 code implementation19 Dec 2022 Alex Mei, Sharon Levy, William Yang Wang

Users' physical safety is an increasing concern as the market for intelligent systems continues to grow, where unconstrained systems may recommend users dangerous actions that can lead to serious injury.

Attribute

WikiWhy: Answering and Explaining Cause-and-Effect Questions

no code implementations21 Oct 2022 Matthew Ho, Aditya Sharma, Justin Chang, Michael Saxon, Sharon Levy, Yujie Lu, William Yang Wang

As large language models (LLMs) grow larger and more sophisticated, assessing their "reasoning" capabilities in natural language grows more challenging.

Question Answering

SafeText: A Benchmark for Exploring Physical Safety in Language Models

no code implementations18 Oct 2022 Sharon Levy, Emily Allaway, Melanie Subbiah, Lydia Chilton, Desmond Patton, Kathleen McKeown, William Yang Wang

Understanding what constitutes safe text is an important issue in natural language processing and can often prevent the deployment of models deemed harmful and unsafe.

Text Generation

Mitigating Covertly Unsafe Text within Natural Language Systems

no code implementations17 Oct 2022 Alex Mei, Anisha Kabir, Sharon Levy, Melanie Subbiah, Emily Allaway, John Judge, Desmond Patton, Bruce Bimber, Kathleen McKeown, William Yang Wang

An increasingly prevalent problem for intelligent technologies is text safety, as uncontrolled systems may generate recommendations to their users that lead to injury or life-threatening consequences.

Towards Understanding Gender-Seniority Compound Bias in Natural Language Generation

1 code implementation LREC 2022 Samhita Honnavalli, Aesha Parekh, Lily Ou, Sophie Groenwold, Sharon Levy, Vicente Ordonez, William Yang Wang

Our results show that GPT-2 amplifies bias by considering women as junior and men as senior more often than the ground truth in both domains.

Text Generation

HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Data

no code implementations Findings (ACL) 2022 Kai Nakamura, Sharon Levy, Yi-Lin Tuan, Wenhu Chen, William Yang Wang

A pressing challenge in current dialogue systems is to successfully converse with users on topics with information distributed across different modalities.

Response Generation Retrieval

Addressing Issues of Cross-Linguality in Open-Retrieval Question Answering Systems For Emergent Domains

1 code implementation26 Jan 2022 Alon Albalak, Sharon Levy, William Yang Wang

Open-retrieval question answering systems are generally trained and tested on large datasets in well-established domains.

Question Answering Retrieval +1

Modeling Disclosive Transparency in NLP Application Descriptions

1 code implementation EMNLP 2021 Michael Saxon, Sharon Levy, Xinyi Wang, Alon Albalak, William Yang Wang

Broader disclosive transparency$-$truth and clarity in communication regarding the function of AI systems$-$is widely considered desirable.

Fairness Language Modelling +1

Investigating African-American Vernacular English in Transformer-Based Text Generation

1 code implementation EMNLP 2020 Sophie Groenwold, Lily Ou, Aesha Parekh, Samhita Honnavalli, Sharon Levy, Diba Mirza, William Yang Wang

The growth of social media has encouraged the written use of African American Vernacular English (AAVE), which has traditionally been used only in oral contexts.

Text Generation

Fakeddit: A New Multimodal Benchmark Dataset for Fine-grained Fake News Detection

no code implementations LREC 2020 Kai Nakamura, Sharon Levy, William Yang Wang

We construct hybrid text+image models and perform extensive experiments for multiple variations of classification, demonstrating the importance of the novel aspect of multimodality and fine-grained classification unique to Fakeddit.

Classification Cultural Vocal Bursts Intensity Prediction +2

Evaluating Transformer-Based Multilingual Text Classification

no code implementations29 Apr 2020 Sophie Groenwold, Samhita Honnavalli, Lily Ou, Aesha Parekh, Sharon Levy, Diba Mirza, William Yang Wang

As NLP tools become ubiquitous in today's technological landscape, they are increasingly applied to languages with a variety of typological structures.

General Classification Language Modelling +4

r/Fakeddit: A New Multimodal Benchmark Dataset for Fine-grained Fake News Detection

3 code implementations10 Nov 2019 Kai Nakamura, Sharon Levy, William Yang Wang

We construct hybrid text+image models and perform extensive experiments for multiple variations of classification, demonstrating the importance of the novel aspect of multimodality and fine-grained classification unique to Fakeddit.

Classification Cultural Vocal Bursts Intensity Prediction +2

SafeRoute: Learning to Navigate Streets Safely in an Urban Environment

1 code implementation3 Nov 2018 Sharon Levy, Wenhan Xiong, Elizabeth Belding, William Yang Wang

We propose SafeRoute, a novel solution to the problem of navigating cities and avoiding street harassment and crime.

Navigate Representation Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.