Search Results for author: Jiao Sun

Found 22 papers, 12 papers with code

AESOP: Paraphrase Generation with Adaptive Syntactic Control

1 code implementation EMNLP 2021 Jiao Sun, Xuezhe Ma, Nanyun Peng

We propose to control paraphrase generation through carefully chosen target syntactic structures to generate more proper and higher quality paraphrases.

Data Augmentation Language Modelling +2

ESTER: A Machine Reading Comprehension Dataset for Reasoning about Event Semantic Relations

no code implementations EMNLP 2021 Rujun Han, I-Hung Hsu, Jiao Sun, Julia Baylon, Qiang Ning, Dan Roth, Nanyun Peng

While these tasks partially evaluate machines’ ability of narrative understanding, human-like reading comprehension requires the capability to process event-based information beyond arguments and temporal reasoning.

Machine Reading Comprehension Natural Language Queries +1

Rich Human Feedback for Text-to-Image Generation

1 code implementation15 Dec 2023 Youwei Liang, Junfeng He, Gang Li, Peizhao Li, Arseniy Klimovskiy, Nicholas Carolan, Jiao Sun, Jordi Pont-Tuset, Sarah Young, Feng Yang, Junjie Ke, Krishnamurthy Dj Dvijotham, Katie Collins, Yiwen Luo, Yang Li, Kai J Kohlhoff, Deepak Ramachandran, Vidhya Navalpakkam

We show that the predicted rich human feedback can be leveraged to improve image generation, for example, by selecting high-quality training data to finetune and improve the generative models, or by creating masks with predicted heatmaps to inpaint the problematic regions.

Text-to-Image Generation

DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedback

no code implementations29 Nov 2023 Jiao Sun, Deqing Fu, Yushi Hu, Su Wang, Royi Rassin, Da-Cheng Juan, Dana Alon, Charles Herrmann, Sjoerd van Steenkiste, Ranjay Krishna, Cyrus Rashtchian

Then, it uses two VLMs to select the best generation: a Visual Question Answering model that measures the alignment of generated images to the text, and another that measures the generation's aesthetic quality.

Question Answering Text-to-Image Generation +1

Evaluating Large Language Models on Controlled Generation Tasks

1 code implementation23 Oct 2023 Jiao Sun, Yufei Tian, Wangchunshu Zhou, Nan Xu, Qian Hu, Rahul Gupta, John Frederick Wieting, Nanyun Peng, Xuezhe Ma

While recent studies have looked into the abilities of large language models in various benchmark tasks, including question generation, reading comprehension, multilingual and etc, there have been few studies looking into the controllability of large language models on generation tasks.

Question Generation Question-Generation +2

"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters

1 code implementation13 Oct 2023 Yixin Wan, George Pu, Jiao Sun, Aparna Garimella, Kai-Wei Chang, Nanyun Peng

Through benchmarking evaluation on 2 popular LLMs- ChatGPT and Alpaca, we reveal significant gender biases in LLM-generated recommendation letters.

Benchmarking Fairness +1

LIMA: Less Is More for Alignment

5 code implementations NeurIPS 2023 Chunting Zhou, PengFei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, Omer Levy

Large language models are trained in two stages: (1) unsupervised pretraining from raw text, to learn general-purpose representations, and (2) large scale instruction tuning and reinforcement learning, to better align to end tasks and user preferences.

Language Modelling reinforcement-learning

Towards Robust NLG Bias Evaluation with Syntactically-diverse Prompts

1 code implementation3 Dec 2022 Arshiya Aggarwal, Jiao Sun, Nanyun Peng

These fixed prefix templates could themselves be specific in terms of styles or linguistic structures, which may lead to unreliable fairness conclusions that are not representative of the general trends from tone varying prompts.

Fairness Text Generation

A Moral- and Event- Centric Inspection of Gender Bias in Fairy Tales at A Large Scale

no code implementations25 Nov 2022 Zhixuan Zhou, Jiao Sun, Jiaxin Pei, Nanyun Peng, JinJun Xiong

Our analysis further reveal stereotypical portrayals of both male and female characters in terms of moral foundations and events.

Fairness

Dialect-robust Evaluation of Generated Text

no code implementations2 Nov 2022 Jiao Sun, Thibault Sellam, Elizabeth Clark, Tu Vu, Timothy Dozat, Dan Garrette, Aditya Siddhant, Jacob Eisenstein, Sebastian Gehrmann

Evaluation metrics that are not robust to dialect variation make it impossible to tell how well systems perform for many groups of users, and can even penalize systems for producing text in lower-resource dialects.

nlg evaluation

Context-Situated Pun Generation

1 code implementation24 Oct 2022 Jiao Sun, Anjali Narayan-Chen, Shereen Oraby, Shuyang Gao, Tagyoung Chung, Jing Huang, Yang Liu, Nanyun Peng

In this work, we propose a new task, context-situated pun generation, where a specific context represented by a set of keywords is provided, and the task is to first identify suitable pun words that are appropriate for the context, then generate puns based on the context keywords and the identified pun words.

Retrieval

ExPUNations: Augmenting Puns with Keywords and Explanations

1 code implementation24 Oct 2022 Jiao Sun, Anjali Narayan-Chen, Shereen Oraby, Alessandra Cervone, Tagyoung Chung, Jing Huang, Yang Liu, Nanyun Peng

The tasks of humor understanding and generation are challenging and subjective even for humans, requiring commonsense and real-world knowledge to master.

Explanation Generation Natural Language Understanding +1

Investigating the Benefits of Free-Form Rationales

no code implementations25 May 2022 Jiao Sun, Swabha Swayamdipta, Jonathan May, Xuezhe Ma

After controlling for instances where rationales leak the correct answer while not providing additional background knowledge, we find that incorporating only 5% of rationales during training can boost model performance by 47. 22% for CoS-E and 57. 14% for ECQA during inference.

Helpfulness and Fairness of Task-Oriented Dialogue Systems

no code implementations25 May 2022 Jiao Sun, Yu Hou, Jiin Kim, Nanyun Peng

Then, we collect human annotations for the helpfulness of dialogue responses based on our definition and build a classifier to automatically determine the helpfulness of a response.

Fairness Goal-Oriented Dialogue Systems +1

Investigating Explainability of Generative AI for Code through Scenario-based Design

no code implementations10 Feb 2022 Jiao Sun, Q. Vera Liao, Michael Muller, Mayank Agarwal, Stephanie Houde, Kartik Talamadupula, Justin D. Weisz

Using scenario-based design and question-driven XAI design approaches, we explore users' explainability needs for GenAI in three software engineering use cases: natural language to code, code translation, and code auto-completion.

Code Translation Explainable Artificial Intelligence (XAI)

HAGEN: Homophily-Aware Graph Convolutional Recurrent Network for Crime Forecasting

no code implementations27 Sep 2021 Chenyu Wang, Zongyu Lin, Xiaochen Yang, Jiao Sun, Mingxuan Yue, Cyrus Shahabi

Based on the homophily assumption of GNN, we propose a homophily-aware constraint to regularize the optimization of the region graph so that neighboring region nodes on the learned graph share similar crime patterns, thus fitting the mechanism of diffusion convolution.

Crime Prediction Graph Learning

On Measures of Biases and Harms in NLP

no code implementations7 Aug 2021 Sunipa Dev, Emily Sheng, Jieyu Zhao, Aubrie Amstutz, Jiao Sun, Yu Hou, Mattie Sanseverino, Jiin Kim, Akihiro Nishi, Nanyun Peng, Kai-Wei Chang

Recent studies show that Natural Language Processing (NLP) technologies propagate societal biases about demographic groups associated with attributes such as gender, race, and nationality.

Men Are Elected, Women Are Married: Events Gender Bias on Wikipedia

1 code implementation ACL 2021 Jiao Sun, Nanyun Peng

Human activities can be seen as sequences of events, which are crucial to understanding societies.

Event Detection

ESTER: A Machine Reading Comprehension Dataset for Event Semantic Relation Reasoning

1 code implementation16 Apr 2021 Rujun Han, I-Hung Hsu, Jiao Sun, Julia Baylon, Qiang Ning, Dan Roth, Nanyun Peng

While these tasks partially evaluate machines' ability of narrative understanding, human-like reading comprehension requires the capability to process event-based information beyond arguments and temporal reasoning.

Machine Reading Comprehension Natural Language Queries +2

EventPlus: A Temporal Event Understanding Pipeline

1 code implementation NAACL 2021 Mingyu Derek Ma, Jiao Sun, Mu Yang, Kung-Hsiang Huang, Nuan Wen, Shikhar Singh, Rujun Han, Nanyun Peng

We present EventPlus, a temporal event understanding pipeline that integrates various state-of-the-art event understanding components including event trigger and type detection, event argument detection, event duration and temporal relation extraction.

Common Sense Reasoning Event Extraction +1

FAWA: Fast Adversarial Watermark Attack on Optical Character Recognition (OCR) Systems

1 code implementation15 Dec 2020 Lu Chen, Jiao Sun, Wei Xu

In both letter-level and word-level attacks, our experiments show that in addition to natural appearance, FAWA achieves a 100% attack success rate with 60% less perturbations and 78% fewer iterations on average.

Optical Character Recognition Optical Character Recognition (OCR)

Catching Loosely Synchronized Behavior in Face of Camouflage

no code implementations21 Oct 2018 Yikun Ban, Jiao Sun, Xin Liu

Fraud has severely detrimental impacts on the business of social networks and other online applications.

Social and Information Networks Cryptography and Security

Cannot find the paper you are looking for? You can Submit a new open access paper.