DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedback

no code implementations • 29 Nov 2023 • Jiao Sun, Deqing Fu, Yushi Hu, Su Wang, Royi Rassin, Da-Cheng Juan, Dana Alon, Charles Herrmann, Sjoerd van Steenkiste, Ranjay Krishna, Cyrus Rashtchian

Then, it uses two VLMs to select the best generation: a Visual Question Answering model that measures the alignment of generated images to the text, and another that measures the generation's aesthetic quality.

Question Answering Text-to-Image Generation +1

Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment

1 code implementation • NeurIPS 2023 • Royi Rassin, Eran Hirsch, Daniel Glickman, Shauli Ravfogel, Yoav Goldberg, Gal Chechik

This reflects an impaired mapping between linguistic binding of entities and modifiers in the prompt and visual binding of the corresponding elements in the generated image.

Attribute Sentence +1

Conjunct Resolution in the Face of Verbal Omissions

no code implementations • 26 May 2023 • Royi Rassin, Yoav Goldberg, Reut Tsarfaty

In this work we propose a conjunct resolution task that operates directly on the text and makes use of a split-and-rephrase paradigm in order to recover the missing elements in the coordination structure.

Missing Elements Sentence +1

DALLE-2 is Seeing Double: Flaws in Word-to-Concept Mapping in Text2Image Models

no code implementations • 19 Oct 2022 • Royi Rassin, Shauli Ravfogel, Yoav Goldberg

We study the way DALLE-2 maps symbols (words) in the prompt to their references (entities or properties of entities in the generated image).