Story Visualization
20 papers with code • 3 benchmarks • 1 datasets
Story Visualization is the task of generating coherent and aligned sequence of images given a sequence of textual captions representing description of a story. It mainly consists of two tasks: story generation and story continuation, where story continuation uses additional ground truth information in the form of the first frame.
Benchmarks
These leaderboards are used to track progress in Story Visualization
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
StoryBench
Latest papers with no code
CogCartoon: Towards Practical Story Visualization
The state-of-the-art methods for story visualization demonstrate a significant demand for training data and storage, as well as limited flexibility in story presentation, thereby rendering them impractical for real-world applications.
Make-A-Storyboard: A General Framework for Storyboard with Disentangled and Merged Control
Story Visualization aims to generate images aligned with story prompts, reflecting the coherence of storybooks through visual consistency among characters and scenes. Whereas current approaches exclusively concentrate on characters and neglect the visual consistency among contextually correlated scenes, resulting in independent character images without inter-image coherence. To tackle this issue, we propose a new presentation form for Story Visualization called Storyboard, inspired by film-making, as illustrated in Fig. 1. Specifically, a Storyboard unfolds a story into visual representations scene by scene.
AutoStory: Generating Diverse Storytelling Images with Minimal Human Effort
We empirically find that sparse control conditions, such as bounding boxes, are suitable for layout planning, while dense control conditions, e. g., sketches and keypoints, are suitable for generating high-quality image content.
Style Generation: Image Synthesis based on Coarsely Matched Texts
In this work, we attempt to stylize an input image using such coarsely matched text as guidance.
Improved Visual Story Generation with Adaptive Context Modeling
Diffusion models developed on top of powerful text-to-image generation models like Stable Diffusion achieve remarkable success in visual story generation.
Counterfactual Edits for Generative Evaluation
Evaluation of generative models has been an underrepresented field despite the surge of generative architectures.
An Impartial Transformer for Story Visualization
Story Visualization is an advanced task of computed vision that targets sequential image synthesis, where the generated samples need to be realistic, faithful to their conditioning and sequentially consistent.
Learning to Model Multimodal Semantic Alignment for Story Visualization
Story visualization aims to generate a sequence of images to narrate each sentence in a multi-sentence story, where the images should be realistic and keep global consistency across dynamic scenes and characters.
Generating a Temporally Coherent Visual Story by Multimodal Recurrent Transformers
Story visualization is a challenging text-to-image generation task for the difficulty of rendering visual details from abstract text descriptions.
Generating a Temporally Coherent Image Sequence for a Story by Multimodal Recurrent Transformers
Story visualization is a challenging text-to-image generation task for the difficulty of rendering visual details from abstract text descriptions.
