Image Captioning
620 papers with code • 32 benchmarks • 67 datasets
Image Captioning is the task of describing the content of an image in words. This task lies at the intersection of computer vision and natural language processing. Most image captioning systems use an encoder-decoder framework, where an input image is encoded into an intermediate representation of the information in the image, and then decoded into a descriptive text sequence. The most popular benchmarks are nocaps and COCO, and models are typically evaluated according to a BLEU or CIDER metric.
( Image credit: Reflective Decoding Network for Image Captioning, ICCV'19)
Benchmarks
These leaderboards are used to track progress in Image Captioning
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Image Captioning models and implementationsDatasets
MS COCO
Flickr30k
Conceptual Captions
COCO Captions
Hateful Memes
VizWiz
NoCaps
ReferItGame
TextCaps
Subtasks
Latest papers with no code
A Survey on Large Language Models from Concept to Implementation
Recent advancements in Large Language Models (LLMs), particularly those built on Transformer architectures, have significantly broadened the scope of natural language processing (NLP) applications, transcending their initial use in chatbot technology.
The Solution for the ICCV 2023 1st Scientific Figure Captioning Challenge
In this paper, we propose a solution for improving the quality of captions generated for figures in papers.
Visual Hallucination: Definition, Quantification, and Prescriptive Remediations
The troubling rise of hallucination presents perhaps the most significant impediment to the advancement of responsible AI.
Semi-Supervised Image Captioning Considering Wasserstein Graph Matching
Image captioning can automatically generate captions for the given images, and the key challenge is to learn a mapping function from visual features to natural language features.
Automated Report Generation for Lung Cytological Images Using a CNN Vision Classifier and Multiple-Transformer Text Decoders: Preliminary Study
Independent text decoders for benign and malignant cells are prepared for text generation, and the text decoder switches according to the CNN classification results.
Image Captioning in news report scenario
Image captioning strives to generate pertinent captions for specified images, situating itself at the crossroads of Computer Vision (CV) and Natural Language Processing (NLP).
Cognitive resilience: Unraveling the proficiency of image-captioning models to interpret masked visual content
This study explores the ability of Image Captioning (IC) models to decode masked visual content sourced from diverse datasets.
MyVLM: Personalizing VLMs for User-Specific Queries
To effectively recognize a variety of user-specific concepts, we augment the VLM with external concept heads that function as toggles for the model, enabling the VLM to identify the presence of specific target concepts in a given image.
Improved Baselines for Data-efficient Perceptual Augmentation of LLMs
The abilities of large language models (LLMs) have recently progressed to unprecedented levels, paving the way to novel applications in a wide variety of areas.
Inserting Faces inside Captions: Image Captioning with Attention Guided Merging
In this work we introduce AstroCaptions, a dataset for the image captioning task.
