Medical Report Generation
29 papers with code • 2 benchmarks • 3 datasets
Medical report generation (MRG) is a task which focus on training AI to automatically generate professional report according the input image data. This can help clinicians make faster and more accurate decision since the task itself is both time consuming and error prone even for experienced doctors.
Deep neural network and transformer based architecture are currently the most popular methods for this certain task, however, when we try to transfer out pre-trained model into this certain domain, their performance always degrade.
The following are some of the reasons why RSG is hard for pre-trained models:
- Language datasets in a particular domain can sometimes be quite different from the large number of datasets available on the Internet
- During the fine-tuning phase, datasets in the medical field are often unevenly distributed
More recently, multi-modal learning and contrastive learning have shown some inspiring results in this field, but it's still challenging and requires further attention.
Here are some additional readings to go deeper on the task:
- On the Automatic Generation of Medical Imaging Reports
https://doi.org/10.48550/arXiv.1711.08195
- A scoping review of transfer learning research on medical image analysis using ImageNet
https://arxiv.org/abs/2004.13175
- A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis
https://arxiv.org/abs/2004.12150
(Image credit : Transformers in Medical Imaging: A Survey)
Libraries
Use these libraries to find Medical Report Generation models and implementations
MIMIC-CXR
PathVQA
HistGen WSI-Report Dataset
Latest papers with no code
Customizing General-Purpose Foundation Models for Medical Report Generation
In this work, we propose customizing off-the-shelf general-purpose large-scale pre-trained models, i. e., foundation models (FMs), in computer vision and natural language processing with a specific focus on medical report generation.
Boosting Radiology Report Generation by Infusing Comparison Prior
To tackle this issue, we propose a novel approach that leverages a rule-based labeler to extract comparison prior information from radiology reports.
MvCo-DoT:Multi-View Contrastive Domain Transfer Network for Medical Report Generation
In clinical scenarios, multiple medical images with different views are usually generated at the same time, and they have high semantic consistency.
Cyclic Generative Adversarial Networks With Congruent Image-Report Generation For Explainable Medical Image Analysis
Apart from enabling transparent medical image labeling and interpretation, we achieve report and image-based labeling comparable to prior methods, including state-of-the-art performance in some cases as evidenced by experiments on the Indiana Chest X-ray dataset
Hybrid Reinforced Medical Report Generation with M-Linear Attention and Repetition Penalty
In this article, we propose a hybrid reinforced medical report generation method with m-linear attention and repetition penalty mechanism (HReMRG-MR) to overcome these problems.
Representative Image Feature Extraction via Contrastive Learning Pretraining for Chest X-ray Report Generation
Medical report generation is a challenging task since it is time-consuming and requires expertise from experienced radiologists.
A Medical Semantic-Assisted Transformer for Radiographic Report Generation
Automated radiographic report generation is a challenging cross-domain task that aims to automatically generate accurate and semantic-coherence reports to describe medical images.
Competence-based Multimodal Curriculum Learning for Medical Report Generation
Medical report generation task, which targets to produce long and coherent descriptions of medical images, has attracted growing research interests recently.
A Self-Guided Framework for Radiology Report Generation
Moreover, SGF successfully improves the accuracy and length of medical report generation by incorporating a similarity comparison mechanism that imitates the process of human self-improvement through compar-ative practice.
Cross-modal Clinical Graph Transformer for Ophthalmic Report Generation
To endow models with the capability of incorporating expert knowledge, we propose a Cross-modal clinical Graph Transformer (CGT) for ophthalmic report generation (ORG), in which clinical relation triples are injected into the visual features as prior knowledge to drive the decoding procedure.
