Medical Report Generation
29 papers with code • 2 benchmarks • 3 datasets
Medical report generation (MRG) is a task which focus on training AI to automatically generate professional report according the input image data. This can help clinicians make faster and more accurate decision since the task itself is both time consuming and error prone even for experienced doctors.
Deep neural network and transformer based architecture are currently the most popular methods for this certain task, however, when we try to transfer out pre-trained model into this certain domain, their performance always degrade.
The following are some of the reasons why RSG is hard for pre-trained models:
- Language datasets in a particular domain can sometimes be quite different from the large number of datasets available on the Internet
- During the fine-tuning phase, datasets in the medical field are often unevenly distributed
More recently, multi-modal learning and contrastive learning have shown some inspiring results in this field, but it's still challenging and requires further attention.
Here are some additional readings to go deeper on the task:
- On the Automatic Generation of Medical Imaging Reports
https://doi.org/10.48550/arXiv.1711.08195
- A scoping review of transfer learning research on medical image analysis using ImageNet
https://arxiv.org/abs/2004.13175
- A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis
https://arxiv.org/abs/2004.12150
(Image credit : Transformers in Medical Imaging: A Survey)
Libraries
Use these libraries to find Medical Report Generation models and implementations
MIMIC-CXR
PathVQA
HistGen WSI-Report Dataset
Latest papers with no code
Prompt-Guided Generation of Structured Chest X-Ray Report Using a Pre-trained LLM
Our method introduces a prompt-guided approach to generate structured chest X-ray reports using a pre-trained large language model (LLM).
Dia-LLaMA: Towards Large Language Model-driven CT Report Generation
Medical report generation has achieved remarkable advancements yet has still been faced with several challenges.
MedCycle: Unpaired Medical Report Generation via Cycle-Consistency
This approach is based on cycle-consistent mapping functions that transform image embeddings into report embeddings, coupled with report auto-encoding for medical report generation.
Unmasking and Quantifying Racial Bias of Large Language Models in Medical Report Generation
Large language models like GPT-3. 5-turbo and GPT-4 hold promise for healthcare professionals, but they may inadvertently inherit biases during their training, potentially affecting their utility in medical applications.
Dual-modal Dynamic Traceback Learning for Medical Report Generation
Recent generative representation learning methods have demonstrated the benefits of dual-modal learning from both image and text modalities.
Medical Report Generation based on Segment-Enhanced Contrastive Representation Learning
Automated radiology report generation has the potential to improve radiology reporting and alleviate the workload of radiologists.
Improving Medical Report Generation with Adapter Tuning and Knowledge Enhancement in Vision-Language Foundation Models
Medical report generation demands automatic creation of coherent and precise descriptions for medical images.
C^2M-DoT: Cross-modal consistent multi-view medical report generation with domain transfer network
In addition, word-level optimization based on numbers ignores the semantics of reports and medical images, and the generated reports often cannot achieve good performance.
IIHT: Medical Report Generation with Image-to-Indicator Hierarchical Transformer
The classifier module first extracts image features from the input medical images and produces disease-related indicators with their corresponding states.
Customizing General-Purpose Foundation Models for Medical Report Generation
In this work, we propose customizing off-the-shelf general-purpose large-scale pre-trained models, i. e., foundation models (FMs), in computer vision and natural language processing with a specific focus on medical report generation.
