awesome-medical-vision-and-language-pretraining

The collection of medical VLP papars (Chest X-ray)

✨ We will promptly update the relevant papers. Please feel free to contact me to add links or to discuss this area with me.

2024

• [arxiv 2024] [ECAMP] ECAMP: Entity-centered Context-aware Medical Vision Language Pre-training [pdf] [code]
• [arxiv 2024] [MeDSLIP] MeDSLIP: Medical Dual-Stream Language-Image Pre-training for Fine-grained Alignment [pdf]
• [CVPR 2024] [CARZeroP] CARZero: Cross-Attention Alignment for Radiology Zero-Shot Classification [pdf] [code]
• [CVPR 2024] [MAVL] DecomposingDiseaseDescriptionsforEnhancedPathologyDetection:A Multi-AspectVision-LanguagePre-trainingFramework [pdf] [code]
• [CVPR 2024] [MLIP] MLIP: Enhancing Medical Visual Representation with Divergence Encoder and Knowledge-guided Contrastive Learning [pdf] [code]
• [ICML 2024] [Med-ST] Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training [pdf] [code]
• [MICCAI 2024] [ASG] Anatomical Structure-Guided Medical Vision-Language Pre-training [pdf] [code]

2023

• [ICLR 2023] [MRM] Advancing Radiograph Representation Learning with Masked Record Modeling [pdf] [code]
• [CVPR 2023] [BioViL-T] Learning to Exploit Temporal Structure for Biomedical Vision–Language Processing [pdf] [code]
• [MICCAI 2023] [CXR-CLIP] Toward Large Scale Chest X-ray Language-Image Pre-training [pdf] [code]
• [MICCAI 2023] [M-FLAG] M-FLAG: Medical Vision-Language Pre-training with Frozen Language Models and Latent Space Geometry Optimization [pdf] [code]
• [MICCAI 2023] [KoBo] Knowledge Boosting: Rethinking Medical Contrastive Vision-Language Pre-Training [pdf] [code]
• [MICCAI-ws 2023] [K-Diag] K-Diag: Knowledge-enhanced Disease Diagnosis in Radiographic Imaging [pdf] [code]
• [ICCV 2023] [MedKLIP] MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in Radiology [pdf] [code]
• [ICCV 2023] [PTUnifier] Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts [pdf] [code]
• [ICCV 2023] [PRIOR] PRIOR: Prototype Representation Joint Learning from Medical Images and Reports [pdf] [code]
• [ICCV 2023] [LIMITR] LIMITR: Leveraging Local Information for Medical Image-Text Representation [pdf]
• [Nature Com 2023] [KAD] Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Images [pdf] [code]
• [MIA 2023] [SAT] Improving Medical Vision-Language Contrastive Pretraining With Semantics-Aware Triage [pdf] [code]
• [arxiv 2023] [MOTOR] Towards Medical Artificial General Intelligence via Knowledge-Enhanced Multimodal Pretraining [pdf] [code]

2022

• [MICCAI 2022] [BFSPR] Breaking with Fixed Set Pathology Recognition through Report-Guided Contrastive Training [pdf]
• [MICCAI 2022] [M3AE] Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Training [pdf] [code]
• [ACM MM 2022] [ARL] Align, Reason and Learn: Enhancing Medical Vision-and-Language Pre-training with Knowledge [pdf] [code]
• [EMNLP 2022] [MedCLIP] Contrastive Learning from Unpaired Medical Images and Text [pdf] [code]
• [ECCV 2022] [BioViL] Making the Most of Text Semantics to Improve Biomedical Vision–Language Processing [pdf] [code]
• [ECCV 2022] [LoVT] Joint Learning of Localized Representations from Medical Images and Reports [pdf]
• [NIPS 2022] [MGCA] Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learning [pdf] [code]

2021

• [PMLR 2022(arxiv 2020)] [ConViRT] Contrastive Learning of Medical Visual Representations from Paired Images and Text [pdf] [code]
• [ICCV 2021] [GLoRIA] AMultimodal Global-Local Representation Learning Framework for Label-efficient Medical Image Recognition [pdf] [code]