Skip to content

RGB-T Fusion, RGB-T SOD, RGB-T Vehicle Detection, RGB-T Crowd Counting, RGB-T Pedestrian Detection, RGB-T Semantic Segmeantaion, RGB-T Tracking

License

Notifications You must be signed in to change notification settings

VisionVerse/Awesome-RGBT

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 

Repository files navigation

Awesome RGB-T Application Awesome PRs WelcomeStars

avatar A collection of papers, codes, and datasets for RGB-T related tasks based on deep learning.

The main directions involved are RGB-T Fusion, RGB-T Salient Object Detection (SOD), RGB-T Vehicle Detection (VD), RGB-T Crowd Counting (CC), RGB-T Pedestrian Detection (PD), RGB-T Semantic Segmeantaion (SS), RGB-T Tracking.

More than 120+ papers have been included. 🎉🎉🎉

Feel free to star and fork~ 🌟🌟🌟
We will continue to update this repository 🏃🏃🏃

Contents

  1. RGB-T Fusion
  2. RGB-T Salient Object Detection (SOD)
  3. RGB-T Vehicle Detection
  4. RGB-T Crowd Counting
  5. RGB-T Pedestrian Detection
  6. RGB-T Semantic Segmeantaion
  7. RGB-T Tracking
  8. RGB-T ReID
  9. RGB-T Alignment

1. RGB-T Fusion

🚀🚀🚀Update (in 2024-01-09)

No. Year Model Pub. Title Links
4 2024 - arXiv From Text to Pixels: A Context-Aware Semantic Synergy Solution for Infrared and Visible Image Fusion Paper/[Project]/解读-知乎
3 2023 RFVIF Inf. Fusion Feature dynamic alignment and refinement for infrared–visible image fusion: Translation robust fusion Paper/Project
1 2023 MURF TPAMI ✨ MURF:Mutually Reinforcing Multi-Modal Image Registration and Fusion Paper/Project
1 2022 RFNet CVPR ✨ RFNet:Unsupervised Network for Mutually Reinforcing Multi-modal Image Registration and Fusion Paper/Project
1 2020 U2Fusion TPAMI ✨ U2Fusion: A Unified Unsupervised Image Fusion Network Paper/Project
2 2019 - Inf. Fusion Infrared and visible image fusion methods and applications: A survey Paper/[Project]
1 2016 GTF Inf. Fusion Infrared and visible image fusion via gradient transfer and total variation minimization Paper/Project

✨ : Group of Professor Ma at WHU

Datasets

Tools

2. RGB-T Salient Object Detection

🚀🚀🚀Update (in 2024-01-10)

No. Year Model Pub. Title Links
2 2024 UTDNet NN UTDNet: A unified triplet decoder network for multimodal salient object detection Paper/[Project]
1 2024 MSEDNet NN MSEDNet: Multi-scale fusion and edge-supervised network for RGB-T salient object detection Paper/Project
18 2023 LSNet TIP LSNet: Lightweight Spatial Boosting Network for Detecting Salient Objects in RGB-Thermal Images Paper/Project
17 2023 Scribble ICME Scribble-Supervised RGB-T Salient Object Detection Paper/Project
16 2023 EAEFNet RAL Explicit Attention-Enhanced Fusion for RGB-Thermal Perception Tasks Paper/Project
15 2023 TAGFNet EAAI Thermal images-aware guided early fusion network for cross-illumination RGB-T salient object detection Paper/Project
14 2023 MFFNet TMM MFFNet: Multi-modal Feature Fusion Network for V-D-T Salient Object Detection Paper/Project
13 2023 FANet NC Feature aggregation with transformer for RGB-T salient object detection Paper/Project
12 2023 MENet NC MENet: Lightweight multimodality enhancement network for detecting salient objects in RGB-thermal images Paper/Project
11 2023 TIDNet KBS Three-stream interaction decoder network for RGB-thermal salient object detection Paper/Project
10 2023 PRLNet TIP Position-Aware Relation Learning for RGB-Thermal Salient Object Detection Paper/Project
9 2023 MGAINet TCSVT Multiple Graph Affinity Interactive Network and a Variable Illumination Dataset for RGBT Image Salient Object Detection Paper/Project
8 2023 CAVER TIP CAVER: Cross-Modal View-Mixed Transformer for Bi-Modal Salient Object Detection Paper/Project
7 2023 C3A PR Cross-modal co-feedback cellular automata for RGB-T saliency detection Paper/Project
6 2023 WaveNet TIP WaveNet: Wavelet Network With Knowledge Distillation for RGB-T Salient Object Detection Paper/Project
5 2023 xxx ICIP Feature Enhancement and Fusion for RGB-T Salient Object Detection Paper/Project
4 2023 xxx arXiv All in One: RGB, RGB-D, and RGB-T Salient Object Detection Paper/Project
3 2023 xxx ACM MM Saliency Prototype for RGB-D and RGB-T Salient Object Detection Paper/Project
2 2023 FFANet PR Frequency-aware feature aggregation network with dual-task consistency for RGB-T salient object detection Paper/Project
1 2023 xxx arXiv Unified-modal Salient Object Detection via Adaptive Prompt Learning Paper/Project
17 2022 xxx AI RGB-T salient object detection via CNN feature and result saliency map fusion Paper/Project
16 2022 MIA_DPD NC Multi-modal Interactive Attention and Dual Progressive Decoding Network for RGB-D/T Salient Object Detection Paper/Project
19 2023 CMDBIFNet TCSVT Cross-Modality Double Bidirectional Interaction and Fusion Network for RGB-T Salient Object Detection Paper/Project
15 2022 CGMDRNet TCSVT CGMDRNet: Cross-Guided Modality Difference Reduction Network for RGB-T Salient Object Detection Paper/Project
15 2023 MITFNet TCSVT Modality-Induced Transfer-Fusion Network for RGB-D and RGB-T Salient Object Detection Paper/Project
14 2022 Glass arXiv Glass Segmentation with RGB-Thermal Image Pairs Paper/Project
13 2022 DCNet TIP Weakly Alignment-free RGBT Salient Object Detection with Deep Correlation Network Paper/Project
12 2022 OSRNet TIM Real-time One-stream Semantic-guided Refinement Network for RGB-Thermal Salient Object Detection Paper/Project
11 2022 CCFENet TCSVT Cross-Collaborative Fusion-Encoder Network for Robust RGB-Thermal Salient Object Detection Paper/Project
10 2022 xx EAAI Unidirectional RGB-T salient object detection with intertwined driving of encoding and fusion Paper/Project
9 2022 EAF-Net MVA EAF-Net: an enhancement and aggregation–feedback network for RGB-T salient object detection Paper/Project
8 2022 SwinMCNet arXiv Mirror Complementary Transformer Network for RGB-thermal Salient Object Detection Paper/Project
7 2022 xxx CVIU Enabling modality interactions for RGB-T salient object detection Paper/Project
6 2022 MCFNet AI Modal complementary fusion network for RGB-T salient object detection Paper/Project
5 2022 TNet TMM Does Thermal really always matter for RGB-T salient object detection Paper/Project
4 2022 xxx Arxiv Interactive Context-Aware Network for RGB-T Salient Object Detection Paper/Project
3 2022 MFENet DSP MFENet: Multitype fusion and enhancement network for detecting salient objects in RGB-T images Paper/Project
2 2022 xxx PR Cross-modal co-feedback cellular automata for RGB-T saliency detection Paper/Project
1 2022 ACMANet KBS Asymmetric cross-modal activation network for RGB-T salient object detection Paper/Project
10 2021 MIDD TIP Multi-Interactive Dual-Decoder for RGB-Thermal Salient Object Detection Paper/Project
9 2021 ECFFNet TCSVT ECFFNet: Effective and Consistent Feature Fusion Network for RGB-T Salient Object Detection Paper/Results(tx48)
8 2021 xxx TCSVT Unified Information Fusion Network for Multi-Modal RGB-D and RGB-T Salient Object Detection Paper/Project
7 2021 CGFNet TCSVT CGFNet: Cross-Guided Fusion Network for RGB-T Salient Object Detection Paper/Project
6 2021 CSRNet TCSVT Efficient Context-Guided Stacked Refinement Network for RGB-T Salient Object Detection Paper/Project
5 2021 TSFNet SPL TSFNet: Two-Stage Fusion Network for RGB-T Salient Object Detection Paper/Project
4 2021 APNet TETCI APNet: Adversarial Learning Assistance and Perceived Importance Fusion Network for All-Day RGB-T Salient Object Detection Paper/Project
3 2021 SwinNet TCSVT SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection Paper/Project
2 2021 xxx TCSVT Multi-graph Fusion and Learning for RGBT Image Saliency Detection Paper/Project
1 2021 xxx CYBER Salient Target Detection in RGB-T Image based on Multi-level Semantic Information Paper/Project
4 2020 FMNet TIP RGB-T Salient Object Detection via Fusing Multi-Level CNN Features Paper/Project
3 2020 Net TCSVT Revisiting Feature Fusion for RGB-T Salient Object Detection Paper/Project
2 2020 DASOD AAAI Multi-Spectral Salient Object Detection by Adversarial Domain Adaptation Paper/Project
1 2020 DASODv2 TMM Deep Domain Adaptation Based Multi-spectral Salient Object Detection Paper/Project
3 2019 M3S-NIR MIPR M3S-NIR: Multi-Modal Multi-Scale Noise-Insensitive Ranking for RGB-T Saliency Detection Paper/Project
2 2019 CGL TMM RGB-T Image Saliency Detection via Collaborative Graph Learning Paper/Project
1 2019 Net TCSVT RGBT Salient Object Detection: Benchmark and A Novel Cooperative Ranking Approach Paper/Project
1 2018 Net IGTA RGB-T Saliency Detection Benchmark: Dataset, Baselines, Analysis and a Novel Approach Paper/Project
1 2017 Net ISCID Learning Multiscale Deep Features and SVM Regressors for Adaptive RGB-T Saliency Detection Paper/Project

RGB, RGB-D SOD, et al.:

No. Year Model Pub. Title Links
1 2024 - TIP Rethinking Object Saliency Ranking: A Novel Whole-Flow Processing Paradigm Paper/Project

Datasets

Tools

3. RGB-T Vehicle Detection

🚀🚀🚀Update (in 2023-12-26)

No. Year Model Pub. Title Links
3 2023 Net ACM MM Multispectral Object Detection via Cross-Modal Conflict-Aware Learning Paper
3 2023 Net TNNLS LRAF-Net: Long-Range Attention Fusion Network for Visible–Infrared Object Detection Paper
2 2023 Net RS GF-Detection: Fusion with GAN of Infrared and Visible Images for Vehicle Detection at Nighttime Paper
1 2023 Net PR Cross-modality attentive feature fusion for object detection in multispectral remote sensing imagery Paper
3 2022 Net ECCV Translation, Scale and Rotation: Cross-Modal Alignment Meets RGB-Infrared Vehicle Detection Paper
2 2022 Net TCSVT Drone-based RGB-Infrared Cross-Modality Vehicle Detection via Uncertainty-Aware Learning Paper
1 2022 Net RS Improving RGB-Infrared Object Detection by Reducing Cross-Modality Redundancy Paper

Datasets

  • DroneVehicle: partially aligned, link
  • VEDAI: strictly aligned, link
  • Multispectral Datasets for Detection and Segmentation: with Segmentation annotation, link

Tools

4. RGB-T Crowd Counting

🚀🚀🚀Update (in 2023-12-26)

No. Year Model Pub. Title Links
2 2022 XXNet arXiv MAFNet: A Multi-Attention Fusion Network for RGB-T Crowd Counting Paper
1 2022 XXNet ICME Multimodal Crowd Counting with Mutual Attention Transformers Paper
1 2021 XXNet CVPR Cross-Modal Collaborative Representation Learning and a Large-Scale RGBT Benchmark for Crowd Counting Paper/Project
1 2020 XXNet ACCV RGB-T Crowd Counting from Drone: A Benchmark and MMCCN Network Paper/Project

Datasets

Tools

5. RGB-T Pedestrian Detection

🚀🚀🚀Update (in 2024-08-29)

No. Year Model Pub. Title Links
9 2024 TFDet TNNLS TFDet: Target-Aware Fusion for RGB-T Pedestrian Detection Paper/Project
8 2024 M2FNet TMM M2FNet: Mask-guided Multi-level Fusion for RGB-T Pedestrian Detection Paper/Project
7 2023 AANet ACM MM Attentive Alignment Network for Multispectral Pedestrian Detection Paper/[Project]
6 2023 CALNet ACM MM Multispectral Object Detection via Cross-Modal Conflict-Aware Learning Paper/Project
5 2023 XXNet TCSVT Stabilizing Multispectral Pedestrian Detection with Evidential Hybrid Fusion Paper
4 2023 XXNet CVPRW Multimodal Object Detection by Channel Switching and Spatial Attention Paper
3 2023 XXNet TITS Multi-Modal Feature Pyramid Transformer for RGB-Infrared Object Detection Paper
2 2023 XXNet TMM Multiscale Cross-modal Homogeneity Enhancement and Confidence-aware Fusion for Multispectral Pedestrian Detection Paper
1 2023 XXNet RS HAFNet: Hierarchical Attentive Fusion Network for Multispectral Pedestrian Detection Paper
7 2022 XXNet ECCV Multimodal Object Detection via Probabilistic Ensembling Paper
6 2022 XXNet ACM MM Learning a Dynamic Cross-Modal Network for Multispectral Pedestrian Detection Paper
5 2022 XXNet TMM Confidence-aware Fusion using Dempster-Shafer Theory for Multispectral Pedestrian Detection Paper
4 2022 XXNet PRCV Attention-Guided Multi-modal and Multi-scale Fusion for Multispectral Pedestrian Detection Paper
3 2022 XXNet ICIP Improving RGB-Infrared Pedestrian Detection by Reducing Cross-Modality Redundancy Paper
2 2022 XXNet TITS Spatio-contextual deep network-based multimodal pedestrian detection for autonomous driving Paper
1 2022 XXNet Sensors Adopting the YOLOv4 Architecture for Low-LatencyMultispectral Pedestrian Detection in Autonomous Driving Paper
5 2021 XXNet ICIP Deep Active Learning from Multispectral Data Through Cross-Modality Prediction Inconsistency Paper
4 2021 XXNet Sensors Attention Fusion for One-Stage Multispectral Pedestrian Detection Paper
3 2021 XXNet TCSVT Uncertainty-Guided Cross-Modal Learning for Robust Multispectral Pedestrian Detection Paper
2 2021 XXNet TCSVT Deep Cross-modal Representation Learning and Distillation for Illumination-invariant Pedestrian Detection Paper
1 2021 XXNet WACV Guided Attentive Feature Fusion for Multispectral Pedestrian Detection Paper
4 2020 XXNet BMVC Anchor-free Small-scale Multispectral Pedestrian Detection Paper/Project
3 2020 XXNet ICIP Multispectral Fusion for Object Detection with Cyclic Fuse-and-Refine Blocks Paper
2 2020 XXNet ECCV Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problems Paper/Project
1 2020 XXNet BMVC Anchor-free Small-scale Multispectral Pedestrian Detection Paper/Project
4 2019 XXNet ICCV Weakly Aligned Cross-Modal Learning for Multispectral Pedestrian Detection Paper/Project
3 2019 XXNet ISPRS Box-level Segmentation Supervised Deep Neural Networks for Accurate and Real-time Multispectral Pesdestrian Detecion Paper/Project
2 2019 XXNet Information Fusion Cross-modality interactive attention network for multispectral pedestrian detection Paper/Project
1 2019 XXNet Information Fusion Pedestrian detection with unsupervised multispectral feature learning using deep neural networks Paper
2 2018 XXNet BMVC Multispectral Pedestrian Detection via Simultaneous Detection and Segmentation Paper/Project/Project
1 2018 XXNet PR Unified Multi-spectral Pedestrian Detection Based on Probabilistic Fusion Networks Paper
1 2016 XXNet BMVC Multispectral Deep Neural Networks for Pedestrian Detection Paper/Project
1 2015 XXNet CVPR Multispectral Pedestrian Detection Benchmark Dataset and Baseline Paper/Project

Datasets

KAIST dataset, CVC-14 dataset , FLIR dataset, LLVIP dataset, M3FD dataset

Tools

6. RGB-T Semantic Segmeantaion

🚀🚀🚀Update (in 2024-01-14)

No. Year Model Pub. Title Links
01 2024 CAITNet TMM Context-Aware Interaction Network for RGB-T Semantic Segmentation Paper/Project
12 2023 - TITS Embedded Control Gate Fusion and Attention Residual Learning for RGB–Thermal Urban Scene Parsing Paper/Project
11 2023 EAEFNet RAL Explicit Attention-Enhanced Fusion for RGB-Thermal Perception Tasks Paper/Project
10 2023 - PR Complementarity-aware cross-modal feature fusion network for RGB-T semantic segmentation Paper/Project
9 2023 MMSMCNet TCSVT MMSMCNet: Modal Memory Sharing and Morphological Complementary Networks for RGB-T Urban Scene Semantic Segmentation Paper/Project
8 2023 CACFNet TIV CACFNet: Cross-Modal Attention Cascaded Fusion Network for RGB-T Urban Scene Parsing Paper/Project
7 2023 DBCNet TSMC DBCNet: Dynamic Bilateral Cross-Fusion Network for RGB-T Urban Scene Understanding in Intelligent Vehicles Paper/Project
6 2023 SGFNet TCSVT SGFNet: Semantic-Guided Fusion Network for RGB-Thermal Semantic Segmentation Paper/Project
5 2023 DPLNet arxiv Efficient Multimodal Semantic Segmentation via Dual-Prompt Learning Paper/Project
4 2023 - TITS A RGB-Thermal Image Segmentation Method Based on Parameter Sharing and Attention Fusion for Safe Autonomous Driving Paper/Project
3 2023 UTFNet GRSL UTFNet: Uncertainty-Guided Trustworthy Fusion Network for RGB-Thermal Semantic Segmentation Paper/Project
2 2023 SFAF-MA TIM SFAF-MA: Spatial Feature Aggregation and Fusion With Modality Adaptation for RGB-Thermal Semantic Segmentation Paper/Project
1 2023 SASEM TIV On Exploring Shape and Semantic Enhancements for RGB-X Semantic Segmentation Paper/Project
8 2022 EGFNet AAAI Edge-aware guidance fusion network for RGB–thermal scene parsing Paper/Project
7 2022 MTANet TIV MTANet: Multitask-Aware Network with Hierarchical Multimodal Fusion for RGB-T Urban Scene Understanding Paper/Project
6 2022 CMX TITS CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers Paper/Project
5 2022 ARTSeg ACPR ARTSeg: Employing Attention for Thermal Images Semantic Segmentation Paper/Project
4 2022 GCNet Neurocomputing GCNet: Grid-Like Context-Aware Network for RGB-Thermal Semantic Segmentation Paper/Project
3 2022 LASNet TCSVT RGB-T Semantic Segmentation with Location, Activation, and Sharpening Paper/Project
2 2022 GEBNet SPL GEBNet: Graph-Enhancement Branch Network for RGB-T Scene Parsing Paper/Project
1 2022 - TCSVT A Feature Divide-and-Conquer Network for RGB-T Semantic Segmentation Paper/Project
8 2021 GMNet TIP GMNet: Graded-Feature Multilabel-Learning Network for RGB-Thermal Urban Scene Semantic Segmentation Paper/Project
7 2021 ABMDRNet CVPR ABMDRNet: Adaptive-weighted Bi-directional Modality Difference Reduction Network for RGB-T Semantic Segmentation Paper/Project
6 2021 FEANet IROS FEANet: Feature-Enhanced Attention Network for RGB-Thermal Real-time Semantic Segmentation Paper/Project
5 2021 - Measurement Robust semantic segmentation based on RGB-thermal in variable lighting scenes Paper/Project
4 2021 MFFENet TMM MFFENet: Multiscale Feature Fusion and Enhancement Network for RGBThermal Urban Road Scene Parsing Paper/Project
3 2021 MMNet Applied Intelligence MMNet: Multi-modal multi-stage network for RGB-T image semantic segmentation Paper/Project
2 2021 CCAFFMNet Neurocomputing CCAFFMNet: Dual-spectral semantic segmentation network with channel-coordinate attention feature fusion module Paper/Project
1 2021 HeatNet IROS HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images Paper/Project
3 2020 PST900 ICRA PST900: RGB-Thermal Calibration, Dataset and Segmentation Network Paper/Project
2 2020 FuseSeg TASE FuseSeg: Semantic Segmentation of Urban Scenes Based on RGB and Thermal Data Fusion Paper/Project
1 2020 - CINE Using thermal intensities to build conditional random fields for object segmentation at night Paper/Project
1 2019 RTFNet RAL RTFNet: RGB-Thermal Fusion Network for Semantic Segmentation of Urban Scenes Paper/Project
1 2017 MFNet IROS MFNet: Towards Real-Time Semantic Segmentation for Autonomous Vehicles with Multi-Spectral Scenes Paper/Project

Datasets

Tools

7. RGB-T Tracking

🚀🚀🚀Update (in 2024-03-12)

No. Year Model Pub. Title Links
5 2023 MPLT arXiv RGB-T Tracking via Multi-Modal Mutual Prompt Learning Paper/Project
5 2023 XXNet arXiv Object Segmentation by Mining Cross-Modal Semantics Paper/Project
4 2023 XXNet ICME MTNet: Learning Modality-aware Representation with Transformer for RGBT Tracking Paper
3 2023 XXNet CVPR Visual Prompt Multi-Modal Tracking Paper/Project
2 2023 XXNet CVPR Efficient RGB-T Tracking via Cross-Modality Distillation Paper
1 2023 XXNet CVPR Bridging Search Region Interaction with Template for RGB-T Tracking Paper/Project
4 2022 XXNet SCIS RGBT tracking via reliable feature configuration Paper
3 2022 XXNet AAAI Attribute-Based Progressive Fusion Network for RGBT Tracking Paper/Project
2 2022 XXNet ACM MM Dense Feature Aggregation and Pruning for RGBT Tracking Paper
1 2022 XXNet ACM MM Prompting for Multi-Modal Tracking Paper
3 2021 XXNet TIP Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking Paper
2 2021 XXNet IJCV Learning Adaptive Attribute-Driven Representation for Real-Time RGB-T Tracking Paper
1 2021 XXNet TIV Quality-Aware Feature Aggregation Network for Robust RGBT Tracking Paper
3 2020 XXNet ECCV Challenge-Aware RGBT Tracking Paper
2 2020 XXNet Information Fusion Object fusion tracking based on visible and infrared images: A comprehensive review Paper
1 2020 XXNet CVPR Cross-Modal Pattern-Propagation for RGB-T Tracking Paper
1 2019 XXNet PR RGB-T object tracking: Benchmark and baseline Paper

Datasets

Tools

8. RGB-T ReID

  • VIReID: visible-infrared person re-identification

🚀🚀🚀Update (in 2024-01-07)

No. Year Model Pub. Title Links
219 2024 FDNM arXiv Frequency Domain Nuances Mining for Visible-Infrared Person Re-identification Paper/[Project]

Datasets

  • SYSU-MM01
  • RegDB
  • LLCM

Tools

9. RGB-T Alignment

🚀🚀🚀Update (in 2023-12-26)

No. Year Model Pub. Title Links
219 2023 XMSNet ACM MM Object Segmentation by Mining Cross-Modal Semantics Paper/Project

Datasets

Tools


Ref.

👋👋👋 Thanks to the above authors for their excellent work!

Releases

No releases published

Packages

No packages published