[ICLR 2024] Official implementation of " 🦙 Time-LLM: Time Series Forecasting by Reprogramming Large Language Models"

Official implementation of "CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understanding" (CVPR, 2022)

【CVPR'2023 Highlight & TPAMI】Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?

【AAAI'2023 & IJCV】Transferring Vision-Language Models for Visual Recognition: A Classifier Perspective

【CVPR'2023】Bidirectional Cross-Modal Knowledge Exploration for Video Recognition with Pre-trained Vision-Language Models

[CVPR 2023 Highlight 💡] Hidden Gems: 4D Radar Scene Flow Learning Using Cross-Modal Supervision

CVPR 2022: Cross-Modal Perceptionist: Can Face Geometry be Gleaned from Voices?

[ICLR 2023] Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?

Code, dataset and models for our CVPR 2022 publication "Text2Pos"

[AAAI 2024] DGL: Dynamic Global-Local Prompt Tuning for Text-Video Retrieval.

[IJCAI 2025] Official implementation of "T2S: High-resolution Time Series Generation with Text-to-Series Diffusion Models"

This is a cross-modal benchmark for industrial anomaly detection.

[IJBHI 2024] This is the official implementation of CAMANet: Class Activation Map Guided Attention Network for Radiology Report Generation accepted to IEEE Journal of Biomedical and Health Informatics (J-BHI), 2023.

Official Pytorch Implementation of SeisMoLLM: Advancing Seismic Monitoring via Cross-modal Transfer with Pre-trained Large Language Model

Original PyTorch implementation of the code for the paper "Straight to the Point: Fast-forwarding Videos via Reinforcement Learning Using Textual Data" at the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020

Code for Limbacher, T., Özdenizci, O., & Legenstein, R. (2022). Memory-enriched computation and learning in spiking neural networks through Hebbian plasticity. arXiv preprint arXiv:2205.11276.

CUCA: Predicting fine-grained cell types from histology images through cross-modal learning in spatial transcriptomics

Code for the "Sample-efficient Integration of New Modalities into Large Language Models" paper

CroMoTEX: Contrastive Cross-Modal Learning for Infusing Chest X-ray Knowledge into ECGs

We design a cross-modal GAN which learns image-to-image modality transformation across cross-domain. This network is able to synthesize Infrared images from VISIBLE images for VEDAI dataset