Ofiicial Implementation for Mamba-ND: Selective State Space Modeling for Multi-Dimensional Data

Updates

• Jul 1: Our paper was accepted at ECCV 2024. We also released K400 results and checkpoints.

Paper

Model Zoo

Image Classification

Checkpoints available at

Syntax	Acc	Weight
Mamba2D-S/8	81.7	weight
Mamba2D-B/8	83.0	weight

Video Classification

Syntax	Acc	Weight
UCF-101 (Scratch)	89.6	weight
HMDB-51 (Scratch)	60.9	weight
K400	81.9	weight

3D Segmentation

Syntax	Feature Size	Dice	Weight
Mamba3D-S/16	32	83.1	weight
Mamba3D-S+/16	32	83.9	weight
Mamba3D-B/16	32	82.7	weight
Mamba3D-B/16	64	84.7	weight

Environment Setup

pip install causal-conv1d>=1.2.0
git install git+https://github.com/state-spaces/mamba.git

For image classification, mmpretrain is required. For video classification, mmaction is required. Please see offical documentation for installation instructions.

Training

Please see refer to the following instructions for each task:

Image classification Video classification Video classification (K400 Pretraining) 3D segmentation

Citation

@article{li2024mamba,
  title={Mamba-ND: Selective State Space Modeling for Multi-Dimensional Data},
  author={Li, Shufan and Singh, Harkanwar and Grover, Aditya},
  journal={arXiv preprint arXiv:2402.05892},
  year={2024}
}