Install the mmsegmentation library
pip install mmcv-full==1.3.0 mmsegmentation==0.11.0
Please follow the datasets guide of mmseg to prepare the ADE20k dataset.
| Backbone | patch size | mIoU | Config | Weights |
|---|---|---|---|---|
| XCiT-Tiny 12 | 16x16 | 38.1 | config | download |
| XCiT-Tiny 12 | 8x8 | 39.9 | config | download |
| XCiT-Small 12 | 16x16 | 43.9 | config | download |
| XCiT-Small 12 | 8x8 | 44.2 | config | download |
| XCiT-Small 24 | 16x16 | 44.6 | config | download |
| XCiT-Small 24 | 8x8 | 47.1 | config | download |
| XCiT-Medium 24 | 16x16 | 45.9 | config | download |
| XCiT-Medium 24 | 8x8 | 46.9 | config | download |
| Backbone | patch size | mIoU | Config | Weights |
|---|---|---|---|---|
| XCiT-Tiny 12 | 16x16 | 41.5 | config | download |
| XCiT-Tiny 12 | 8x8 | 43.5 | config | download |
| XCiT-Small 12 | 16x16 | 45.9 | config | download |
| XCiT-Small 12 | 8x8 | 46.6 | config | download |
| XCiT-Small 24 | 16x16 | 46.9 | config | download |
| XCiT-Small 24 | 8x8 | 48.1 | config | download |
| XCiT-Medium 24 | 16x16 | 47.6 | config | download |
| XCiT-Medium 24 | 8x8 | 48.4 | config | download |
tools/dist_train.sh <CONFIG_PATH> <NUM_GPUS> --work-dir <SAVE_PATH> --seed 0 --deterministic --options model.pretrained=<IMAGENET_CHECKPOINT_PATH/URL>
For example, using an XCiT-S12/16 backbone with Semantic-FPN
tools/dist_train.sh configs/xcit/sem_fpn/sem_fpn_xcit_small_12_p16_80k_ade20k.py 8 --work-dir /path/to/save --seed 0 --deterministic --options model.pretrained=https://dl.fbaipublicfiles.com/xcit/xcit_small_12_p16_384_dist.pth
tools/dist_test.sh <CONFIG_PATH> <CHECKPOINT_PATH> <NUM_GPUS> --eval mIoU
For example, using an XCiT-S12/16 backbone with Semantic-FPN
tools/dist_test.sh configs/xcit/sem_fpn/sem_fpn_xcit_small_12_p16_80k_ade20k.py https://dl.fbaipublicfiles.com/xcit/ade/sem_fpn_xcit_small_12_p16.pth 1 --eval mIoU
This code is built using the mmsegmentation library. The optimization hyperparameters we use are adopted from Swin Transformer repository.