Ef-RAFT

This repository contains the source code for Ef-RAFT: Rethinking RAFT for Efficient Optical Flow

Requirements

The code has been tested with PyTorch 1.6 and Cuda 10.1.

conda create --name efraft
conda activate raft
conda install pytorch=1.6.0 torchvision=0.7.0 cudatoolkit=10.1 matplotlib tensorboard scipy opencv -c pytorch

Required Data

To evaluate/train RAFT, you will need to download the required datasets and put them in datasets/ directory.

• FlyingChairs
• FlyingThings3D
• Sintel
• KITTI

How to run?

For training on 2 GPUs, run the following command. Training logs will be written to the runs directory, which can be visualized using tensorboard.

./train_standard.sh

For running on a single RTX GPU, training can be accelerated using mixed precision, and can be done with the following command. You can expect similiar results in this setting (1 GPU).

./train_mixed.sh

You can evaluate a trained model using evaluate.py.

python evaluate.py --model=models/raft-things.pth --dataset=sintel --mixed_precision

Quantitative Results

Comparison of the proposed method with existing techniques on the Sintel and KITTI datasets. Metrics in green, blue, and red denote the first, second, and third-best results, respectively.

Qualitative Results

Qualitative comparison between the proposed method and RAFT. Frames with orange and blue labels are from Sintel and KITTI datasets, respectively.

Citation

If you use this repository for your research or wish to refer to our method, please use the following BibTeX entry:

@inproceedings{eslami2024rethinking,
  title={Rethinking RAFT for efficient optical flow},
  author={Eslami, Navid and Arefi, Farnoosh and Mansourian, Amir M and Kasaei, Shohreh},
  booktitle={2024 13th Iranian/3rd International Machine Vision and Image Processing Conference (MVIP)},
  pages={1--7},
  year={2024},
  organization={IEEE}
}

Acknowledgement

This codebase is heavily borrowed from RAFT: Recurrent All Pairs Field Transforms for Optical Flow. Thanks for their excellent work.