Python (PyTorch) implementation of the D3S tracker, presented at CVPR 2020.
Alan Lukežič, Jiří Matas and Matej Kristan.
D3S - A Discriminative Single Shot Segmentation Tracker.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020.
Paper
BibTex citation:
@InProceedings{Lukezic_CVPR_2020,
Title = {D3S - A Discriminative Single Shot Segmentation Tracker},
Author = {Lukezic, Alan and Matas, Jiri and Kristan, Matej},
Booktitle = {CVPR},
Year = {2020}
}
Template-based discriminative trackers are currently the dominant tracking paradigm due to their robustness, but are restricted to bounding box tracking and a limited range of transformation models, which reduces their localization accuracy. We propose a discriminative single-shot segmentation tracker -- D3S, which narrows the gap between visual object tracking and video object segmentation. A single-shot network applies two target models with complementary geometric properties, one invariant to a broad range of transformations, including non-rigid deformations, the other assuming a rigid object to simultaneously achieve high robustness and online target segmentation. Without per-dataset finetuning and trained only for segmentation as the primary output, D3S outperforms all trackers on VOT2016, VOT2018 and GOT-10k benchmarks and performs close to the state-of-the-art trackers on the TrackingNet. D3S outperforms the leading segmentation tracker SiamMask on video object segmentation benchmarks and performs on par with top video object segmentation algorithms, while running an order of magnitude faster, close to real-time.
git clone https://github.com/alanlukezic/d3s.git .Run the installation script to install all the dependencies. You need to provide the conda install path (e.g. ~/anaconda3) and the name for the created conda environment (here pytracking).
bash install.sh conda_install_path pytrackingTo install the dependencies on a Windows machine, use the install.bat script.
The pre-trained network for the D3S is not part of this repository. You can download it here.
The tracker was tested on the Ubuntu 16.04 machine with a NVidia GTX 1080 graphics card and cudatoolkit version 9. It was tested on Window 10 as well, but network training is tested on Linux only.
1.) Specify the path to the D3S pre-trained segmentation network by setting the params.segm_net_path in the pytracking/parameters/segm/default_params.py.
2.) Specify the path to the VOT 2018 dataset by setting the vot18_path in the pytracking/evaluation/local.py.
3.) Activate the conda environment
conda activate pytracking4.) Run the script pytracking/run_tracker.py to run D3S using VOT18 sequences.
cd pytracking
python run_tracker.py segm default_params --dataset vot18 --sequence <seq_name> --debug 1We provide a VOT Matlab toolkit integration for the D3S tracker. There is the tracker_D3S.m Matlab file in the pytracking/utils, which can be connected with the toolkit. It uses the vot_wrapper.py script to integrate the tracker to the toolkit.
The D3S is pre-trained for segmentation task only on the YouTube VOS dataset. Download the VOS training dataset (2018 version) and copy the files vos-list-train.txt and vos-list-val.txt from ltr/data_specs to the train directory of the VOS dataset.
Set the vos_dir variable in ltr/admin/local.py to the VOS train directory on your machine.
Download the bounding boxes from this link and copy them to the sequence directories.
Run training by running the following command:
python run_training.py segm segm_defaultThis is a modified version of the python framework pytracking based on PyTorch. We would like to thank the authors Martin Danelljan and Goutam Bhat for providing such a great framework.
Check out the video with tracking and segmentation results of the D3S tracker.
- Alan Lukežič (email: alan.lukezic@fri.uni-lj.si)