SRFeat: Learning Locally Accurate and Globally Consistent Non-Rigid Shape Correspondence

By Lei Li, Souhaib Attaiki, Maks Ovsjanikov. (3DV 2022)

In this work, we present a novel learning-based framework that combines the local accuracy of contrastive learning with the global consistency of geometric approaches, for robust non-rigid matching. We first observe that while contrastive learning can lead to powerful point-wise features, the learned correspondences commonly lack smoothness and consistency, owing to the purely combinatorial nature of the standard contrastive losses. To overcome this limitation we propose to boost contrastive feature learning with two types of smoothness regularization that inject geometric information into correspondence learning. With this novel combination in hand, the resulting features are both highly discriminative across individual points, and, at the same time, lead to robust and consistent correspondences, through simple proximity queries. Our framework is general and is applicable to local feature learning in both the 3D and 2D domains. We demonstrate the superiority of our approach through extensive experiments on a wide range of challenging matching benchmarks, including 3D non-rigid shape correspondence and 2D image keypoint matching.

Link

Paper

Citation

@inproceedings{li2022srfeat,
  title={{SRFeat}: Learning Locally Accurate and Globally Consistent Non-Rigid Shape Correspondence},
  author={Li, Lei and Attaiki, Souhaib and Ovsjanikov, Maks},
  booktitle={International Conference on 3D Vision (3DV)},
  year={2022},
  organization={IEEE}
}

3D Shape Matching

Dependencies

• CUDA 11
• Python 3.8
• Pytorch 1.8.1

Other used Python packages are listed in requirements.txt. It is preferable to create a new conda environment for installing the packages.

The docker image that has a complete running environment can be found here.

Data

The data and pretrained models can be found here. Extract the content of the zipped file to the root directory of the code.

Training & Testing

python trainer_srfeatd.py skip_train=False config=exp/srfeatd_***.yml 

python trainer_srfeats.py skip_train=False config=exp/srfeats_***.yml

Use Pretrained Model

python <<trainer_srfeatd.py/trainer_srfeats.py>> skip_train=True test_ckpt=<<exp/../ckpt_latest.pth>> path_prefix=. log_dir=exp/log data.data_root=exp/data

Evaluation

First, compute geodesic distance matrices by running scripts/computeGeoDistMat.m.

Then,

python eval_corr.py --data_root exp/data --test_roots exp/log/<<FolderName>> ...

Image Keypoint Matching

Please check the branch dgmc.

References

1. Donati et al. Deep Geometric Maps: Robust Feature Learning for Shape Correspondence. CVPR 2020.
2. Xie et al. PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding. ECCV 2020.
3. Sharp et al. DiffusionNet: Discretization Agnostic Learning on Surfaces. TOG 2022.
4. Fey et al. Deep Graph Matching Consensus. ICLR 2020.

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.