Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
btilmon authored Jan 1, 2025
1 parent 93bb71d commit 218adcf
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# illumiGrad

Automatically calibrate RGBD cameras online with PyTorch. The main idea is that the camera matrices are wrapped in ```torch.nn.Parameter``` so they can be updated with Adam optimizer based on the reprojection error after differentiably projecting the depth camera into the color camera, which is essentially bundle adjustment without the pose graph. In practice, illumiGrad can improve your computer vision accuracy in the wild without needing precise calibration since the calibration is continuously updated for you, a decent initial calibration is necessary though. No calibration targets, such as checkerboards, are required. I tested on semi-rectified color and Kinect V1 depth cameras from the [NYU Depth V2 dataset](https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html). Related work using reprojection error as a loss signal: [Direct Visual Odometry](https://openaccess.thecvf.com/content_iccv_2013/papers/Engel_Semi-dense_Visual_Odometry_2013_ICCV_paper.pdf), [RGBD Direct Visual Odometry](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.402.5544&rep=rep1&type=pdf), [CVPR 2017](https://arxiv.org/pdf/1704.07813.pdf), [ICCV 2019](https://arxiv.org/pdf/1806.01260.pdf).
Automatically calibrate RGBD cameras online with PyTorch. The main idea is that the camera matrices are wrapped in ```torch.nn.Parameter``` so they can be updated with Adam optimizer based on the reprojection error after differentiably projecting the depth camera into the color camera with ```torch.nn.functional.grid_sample```, which is essentially bundle adjustment without the pose graph. In practice, illumiGrad can improve your computer vision accuracy in the wild without needing precise calibration since the calibration is continuously updated for you, a decent initial calibration is necessary though. No calibration targets, such as checkerboards, are required. I tested on semi-rectified color and Kinect V1 depth cameras from the [NYU Depth V2 dataset](https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html). Related work using reprojection error as a loss signal: [Direct Visual Odometry](https://openaccess.thecvf.com/content_iccv_2013/papers/Engel_Semi-dense_Visual_Odometry_2013_ICCV_paper.pdf), [RGBD Direct Visual Odometry](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.402.5544&rep=rep1&type=pdf), [CVPR 2017](https://arxiv.org/pdf/1704.07813.pdf), [ICCV 2019](https://arxiv.org/pdf/1806.01260.pdf).

<p align="center">
<img src="results/result.gif" alt="example input output gif" />
Expand Down

0 comments on commit 218adcf

Please sign in to comment.