Awesome-LiDAR-Camera-Calibration

A Collection of LiDAR-Camera-Calibration Papers, Toolboxes and Notes.

Outline

• Introduction
• Target-based methods
• Targetless methods
  • Motion-based methods
  • Scene-based-methods
    • Traditional Methods
    • Deep-learning Methods
• Other Toolboxes

0. Introduction

For applications such as autonomous driving, robotics, navigation systems, and 3-D scene reconstruction, data of the same scene is often captured using both lidar and camera sensors. To accurately interpret the objects in a scene, it is necessary to fuse the lidar and the camera outputs together. Lidar camera calibration estimates a rigid transformation matrix (extrinsics, rotation+translation, 6 DoF) that establishes the correspondences between the points in the 3-D lidar plane and the pixels in the image plane.

1. Target-based methods

Paper	Target	Feature	Optimization	Toolbox	Note
Extrinsic Calibration of a Camera and Laser Range Finder (improves camera calibration), 2004	checkerboard	C:Plane (a), L: pts in plane (m)	point-to-plane	CamLaserCalibraTool	CN
Fast Extrinsic Calibration of a Laser Rangefinder to a Camera, 2005	checkerboard	C: Plane (a), L: Plane (m)	plane(n/d) correspondence, point-to-plane	LCCT	*
Extrinsic calibration of a 3D laser scanner and an omnidirectional camera, 2010	checkerboard	C: plane (a), L: pts in plane (m)	point-to-plane	cam_lidar_calib	*
LiDAR-Camera Calibration using 3D-3D Point correspondences, 2017	cardboard + ArUco	C: 3D corners (a), L: 3D corners (m)	ICP	lidar_camera_calibration	*
Reflectance Intensity Assisted Automatic and Accurate Extrinsic Calibration of 3D LiDAR and Panoramic Camera Using a Printed Chessboard, 2017	checkerboard	C: 2D corners (a), L: 3D corners (a)	PnP, angle difference	ILCC	*
Extrinsic Calibration of Lidar and Camera with Polygon, 2018	regular cardboard	C: 2D edge, corners (a), L: 3D edge, pts in plane (a)	point-to-line, point-inside-polygon	ram-lab/plycal	*
Automatic Extrinsic Calibration of a Camera and a 3D LiDAR using Line and Plane Correspondences, 2018	checkerboard	C: 3D edge, plane(a), L: 3D edge, pts in plane (a)	direcion/normal, point-to-line, point-to-plane	Matlab LiDAR Toolbox	*
Improvements to Target-Based 3D LiDAR to Camera Calibration, 2020	cardboard with ArUco	C: 2d corners (a), L: 3D corners (a)	PnP, IOU	github	*
ACSC: Automatic Calibration for Non-repetitive Scanning Solid-State LiDAR and Camera Systems, 2020	checkerboard	C: 2D corners (a), L: 3D corners (a)	PnP	ACSC	*
Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups, 2021	cardboard with circle & Aruco	C: 3D points (a), L: 3D points (a)	ICP	velo2cam_ calibration	*
Single-Shot is Enough: Panoramic Infrastructure Based Calibration of Multiple Cameras and 3D LiDARs,2021 IROS	panoramic infrastructure	C: CCTag(a), L: corner points and vectors(a)	PnP, ICP	multiple-cameras-and-3D-LiDARs-extrinsic-calibration	*
Joint Camera Intrinsic and LiDAR-Camera Extrinsic Calibration, 2023, ICRA	chessboard + circles	C: chessboard corners, circle centers, L: circle centers	PnP, intrinsic constraints (corners, cirlce center)	OpenCalib/JointCalib	CN
LCE-Calib: Automatic LiDAR-Frame/Event Camera Extrinsic Calibration With a Globally Optimal Solution	checkboard	3D point-to-3D plan, 3D point-to-3D edge	point-to-plane, point-to-edge	LCECalib	*

C: camera, L: LiDAR, a: automaic, m: manual

2. Targetless methods

2.1. Motion-based methods

Paper	Feature	Optimization	Toolbox	Note
LiDAR and Camera Calibration Using Motions Estimated by Sensor Fusion Odometry, 2018	C: motion (ICP), L: motion (VO)	hand-eye calibration	*	*

2.2. Scene-based methods

2.2.1. Traditional methods

Paper	Feature	Optimization	Toolbox	Note
Automatic Targetless Extrinsic Calibration of a 3D Lidar and Camera by Maximizing Mutual Information, 2012	C：grayscale, L: reflectivity	mutual information, BB steepest gradient ascent	Extrinsic Calib	*
Automatic Calibration of Lidar and Camera Images using Normalized Mutual Information, 2013	C:grayscale, L: reflectivity, noraml	normalized MI, particle swarm	*	*
Automatic Online Calibration of Cameras and Lasers, 2013	C: Canny edge, L: depth-discontinuous edge	correlation, grid search	*	*
SOIC: Semantic Online Initialization and Calibration for LiDAR and Camera, 2020	semantic centroid	PnP	*	*
A Low-cost and Accurate Lidar-assisted Visual SLAM System, 2021	C: edge(grayscale), L: edge (reflectivity, depth projection)	ICP, coordinate descent algorithms	CamVox	*
Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments,2021	C:Canny edge(grayscale), L: depth-continuous edge	point-to-line, Gaussian-Newton	livox_camera_calib	*
CRLF: Automatic Calibration and Refinement based on Line Feature for LiDAR and Camera in Road Scenes, 2021	C:straight line, L: straight line	perspective3-lines (P3L)	*	CN
Fast and Accurate Extrinsic Calibration for Multiple LiDARs and Cameras,2021	C:Canny edge(grayscale), L: depth-continuous edge	point-to-plane, point-to-edge	mlcc	*
General, Single-shot, Target-less, and Automatic LiDAR-Camera Extrinsic Calibration Toolbox,ICRA 2023	C:keypoints, L: keypoints on intensity images (by SuperGlue)	reprojection error minimization, RANSAC	direct_visual_lidar_calibration	*
Calib-Anything: Zero-training LiDAR-Camera Extrinsic Calibration Method Using Segment Anything, 2023	C: Segmentation, L: normal estimation, plane fitting & euclidean cluster, intensity normalization	random search, grid search	OpenCalib	*

2.2.2. Deep-learning methods

Paper	Toolbox	Note
RegNet: Multimodal sensor registration using deep neural networks, 2017,IV	regnet	*
CalibNet: Geometrically supervised extrinsic calibration using 3d spatial transformer networks,2018,IROS	CalibNet	*
LCCNet: Lidar and Camera Self-Calibration using CostVolume Network,2021,CVPRW	LCCNet	*
Calib-Anything: Zero-training LiDAR-Camera Extrinsic Calibration Method Using Segment Anything,2023	Calib-Anything	*

3. Other toolboxes

Toolbox	Introduction	Note
Apollo sensor calibration tools	targetless method, no source code	CN
Autoware camera lidar calibrator	pick points mannually, PnP	*
Autoware calibration camera lidar	checkerboard, similar to LCCT	CN
livox_camera_lidar_calibration	pick points mannually, PnP	*
OpenCalib	target-based, target-less, mannual	OpenCalib: A Multi-sensor Calibration Toolbox for Autonomous Driving
tier4/CalibrationTools	target-based, mannual	*