Undistorting the T265 video feed using the onboard intrinsics, wrong values?

[antithing]

---

Required Info
Camera Model	{ t265 }
Operating System & Version	{Win (10)

| Platform | PC |
| SDK Version | { legacy / 2.20 } |
| Language | {C++} |

Issue Description

I am trying to write a triangulation function with my t265 camera, in order to create a sparse point cloud from feature points.

I am having trouble getting the images to undistort, despite using the factory calibrated values.
I use the intrinsics from the sdk:

		rs2::stream_profile fisheye_stream = pp.get_stream(RS2_STREAM_FISHEYE, 1);
		rs2_intrinsics intrinsicsLeft = fisheye_stream.as<rs2::video_stream_profile>().get_intrinsics();

		
		rs2::stream_profile fisheye_streamR = pp.get_stream(RS2_STREAM_FISHEYE, 2);
		rs2_intrinsics intrinsicsRight = fisheye_streamR.as<rs2::video_stream_profile>().get_intrinsics();

		cv::Mat intrinsicsL = (cv::Mat_<double>(3, 3) <<
			intrinsicsLeft.fx, 0, intrinsicsLeft.ppx,
			0, intrinsicsLeft.fy, intrinsicsLeft.ppy, 0, 0, 1);

		cv::Mat intrinsicsR = (cv::Mat_<double>(3, 3) <<
			intrinsicsRight.fx, 0, intrinsicsRight.ppx,
			0, intrinsicsRight.fy, intrinsicsRight.ppy, 0, 0, 1);

	
		cv::Mat distCoeffsL = cv::Mat(1, 5, CV_32F, intrinsicsLeft.coeffs);
		cv::Mat distCoeffsR = cv::Mat(1, 5, CV_32F, intrinsicsRight.coeffs);

and then run an open cv undistort, using these matrices:

cv::Mat imageUndistorted;
	cv::undistort(left, imageUndistorted, intrinsicsL , distCoeffsL );

and it seems to have only very little effect.

original image:

'undistorted' image:

Where am i going wrong? Are these intrinsics correct?

Thank you,.

[bfulkers-i]

Hi @antithing, because T265 has such a wide field of view, it uses a different distortion model (known as Kannala-Brandt) than the OpenCV default. In OpenCV, this model is referred to in some places as fisheye. Try using this undistortion function:

https://docs.opencv.org/3.4/db/d58/group__calib3d__fisheye.html#ga167df4b00a6fd55287ba829fbf9913b9

[antithing]

Thank you! that is working better. I am now having some problems triangulating the points though.

Is there a function to get the Projection Matrices for the left and right cameras in the sdk?

Are there any plans to expose the points and features that the slam uses to the sdk?

[bfulkers-i]

Hi @antithing, you may be interested in this python example which uses OpenCV to undistort the fisheye images and compute a passive stereo image on the center of the field of view: #3951

[antithing]

Thank you! That looks like what I need. i don't suppose there is a c++ version of this I can look at?

:)