Plotting "line number" rather than "frame index" in optimization plots

[bpurinton]

Related to #35 and #42

Below taken from email thread:

Currently, for camera optimization plots our x-axis is "Frame Index" (or "Linescan Sample"), which is just the position / rotation sampling index taken from the CSM JSON file. In theory, is there a way that we could, perhaps with some known sampling interval, retrieve the approximate Line Number for the x-axis in my updated summary plots, instead of the “Frame Index” (which is really some CSM sampling interval)? Perhaps the answer lies in the original XML camera file (for the original, pre-CSM conversion)?

---

Another way to put this - we need to know where Line 1 of the actual image is in the csm records, as the ephemeris and attitude records start before the first line, and continue after the last line. We want to chop these plots from ~600-1200 on the x axis, assuming those are the indices for the start and end of the linescan image.

---

What you want is rather hairy as this is deep inside the internals of the CSM model.

Here's the C++ logic:

https://github.com/NeoGeographyToolkit/StereoPipeline/blob/master/src/asp/Camera/CsmUtils.cc#L500
https://github.com/NeoGeographyToolkit/StereoPipeline/blob/master/src/asp/Camera/CsmUtils.cc#L472
https://github.com/DOI-USGS/usgscsm/blob/main/src/UsgsAstroLsSensorModel.cpp#L1328

The short story is that you need to go from position index (and quaternion index) to time, for which there is an affine relationship (y = a*x + b). Then, from time, need to go to image line, for which there is also an affine relationship. Unfortunately, the devil is in the details.

I must say that the USGS CSM model is overly complicated, and one could spend easily a few hours trying to sort things out. As you can see in the code, there is m_intTimeLines, m_intTimeStartTimes, m_intTimes, and one also assumes a piecewise linear relationship between time and lines (though in our case there is only one piece, so the relationship is affine).

So, lots of pain, really, to get to the bottom of it, given how simple the concept is.

---

So there not an existing Python function for this somewhere in the USGS repos?

Here: https://github.com/DOI-USGS/swigcsm

In particular: https://github.com/DOI-USGS/swigcsm/blob/main/python/tests/test_model.py

If no luck, that C++ code should be rather easy to imitate. One should just remember that m_intTimeLines and the rest of arrays have size 1, so need to access only 0th element, rather than doing a search. Those values give the slope and the intercept of the affine function that goes from line to time

[oleg-alexandrov]

I added to orbit_plot.py logic to go from time to image line and vice versa, and for how to go from the index in the list of orientations to time for that orientation.

The implementation can be seen in this diff: oleg-alexandrov/StereoPipeline@cb84a97

It is likely better to just copy the needed functions over as they are small.

It is a bit unclear from that code how to go from pose index to time at that pose. The formula is time = t0 + index * dt, where t0 and dt are read from the json file fields m_t0Quat and m_dtQuat.

[bpurinton]

🙌 Nice! I'll work on porting that logic over in the next days.

[oleg-alexandrov]

For the record, I found a bug in my logic for going from line number to quaternion index. I put a fix here: oleg-alexandrov/StereoPipeline@c4c8ff6

[oleg-alexandrov]

The tool should be able to handle the two linescan cameras having different sampling rates, because the jitter solver can resample the positions and orientations. I put a fix for this (for orientations only) in orbit_plot at: NeoGeographyToolkit/StereoPipeline@f61c085

[bpurinton]

Nice, I already handled that case in the code I wrote previously. On a quick read, it looks like both overall approaches are comparable and come down to a conditional linear interpolation. So I won't port over the same change to my version.

Here are those calculations in my code:

asp_plot/asp_plot/csm_camera.py

Lines 82 to 93 in e6b9fb0

	# Interpolate original values if lengths don't match
	if len(original_positions_ecef) != len(optimized_positions_ecef):
	original_positions_ecef = np.array(
	[
	np.interp(
	np.linspace(0, 1, len(optimized_positions_ecef)),
	np.linspace(0, 1, len(original_positions_ecef)),
	original_positions_ecef[:, i],
	)
	for i in range(3)
	]
	).T

and:

asp_plot/asp_plot/csm_camera.py

Lines 105 to 121 in e6b9fb0

	# Interpolate original angles if lengths don't match
	if len(original_roll) != len(optimized_roll):
	original_roll = np.interp(
	np.linspace(0, 1, len(optimized_roll)),
	np.linspace(0, 1, len(original_roll)),
	original_roll,
	)
	original_pitch = np.interp(
	np.linspace(0, 1, len(optimized_pitch)),
	np.linspace(0, 1, len(original_pitch)),
	original_pitch,
	)
	original_yaw = np.interp(
	np.linspace(0, 1, len(optimized_yaw)),
	np.linspace(0, 1, len(original_yaw)),
	original_yaw,
	)