drivers: Accept entire distance sensor signal strength range

[sverrevr]

Solved Problem

The reason for this commit is that we had a flight where the distance sensor was in practice disabled for large parts of the flight as the signal strength being read from the sensor was lower than 70/255 which is today's threshold. Looking at the sensor data itself we see that there were a lot of measurements with good quality that should not have been discarded.

See for example this section here where all data is discarded for more than 20 seconds even thought the sensor data is perfect

To explore where the limit should be, we set the minimal signal strength to 0 so that the signal strength is directly mapped to distance_sensor/signal_quality

What we saw from our experiments were that we only had outlier measurements where the signal strength was 0. Any non zero values corresponded to a good measurement. See comments below for plots

Looking at the datasheet we have the following documentation: https://static.garmin.com/pumac/LIDAR-Lite_v3HP_Instructions_EN.pdf

So signal strength is already capped with a lower threshold calculated based on the noise floor. From this its not unreasonable that all non-zero measurements are likely good.

Furthermore, at least for the terrain estimator, its not a problem if outliers are not removed at this step, as there already is outlier detection there. Is this also the case if the distance sensor is used directly in the EKF?

Changelog Entry

For release notes:

Accept entire distance sensor signal strength
New parameter: None
Documentation:  

Test coverage

Flying again in the same location where the problem was first observed yielded good result where the distance sensor was used and no bad measurements were accepted.

Context

Related links, screenshot before/after, video

[sverrevr]

Logs from mapping entire signal strength to 0-100 signal quality:

Note that most of the "spikes" here are actual things covering the sensor, such as the drone flying over a wall.

The only thing that looks like a measurement error is this which has signal strength 0

Note that the signal quality drops when entering and exiting an edge:


Some of these have non zero value, but the outliers are not very large.

Here is a region with low signal strength, but good data:

For our use-case it would be better to keep these small outliers when entering/exiting an edge, than loosing a lot of good data under normal flight.

[sverrevr]

Very open for discussion on philosophy on how this value should be chosen.

[sverrevr]

The parameters where introduced here: #12756

[DronecodeBot]

This pull request has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/px4-community-q-a-november-22-2023/35324/1

[MaEtUgR]

Communitiy Q&A call: Let's open the full sensor quality range on the driver level but add a quality threshold downstream at the EKF to make sure it can be tuned for the particular sensor and use case. As a reference it should be similar to EKF2_RNG_QLTY_T but a threshold for the quality such that it's not just zero or higher.

[sverrevr]

Communitiy Q&A call: Let's open the full sensor quality range on the driver level but add a quality threshold downstream at the EKF to make sure it can be tuned for the particular sensor and use case. As a reference it should be similar to EKF2_RNG_QLTY_T but a threshold for the quality such that it's not just zero or higher.

Do we want the default value to be accepting the entire range or still have the threshold at 70/255?

[dagar]

Communitiy Q&A call: Let's open the full sensor quality range on the driver level but add a quality threshold downstream at the EKF to make sure it can be tuned for the particular sensor and use case. As a reference it should be similar to EKF2_RNG_QLTY_T but a threshold for the quality such that it's not just zero or higher.

Do we want the default value to be accepting the entire range or still have the threshold at 70/255?

For now it should still default to the current behaviour (ekf2 takes it unless 0/invalid), but someone with this newer lidarlite can start setting it if truly necessary.

• new parameter in ekf2 (search for ekf2_of_qmin for a somewhat similar example) https://github.com/PX4/PX4-Autopilot/blob/e280a6c6e68fdc2804cf63754e83a412c1421bf6/src/modules/ekf2/EKF2.cpp#L185C31-L185C44
• range finder validity is checked in this class here

  PX4-Autopilot/src/modules/ekf2/EKF/sensor_range_finder.cpp

  Lines 77 to 80 in e280a6c

  	if (_sample.quality == 0) {
  	_time_bad_quality_us = current_time_us;
  	} else if (current_time_us - _time_bad_quality_us > _quality_hyst_us) {

• pass the new quality min parameter into SensorRangeFinder here

  PX4-Autopilot/src/modules/ekf2/EKF/range_height_control.cpp

  Line 55 in e280a6c

  _range_sensor.setQualityHysteresis(_params.range_valid_quality_s);

[sverrevr]

From flight test with the new parameter.
At the start i had EKF2_RNG_QMIN = 90, and then only small sections of the data was accepted
Then i changed to EKF2_RNG_QMIN = 0 and moslty everything was accepted (bit unsure what happened a bit after the 1:1:40 mark)
Then I changed to EKF2_RNG_QMIN = 100 and everything was rejected

[sverrevr]

I think the PR is ready for a new review