BICAS: (1) Reorg. L2 saturation quality flags; enable L3 to (2a) only use non-saturated channels and (2b) raise L3 QUALITY_FLAG

[ErikPGJ]

Problem

L2 datasets only contain one "full saturation" quality flag (boolean/bit) which is set if any L2 channel is saturated. This means that the code for deriving of L3 data can not even in principle (1) tell whether or not saturation occurs on an irrelevant channel (e.g. AC diff), (2) derive values from a smaller number of non-saturated channels (avoid saturated channels).

Problem has been discussed on Slack:

• solar_orbiter_bias 2024-05-07
• solar_orbiter_science 2024-01-11 to 20
• solar_orbiter_science 2024-06-04
• (and maybe more).

Solution to be implemented

It was agreed on IRFU RPW/BIAS calibration meeting 2024-06-04 to implement functionality for reorganizing saturation quality flags and let code for deriving L3 use them.

• Saturation quality flags (zVariable L2_QUALITY_BITMASK):

  • Old (to be removed):
    • "Full saturation": Set automatically and through NSO. Set if at least one channel is saturated.
    • "Partial saturation": Set through NSO. Few examples. Unclear definition.
  • New:
    • V1, V2, V3: One per single
    • E12, E13, E23: One per diff DC+AC. Set if any one of the AC or DC diff is saturated.
      • Note: One can not have DC diff and AC diff data simultaneously due to the h/w design.

• Notes:

  • The L2 dataset change is "backwards-incompatible" since old bits are redefined.
  • L2_QUALITY_BITMASK is copied to L3 datasets due to archiving conventions.
  • L2_QUALITY_BITMASK has 16 bits, though in order to be strict with respect to fill values, one should only use 15 bits.
  • According to ROC's old archiving rules, zVariable QUALITY_FLAG can never be higher than in its parent datasets. L2 saturation lowers the L2 QUALITY_FLAG while with the suggested scheme, L3 QUALITY_FLAG could legitimately be higher than the parent L2 dataset, e.g. if only channels irrelevant for L3 are saturated in L2. Xavier Bonnin at LESIA has changed the QUALITY flag rules to approve of this. QUALITY_FLAG is now allowed to be higher than in parent datasets.

• TODO-DEC: How should BICAS communicate which channels are saturated to solo.psp2ne() and solo.vdccal()?

  • PROPOSAL: Set saturated channel data to NaN.
  • PROPOSAL: Argument for saturation flags.