Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

non-finite values in uncertainty #281

Open
Tracked by #274
gerritholl opened this issue Sep 30, 2018 · 1 comment
Open
Tracked by #274

non-finite values in uncertainty #281

gerritholl opened this issue Sep 30, 2018 · 1 comment
Assignees
@gerritholl
Labels
bug Priority: High

Comments

@gerritholl
Copy link
Contributor

gerritholl commented Sep 30, 2018 •
edited
Loading

While debugging the harmonisation input, I found that for shortwave channels (17, 18, 19), there are cases where

[k for k in harm.data_vars.keys() if not harm[k].dtype.kind.startswith("M") and not numpy.isfinite(harm[k]).all().item()]

results in

['nominal_measurand_uncertainty_structured1']

or

['nominal_measurand_uncertainty_independent1']

I need to investigate when and why this happens.
@gerritholl gerritholl self-assigned this Sep 30, 2018
@gerritholl gerritholl changed the title non-finite values in structured uncertainty non-finite values in uncertainty Sep 30, 2018
gerritholl added a commit that referenced this issue Sep 30, 2018
@gerritholl
Add workaround for non-finite structured uncertainties
b91ff0b 
Add workaround so that non-finite structured uncertainties do not cause
bad data to enter the harmonisation files or the writing of harmonisation
files to fail, but rather skip those pixels completely for now.  This
filter must be removed when in the future a fix for #281 shall have been
introduced.
@gerritholl gerritholl mentioned this issue Oct 8, 2018
Open
7 tasks
@gerritholl
Copy link
Contributor Author

I also find uncertainties in the Earth counts, for example in NOAA-16 channel 7:

In [30]: ds = xarray.open_dataset("/gws/nopw/j04/fiduceo/Data/FCDR/HIRS/v0.8pre2_no_harm/debug/noaa16/2004/08/03/FIDUCEO_FCDR_L1C_HIRS3_NOAA16_20040803003812_20040803022004_DEBUG_v0.8pre2_no_harm_fv2.0.0.nc")

In [31]: print(ds["u_C_Earth"].sel(calibration_cycle="2004-08-03T01:54", calibrated_channel=7).item())
nan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gerritholl gerritholl

Labels
bug Priority: High
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@gerritholl