-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix severe bug in mask_fillvalues
#1823
Conversation
@@ -377,7 +377,7 @@ def count_spells(data, threshold, axis, spell_length): | |||
axis += data.ndim | |||
# Threshold the data to find the 'significant' points. | |||
if not threshold: | |||
data_hits = data |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This line lead to the buggy behavior: Since data_hits
is interpreted as bool
later in the code a 0
in data
leads to (undesirable) False
in the data and thus to wrong masking.
Codecov Report
@@ Coverage Diff @@
## main #1823 +/- ##
=======================================
Coverage 91.51% 91.52%
=======================================
Files 203 203
Lines 10937 10937
=======================================
+ Hits 10009 10010 +1
+ Misses 928 927 -1
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tested this bug fix and it works! Without the fix, values that are zero (0.0) are wrongly masked as missing values, see example of observed precipitation here (note e.g. parts of Antarctica):
With the fix, the zeros are kept as valid points, just as it should be:
Here is the test recipe used: recipe_masking_test.yml.txt
Thanks @schlunma for fixing this long-standing bug!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what an ancient bug! Cheers for spotting this @schlunma and cheers for fixing it 🍺
@ESMValGroup/esmvaltool-coreteam anyone time to merge this? Thanks! |
@schlunma Thanks for fixing this! A severe bug deserves an issue, could you please open it? That will make it much easier to find this problem and that it has been solved for anyone who encounters it. I will merge this now, so you can immediately close it again with a reference to this pull request. |
Good point, Bouwe! I wouldn't call it a severe bug myself, but heyho. More of an old, annoying horsefly-like bug 😁 |
Description
This PR fixes a severe bug which lead to a wrong masking of data with
mask_fillvalues
if the data contains zeros. This has already been discussed 2 years ago and has recently lead to problems.I also fixed some prospector issues along the way.
Related to #487
Closes #1827
Before you get started
Checklist
It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.
To help with the number pull requests: