Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix da.broadcast_to call when the fx cube has different shape than target data cube #1350

Merged
merged 3 commits into from
Oct 8, 2021

Conversation

valeriupredoi
Copy link
Contributor

@valeriupredoi valeriupredoi commented Oct 8, 2021

Description

We spotted a bug in the landsea masking related to the broadcasting of the fx data array onto the cube to be masked data array.

Closes #1349


Before you get started

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.


To help with the number pull requests:

@zklaus
Copy link

zklaus commented Oct 8, 2021

I'm willing to accept this as a hot-fix as-is. But would you consider adding the test? If not, let's open an issue about that.

@codecov
Copy link

codecov bot commented Oct 8, 2021

Codecov Report

Merging #1350 (40b3bf3) into main (d587d0c) will increase coverage by 0.02%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1350      +/-   ##
==========================================
+ Coverage   88.24%   88.26%   +0.02%     
==========================================
  Files         194      194              
  Lines        9831     9831              
==========================================
+ Hits         8675     8677       +2     
+ Misses       1156     1154       -2     
Impacted Files Coverage Δ
esmvalcore/preprocessor/_mask.py 85.21% <100.00%> (+0.86%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d587d0c...40b3bf3. Read the comment docs.

@valeriupredoi valeriupredoi added this to the v2.4.0 milestone Oct 8, 2021
@valeriupredoi valeriupredoi added bug Something isn't working testing labels Oct 8, 2021
@valeriupredoi
Copy link
Contributor Author

@zklaus sorry, was working on the test, now included too, it's a cold-fix now 😁

@sloosvel
Copy link
Contributor

sloosvel commented Oct 8, 2021

Sorry about the slip-up. I think I spotted more buggy things in #1269 , can they be fixed here as well?

@zklaus
Copy link

zklaus commented Oct 8, 2021

If it's small, yes, if it's not so small, perhaps we merge this first and take it separately. What did you discover?

@valeriupredoi
Copy link
Contributor Author

be my guest @sloosvel - cheers! Make sure you pull the latest, I just added a bunch o'changes to the integration test mask 👍 Let's agree we want this in 2.4 so let's hustle up on it, and get it ready before the day of release 🍺

@sloosvel
Copy link
Contributor

sloosvel commented Oct 8, 2021

def _preserve_fx_vars(cube, result):
    vertical_dim = set(cube.coord_dims(cube.coord(axis='z', dim_coords=True)))
    if cube.cell_measures():
        for measure in cube.cell_measures():
            measure_dims = set(cube.cell_measure_dims(measure))
            if vertical_dim.intersection(measure_dims):
                logger.warning(
                    'Discarding use of z-axis dependent cell measure %s '
                    'in variable %s, as z-axis has been interpolated',
                    measure.var_name, result.var_name)
            else:
                add_cell_measure(result, measure, measure.measure)
    if cube.ancillary_variables():
        for ancillary_var in cube.ancillary_variables():
            ancillary_dims = cube.ancillary_variable_dims(ancillary_var)   --> this should be set(cube.ancillary_variable_dims(ancillary_var) )
            if vertical_dim.intersection(ancillary_dims):
                logger.warning(
                    'Discarding use of z-axis dependent ancillary variable %s '
                    'in variable %s, as z-axis has been interpolated',
                    ancillary_var.var_name, result.var_name)
            else:
                add_ancillary_variable(result, ancillary_var)

And I only added tests for climate_statistics, they should also be added for decadal_statistics, annual_statistics, etc.

@zklaus
Copy link

zklaus commented Oct 8, 2021

Since this is already done and the checks are passed, I'd rather merge it as is. The changes you mention in total seem a bit more substantial. Please open a new PR.

@zklaus
Copy link

zklaus commented Oct 8, 2021

@valeriupredoi, could you please take care of the checklist while I do the review?

Copy link

@zklaus zklaus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minimal change requested, also ok as-is.

@valeriupredoi
Copy link
Contributor Author

OK @zklaus cheers for taking a look/reviewing - I agree we can merge and then Saskia can do the other changes in a subsequent PR (bit too much for this PR methinks too). Cheers both for looking into it! 🍺

@zklaus zklaus merged commit 1ee1716 into main Oct 8, 2021
@zklaus zklaus deleted the fix_broadcast_to_issue_mask branch October 8, 2021 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

mask_landsea fails when using 2D fx data on 4D cube data
3 participants