Mobt 496 enforce forecast consistency #1900

brhooper · 2023-04-28T10:53:36Z

Addresses https://github.com/metoppv/mo-blue-team/issues/496

Creates a plugin and corresponding CLI which enables the general enforcement of consistency between a forecast cube and a linear transformation of a reference cube. This plugin specifically includes the functionality to enforce consistency between cloud amount probability forecasts (here), enforcing consistency between wind gust and wind speed percentile forecasts (here), and should also provide a general solution to requirements for enforcing consistency between forecast cubes when future use-cases arise.

The previous specific implementation of a plugin to enforce consistency between cloud amount forecasts is removed.

Acceptance test data: metoppv/improver_test_data#6

Description

Testing:

Ran tests and they passed OK
Added new tests for the new feature(s)

…dd corresponding unit and acceptance tests.

codecov · 2023-04-28T11:02:22Z

Codecov Report

Merging #1900 (834ea2a) into master (c2df37b) will increase coverage by 1.33%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1900      +/-   ##
==========================================
+ Coverage   96.99%   98.32%   +1.33%     
==========================================
  Files         119      120       +1     
  Lines       11329    11376      +47     
==========================================
+ Hits        10989    11186     +197     
+ Misses        340      190     -150

Impacted Files	Coverage Δ
improver/calibration/reliability_calibration.py	`98.94% <ø> (-0.05%)`	⬇️
improver/utilities/enforce_consistency.py	`100.00% <100.00%> (ø)`

... and 10 files with indirect coverage changes

Katie-Howard

I have looked through all your changes, and they generally look sensible and like they do what they are supposed to 👍 I just have a few minor comments for you to look at.

I have run the unit and acceptance tests, and they all run fine, apart from one. There was one failure in improver_tests/test_source_code.py in test_init_files_exist(), where there was an AssertionError.

improver_tests/acceptance/test_enforce_consistent_forecasts.py

improver_tests/utilities/test_enforce_consistency.py

brhooper · 2023-05-04T12:58:43Z

Thanks for your review @Katie-Howard. I've responded to your 2 review comments.

I haven't been able to reproduce the failure of improver_tests/test_source_code.py in test_init_files_exist() which you have been experiencing. Is it possible it could be an issue at your end?

Katie-Howard

I've looked through the comments and changes and I'm happy with this. If the unit tests pass for you, it must be something on my system.

gavinevans

Thanks @brhooper 👍

I've added some very minor comments.

improver/utilities/enforce_consistency.py

improver_tests/acceptance/SHA256SUMS

improver_tests/utilities/test_enforce_consistency.py

improver/cli/enforce_consistent_forecasts.py

gavinevans

Thanks @brhooper 👍

I've added a few queries.

improver/utilities/enforce_consistency.py

improver_tests/utilities/test_enforce_consistency.py

improver/utilities/enforce_consistency.py

improver/cli/enforce_consistent_forecasts.py

…force_forecast_consistency

…it and acceptance tests for realization data. Update checksum for new acceptance test data.

…ability reference forecasts.

gavinevans

Thanks @brhooper 👍

I've added a few minor comments.

improver/cli/enforce_consistent_forecasts.py

improver/utilities/enforce_consistency.py

improver_tests/utilities/test_enforce_consistency.py

improver/cli/enforce_consistent_forecasts.py

improver_tests/utilities/test_enforce_consistency.py

gavinevans

Thanks @brhooper 👍

I've added a few minor comments.

…ance test data.

gavinevans · 2023-05-16T12:23:04Z

This PR now seems fine to me and can be merged when an associated suite change is made.

* Remove obsolete enforce_consistent_probabilities code. Update checksums * Add plugin and CLI for general enforcement of forecast consistency. Add corresponding unit and acceptance tests. * Fix error in acceptance test. * Corrections to formatting. * Remove obsolete module import. * Update docstrings. * Minor change following review. * Minor changes following review * Updated checksums following reduction in acceptance test data size * Update docstrings to include applications to realization data. Add unit and acceptance tests for realization data. Update checksum for new acceptance test data. * Add limitation that only the identity function can be applied to probability reference forecasts. * Formatting corrections * Minor changes following review. Update checksums for change to acceptance test data.

brhooper added 4 commits April 28, 2023 11:08

Remove obsolete enforce_consistent_probabilities code. Update checksums

daede1f

Add plugin and CLI for general enforcement of forecast consistency. A…

3a9c8f0

…dd corresponding unit and acceptance tests.

Fix error in acceptance test.

a337a00

Corrections to formatting.

c2617f8

brhooper mentioned this pull request Apr 28, 2023

Mobt 496 enforce forecast consistency metoppv/improver_test_data#6

Merged

brhooper added 2 commits April 28, 2023 12:03

Remove obsolete module import.

5f53eca

Update docstrings.

79200c5

mspelman07 added this to the IM2023.1 milestone May 2, 2023

brhooper added the don't merge yet label May 2, 2023

Katie-Howard self-requested a review May 3, 2023 09:27

mspelman07 assigned Katie-Howard May 4, 2023

Katie-Howard requested changes May 4, 2023

View reviewed changes

improver_tests/acceptance/test_enforce_consistent_forecasts.py Show resolved Hide resolved

improver_tests/utilities/test_enforce_consistency.py Outdated Show resolved Hide resolved

Katie-Howard assigned brhooper and Katie-Howard and unassigned Katie-Howard and brhooper May 4, 2023

Minor change following review.

2705bb8

brhooper assigned Katie-Howard and unassigned brhooper May 4, 2023

Katie-Howard previously approved these changes May 4, 2023

View reviewed changes

Katie-Howard removed their assignment May 4, 2023

mspelman07 assigned gavinevans May 5, 2023

gavinevans requested changes May 5, 2023

View reviewed changes

gavinevans assigned brhooper and unassigned gavinevans May 5, 2023

gavinevans requested changes May 5, 2023

View reviewed changes

brhooper added 2 commits May 9, 2023 15:14

Minor changes following review

2ab39cd

Merge branch 'master' of github.com:metoppv/improver into mobt_496_en…

c73d3d8

…force_forecast_consistency

brhooper dismissed Katie-Howard’s stale review via c73d3d8 May 9, 2023 14:32

Updated checksums following reduction in acceptance test data size

5891c7a

brhooper assigned gavinevans and unassigned brhooper May 11, 2023

Update docstrings to include applications to realization data. Add un…

a5eef62

…it and acceptance tests for realization data. Update checksum for new acceptance test data.

brhooper assigned brhooper and unassigned gavinevans May 12, 2023

brhooper added 2 commits May 15, 2023 08:51

Add limitation that only the identity function can be applied to prob…

5ef3d87

…ability reference forecasts.

Formatting corrections

b4e002e

brhooper assigned brhooper and gavinevans and unassigned brhooper May 15, 2023

gavinevans requested changes May 15, 2023

View reviewed changes

Minor changes following review. Update checksums for change to accept…

834ea2a

…ance test data.

gavinevans approved these changes May 16, 2023

View reviewed changes

brhooper assigned brhooper and unassigned gavinevans May 16, 2023

brhooper added the Paused label May 22, 2023

gavinevans removed don't merge yet Paused labels May 26, 2023

gavinevans merged commit 597b578 into metoppv:master May 26, 2023

brhooper mentioned this pull request Jun 2, 2023

Mobt 496 enforce forecast between references #1907

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mobt 496 enforce forecast consistency #1900

Mobt 496 enforce forecast consistency #1900

brhooper commented Apr 28, 2023 •

edited

Loading

codecov bot commented Apr 28, 2023 •

edited

Loading

Katie-Howard left a comment

brhooper commented May 4, 2023

Katie-Howard left a comment

gavinevans left a comment

gavinevans left a comment

gavinevans left a comment

gavinevans left a comment

gavinevans commented May 16, 2023

Mobt 496 enforce forecast consistency #1900

Mobt 496 enforce forecast consistency #1900

Conversation

brhooper commented Apr 28, 2023 • edited Loading

codecov bot commented Apr 28, 2023 • edited Loading

Codecov Report

Katie-Howard left a comment

Choose a reason for hiding this comment

brhooper commented May 4, 2023

Katie-Howard left a comment

Choose a reason for hiding this comment

gavinevans left a comment

Choose a reason for hiding this comment

gavinevans left a comment

Choose a reason for hiding this comment

gavinevans left a comment

Choose a reason for hiding this comment

gavinevans left a comment

Choose a reason for hiding this comment

gavinevans commented May 16, 2023

brhooper commented Apr 28, 2023 •

edited

Loading

codecov bot commented Apr 28, 2023 •

edited

Loading