CI: add scheduled testing against nightlies #2376
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This comment was marked as resolved.
This comment was marked as resolved.
rcomer
force-pushed
the
ci-nightly-tests
branch
2 times, most recently
from
663f7e4 to
aa272cf
Compare
|
I now have a run that demonstrates installing nightlies of all five expected packages: https://github.com/rcomer/cartopy/actions/runs/8858366152 I have also learned that if you still get failures after re-spin, you get duplicate issues: rcomer#12, rcomer#13, rcomer#14. I think the flakiness of our tests is another good reason to keep the schedule fairly low frequency. |
greglucas
approved these changes
Apr 28, 2024
.github/workflows/ci-testing.yml
Outdated
|
|
||
| # Separate dependent job to only upload one issue from the matrix of jobs | ||
| create-issue: | ||
| if: ${{ failure() && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') }} |
There was a problem hiding this comment.
Do we want workflow_dispatch to upload an issue as well, or just the scheduled run?
There was a problem hiding this comment.
Fair point. I guess if you've done workflow_dispatch then you are actively looking at it and don't need the issue.
rcomer
force-pushed
the
ci-nightly-tests
branch
from
aa272cf to
b14a31e
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rationale
This PR removes
shapely-devfrom the CI test matrix and instead introduces monthly testing against the Shapely, SciPy and Matplotlib nightly builds (note Numpy and ContourPy nightlies are automatically pulled in with Matplotlib).The guidance from Scientific Python recommends a weekly job for this. At #2371 (comment), @greglucas suggested a monthly job. I also lean towards monthly just because Cartopy doesn't have a lot of resource so with a more frequent run we probably would end up closing lots of duplicate issues that we didn't get around to looking at yet. The choice of 10th of the month is pretty arbitrary but it seems good to move it away from the beginning of the month so we aren't bothered by it on New Year's Day! I also added a
workflow-dispatchtrigger so we can run it if and when we have any reason to worry about recent updates in any package.For the approach I roughly followed Matplotlib's workflow. It feels a little excessive to run the whole matrix but I couldn't find a neat way to restrict it for just these workflow triggers. I took the command to install the nightlies from the Scientific Python page. It's a little different from what Matplotlib does, and what's recommended in Matplotlib's installation documentation but I don't know the reason for the difference or how to choose between them.
Implications