ci: move a chunk of the Rust CI over to GHA #9290
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I guess the check for allowing changes to pipeline.yml work a little too well :D |
nagisa
force-pushed
the
nagisa/cargo-tests-style
branch
4 times, most recently
from
409800f to
06a269c
Compare
nagisa
commented
Sep 11, 2023
nagisa
force-pushed
the
nagisa/cargo-tests-style
branch
9 times, most recently
from
315191a to
4b56709
Compare
nagisa
force-pushed
the
nagisa/cargo-tests-style
branch
3 times, most recently
from
e17417d to
b2f700e
Compare
github-merge-queue bot
pushed a commit
that referenced
this pull request
Oct 2, 2023
…t` (#9608) I got absolutely fed up with waiting for prerequisite infrastructure work for #9290 (why does it have to be _that_ hard?) However the PR in question had some other important improvements that do not necessarily *rely* on changes to said infrastructure to work. In particular: * `nextest` now retries failing tests a few times before giving up, to make sure they aren't spuriously failing; * This should help some timing out integration tests in particular, as those often fail because of some deadlockish situation in my experience. * style checks still run with `cargo nextest` – unfortunately that means that in CI this will run these checks multiple times, but that doesn’t sound particularly terrible of a tradeoff (especially if the other changes mean we won't be retrying entire test suites as often anymore;) * This should allow for a greater portion of the test suite to run on Macs – unfortunately not verified by the CI, but people do complain and this should make the situation better. cc #9367
nagisa
force-pushed
the
nagisa/cargo-tests-style
branch
2 times, most recently
from
48da3c7 to
43fe737
Compare
nagisa
commented
Oct 2, 2023
| @@ -1,8 +1,5 @@ | |||
| [profile.default] | |||
| slow-timeout = { period = "60s", terminate-after = 2, grace-period = "0s" } | |||
| # FIXME(nagisa): use --profile ci in CI instead when we manage to modify CI scripts... | |||
| retries = { backoff = "fixed", count = 3, delay = "1s" } | |||
| failure-output = "final" | |||
There was a problem hiding this comment.
This change may be contentious. On one hand keeping this in makes flaky tests less of a major PiTA during local development. On the other we might end up living with the flaky tests in perpetuity if we keep this as nobody would have any motivation to actually look into these flakies…
nikurt
pushed a commit
that referenced
this pull request
Oct 2, 2023
…t` (#9608) I got absolutely fed up with waiting for prerequisite infrastructure work for #9290 (why does it have to be _that_ hard?) However the PR in question had some other important improvements that do not necessarily *rely* on changes to said infrastructure to work. In particular: * `nextest` now retries failing tests a few times before giving up, to make sure they aren't spuriously failing; * This should help some timing out integration tests in particular, as those often fail because of some deadlockish situation in my experience. * style checks still run with `cargo nextest` – unfortunately that means that in CI this will run these checks multiple times, but that doesn’t sound particularly terrible of a tradeoff (especially if the other changes mean we won't be retrying entire test suites as often anymore;) * This should allow for a greater portion of the test suite to run on Macs – unfortunately not verified by the CI, but people do complain and this should make the situation better. cc #9367
nagisa
force-pushed
the
nagisa/cargo-tests-style
branch
3 times, most recently
from
c55bb2c to
8670d05
Compare
nagisa
force-pushed
the
nagisa/cargo-tests-style
branch
from
b595d7d to
ae5eb0c
Compare
This is much better than denying warnings during build as even with warnings present we can see the rest of the test suite failures at the same time.
nagisa
force-pushed
the
nagisa/cargo-tests-style
branch
from
1a17509 to
3a74f0c
Compare
nagisa
changed the title
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
nikurt
pushed a commit
that referenced
this pull request
Oct 10, 2023
GHA has many significant benefits compared to buildkite, most major of it is that blindly using GHA is not going to by default allow untrusted contributors to run arbitrary code – instead GitHub will present reviewers a button they can click after reviewing the PR. It also makes maintenance of the CI infrastructure somewhat easier, which given our soon-to-be-stretched-very-thin infrastructure team is a huge benefit. In process of implementing this PR I ended up simplifying a lot of the Rust testing as well. In particular instead of running half a dozen of different combinations on just Linux, we now run just the nightly vs non-nightly versions. This saves CPU time on many different rebuilds… Of note, one feature that no longer gets tested is `mock_node`, as it was causing a failure in one of the integration tests. If we wanted to re-enable this particular test, we should figure out how to fix the test, rather than adding a new test configuration. As a final benefit, I’ve also added a m1 macOS-based job. This should help with making sure that people who develop on company-issued laptops can actually be productive, rather than have to tip-toe around a boatload of failing tests. We will have an ability to decide whether we want to block PRs landing on this job in the repository configuration at any point in the future.
nikurt
pushed a commit
that referenced
this pull request
Oct 11, 2023
…t` (#9608) I got absolutely fed up with waiting for prerequisite infrastructure work for #9290 (why does it have to be _that_ hard?) However the PR in question had some other important improvements that do not necessarily *rely* on changes to said infrastructure to work. In particular: * `nextest` now retries failing tests a few times before giving up, to make sure they aren't spuriously failing; * This should help some timing out integration tests in particular, as those often fail because of some deadlockish situation in my experience. * style checks still run with `cargo nextest` – unfortunately that means that in CI this will run these checks multiple times, but that doesn’t sound particularly terrible of a tradeoff (especially if the other changes mean we won't be retrying entire test suites as often anymore;) * This should allow for a greater portion of the test suite to run on Macs – unfortunately not verified by the CI, but people do complain and this should make the situation better. cc #9367
nikurt
pushed a commit
that referenced
this pull request
Oct 11, 2023
GHA has many significant benefits compared to buildkite, most major of it is that blindly using GHA is not going to by default allow untrusted contributors to run arbitrary code – instead GitHub will present reviewers a button they can click after reviewing the PR. It also makes maintenance of the CI infrastructure somewhat easier, which given our soon-to-be-stretched-very-thin infrastructure team is a huge benefit. In process of implementing this PR I ended up simplifying a lot of the Rust testing as well. In particular instead of running half a dozen of different combinations on just Linux, we now run just the nightly vs non-nightly versions. This saves CPU time on many different rebuilds… Of note, one feature that no longer gets tested is `mock_node`, as it was causing a failure in one of the integration tests. If we wanted to re-enable this particular test, we should figure out how to fix the test, rather than adding a new test configuration. As a final benefit, I’ve also added a m1 macOS-based job. This should help with making sure that people who develop on company-issued laptops can actually be productive, rather than have to tip-toe around a boatload of failing tests. We will have an ability to decide whether we want to block PRs landing on this job in the repository configuration at any point in the future.
nikurt
pushed a commit
that referenced
this pull request
Oct 11, 2023
GHA has many significant benefits compared to buildkite, most major of it is that blindly using GHA is not going to by default allow untrusted contributors to run arbitrary code – instead GitHub will present reviewers a button they can click after reviewing the PR. It also makes maintenance of the CI infrastructure somewhat easier, which given our soon-to-be-stretched-very-thin infrastructure team is a huge benefit. In process of implementing this PR I ended up simplifying a lot of the Rust testing as well. In particular instead of running half a dozen of different combinations on just Linux, we now run just the nightly vs non-nightly versions. This saves CPU time on many different rebuilds… Of note, one feature that no longer gets tested is `mock_node`, as it was causing a failure in one of the integration tests. If we wanted to re-enable this particular test, we should figure out how to fix the test, rather than adding a new test configuration. As a final benefit, I’ve also added a m1 macOS-based job. This should help with making sure that people who develop on company-issued laptops can actually be productive, rather than have to tip-toe around a boatload of failing tests. We will have an ability to decide whether we want to block PRs landing on this job in the repository configuration at any point in the future.
frol
reviewed
Oct 12, 2023
Comment on lines
-67
to
-71
| cargo install cargo-deny | ||
| cargo run -p themis --release | ||
| if [ -e deny.toml ]; then | ||
| cargo-deny --all-features check bans | ||
| fi |
There was a problem hiding this comment.
@nagisa Was it intentional to decommission cargo-deny and themis sanity checks?
nikurt
pushed a commit
that referenced
this pull request
Oct 16, 2023
GHA has many significant benefits compared to buildkite, most major of it is that blindly using GHA is not going to by default allow untrusted contributors to run arbitrary code – instead GitHub will present reviewers a button they can click after reviewing the PR. It also makes maintenance of the CI infrastructure somewhat easier, which given our soon-to-be-stretched-very-thin infrastructure team is a huge benefit. In process of implementing this PR I ended up simplifying a lot of the Rust testing as well. In particular instead of running half a dozen of different combinations on just Linux, we now run just the nightly vs non-nightly versions. This saves CPU time on many different rebuilds… Of note, one feature that no longer gets tested is `mock_node`, as it was causing a failure in one of the integration tests. If we wanted to re-enable this particular test, we should figure out how to fix the test, rather than adding a new test configuration. As a final benefit, I’ve also added a m1 macOS-based job. This should help with making sure that people who develop on company-issued laptops can actually be productive, rather than have to tip-toe around a boatload of failing tests. We will have an ability to decide whether we want to block PRs landing on this job in the repository configuration at any point in the future.
nikurt
pushed a commit
that referenced
this pull request
Oct 18, 2023
GHA has many significant benefits compared to buildkite, most major of it is that blindly using GHA is not going to by default allow untrusted contributors to run arbitrary code – instead GitHub will present reviewers a button they can click after reviewing the PR. It also makes maintenance of the CI infrastructure somewhat easier, which given our soon-to-be-stretched-very-thin infrastructure team is a huge benefit. In process of implementing this PR I ended up simplifying a lot of the Rust testing as well. In particular instead of running half a dozen of different combinations on just Linux, we now run just the nightly vs non-nightly versions. This saves CPU time on many different rebuilds… Of note, one feature that no longer gets tested is `mock_node`, as it was causing a failure in one of the integration tests. If we wanted to re-enable this particular test, we should figure out how to fix the test, rather than adding a new test configuration. As a final benefit, I’ve also added a m1 macOS-based job. This should help with making sure that people who develop on company-issued laptops can actually be productive, rather than have to tip-toe around a boatload of failing tests. We will have an ability to decide whether we want to block PRs landing on this job in the repository configuration at any point in the future.
nikurt
pushed a commit
that referenced
this pull request
Oct 19, 2023
GHA has many significant benefits compared to buildkite, most major of it is that blindly using GHA is not going to by default allow untrusted contributors to run arbitrary code – instead GitHub will present reviewers a button they can click after reviewing the PR. It also makes maintenance of the CI infrastructure somewhat easier, which given our soon-to-be-stretched-very-thin infrastructure team is a huge benefit. In process of implementing this PR I ended up simplifying a lot of the Rust testing as well. In particular instead of running half a dozen of different combinations on just Linux, we now run just the nightly vs non-nightly versions. This saves CPU time on many different rebuilds… Of note, one feature that no longer gets tested is `mock_node`, as it was causing a failure in one of the integration tests. If we wanted to re-enable this particular test, we should figure out how to fix the test, rather than adding a new test configuration. As a final benefit, I’ve also added a m1 macOS-based job. This should help with making sure that people who develop on company-issued laptops can actually be productive, rather than have to tip-toe around a boatload of failing tests. We will have an ability to decide whether we want to block PRs landing on this job in the repository configuration at any point in the future.
nikurt
pushed a commit
that referenced
this pull request
Oct 19, 2023
GHA has many significant benefits compared to buildkite, most major of it is that blindly using GHA is not going to by default allow untrusted contributors to run arbitrary code – instead GitHub will present reviewers a button they can click after reviewing the PR. It also makes maintenance of the CI infrastructure somewhat easier, which given our soon-to-be-stretched-very-thin infrastructure team is a huge benefit. In process of implementing this PR I ended up simplifying a lot of the Rust testing as well. In particular instead of running half a dozen of different combinations on just Linux, we now run just the nightly vs non-nightly versions. This saves CPU time on many different rebuilds… Of note, one feature that no longer gets tested is `mock_node`, as it was causing a failure in one of the integration tests. If we wanted to re-enable this particular test, we should figure out how to fix the test, rather than adding a new test configuration. As a final benefit, I’ve also added a m1 macOS-based job. This should help with making sure that people who develop on company-issued laptops can actually be productive, rather than have to tip-toe around a boatload of failing tests. We will have an ability to decide whether we want to block PRs landing on this job in the repository configuration at any point in the future.
nikurt
pushed a commit
that referenced
this pull request
Oct 19, 2023
GHA has many significant benefits compared to buildkite, most major of it is that blindly using GHA is not going to by default allow untrusted contributors to run arbitrary code – instead GitHub will present reviewers a button they can click after reviewing the PR. It also makes maintenance of the CI infrastructure somewhat easier, which given our soon-to-be-stretched-very-thin infrastructure team is a huge benefit. In process of implementing this PR I ended up simplifying a lot of the Rust testing as well. In particular instead of running half a dozen of different combinations on just Linux, we now run just the nightly vs non-nightly versions. This saves CPU time on many different rebuilds… Of note, one feature that no longer gets tested is `mock_node`, as it was causing a failure in one of the integration tests. If we wanted to re-enable this particular test, we should figure out how to fix the test, rather than adding a new test configuration. As a final benefit, I’ve also added a m1 macOS-based job. This should help with making sure that people who develop on company-issued laptops can actually be productive, rather than have to tip-toe around a boatload of failing tests. We will have an ability to decide whether we want to block PRs landing on this job in the repository configuration at any point in the future.
nikurt
pushed a commit
that referenced
this pull request
Oct 19, 2023
GHA has many significant benefits compared to buildkite, most major of it is that blindly using GHA is not going to by default allow untrusted contributors to run arbitrary code – instead GitHub will present reviewers a button they can click after reviewing the PR. It also makes maintenance of the CI infrastructure somewhat easier, which given our soon-to-be-stretched-very-thin infrastructure team is a huge benefit. In process of implementing this PR I ended up simplifying a lot of the Rust testing as well. In particular instead of running half a dozen of different combinations on just Linux, we now run just the nightly vs non-nightly versions. This saves CPU time on many different rebuilds… Of note, one feature that no longer gets tested is `mock_node`, as it was causing a failure in one of the integration tests. If we wanted to re-enable this particular test, we should figure out how to fix the test, rather than adding a new test configuration. As a final benefit, I’ve also added a m1 macOS-based job. This should help with making sure that people who develop on company-issued laptops can actually be productive, rather than have to tip-toe around a boatload of failing tests. We will have an ability to decide whether we want to block PRs landing on this job in the repository configuration at any point in the future.
nikurt
pushed a commit
that referenced
this pull request
Oct 23, 2023
GHA has many significant benefits compared to buildkite, most major of it is that blindly using GHA is not going to by default allow untrusted contributors to run arbitrary code – instead GitHub will present reviewers a button they can click after reviewing the PR. It also makes maintenance of the CI infrastructure somewhat easier, which given our soon-to-be-stretched-very-thin infrastructure team is a huge benefit. In process of implementing this PR I ended up simplifying a lot of the Rust testing as well. In particular instead of running half a dozen of different combinations on just Linux, we now run just the nightly vs non-nightly versions. This saves CPU time on many different rebuilds… Of note, one feature that no longer gets tested is `mock_node`, as it was causing a failure in one of the integration tests. If we wanted to re-enable this particular test, we should figure out how to fix the test, rather than adding a new test configuration. As a final benefit, I’ve also added a m1 macOS-based job. This should help with making sure that people who develop on company-issued laptops can actually be productive, rather than have to tip-toe around a boatload of failing tests. We will have an ability to decide whether we want to block PRs landing on this job in the repository configuration at any point in the future.
nikurt
pushed a commit
that referenced
this pull request
Oct 23, 2023
GHA has many significant benefits compared to buildkite, most major of it is that blindly using GHA is not going to by default allow untrusted contributors to run arbitrary code – instead GitHub will present reviewers a button they can click after reviewing the PR. It also makes maintenance of the CI infrastructure somewhat easier, which given our soon-to-be-stretched-very-thin infrastructure team is a huge benefit. In process of implementing this PR I ended up simplifying a lot of the Rust testing as well. In particular instead of running half a dozen of different combinations on just Linux, we now run just the nightly vs non-nightly versions. This saves CPU time on many different rebuilds… Of note, one feature that no longer gets tested is `mock_node`, as it was causing a failure in one of the integration tests. If we wanted to re-enable this particular test, we should figure out how to fix the test, rather than adding a new test configuration. As a final benefit, I’ve also added a m1 macOS-based job. This should help with making sure that people who develop on company-issued laptops can actually be productive, rather than have to tip-toe around a boatload of failing tests. We will have an ability to decide whether we want to block PRs landing on this job in the repository configuration at any point in the future.
nikurt
pushed a commit
that referenced
this pull request
Oct 24, 2023
GHA has many significant benefits compared to buildkite, most major of it is that blindly using GHA is not going to by default allow untrusted contributors to run arbitrary code – instead GitHub will present reviewers a button they can click after reviewing the PR. It also makes maintenance of the CI infrastructure somewhat easier, which given our soon-to-be-stretched-very-thin infrastructure team is a huge benefit. In process of implementing this PR I ended up simplifying a lot of the Rust testing as well. In particular instead of running half a dozen of different combinations on just Linux, we now run just the nightly vs non-nightly versions. This saves CPU time on many different rebuilds… Of note, one feature that no longer gets tested is `mock_node`, as it was causing a failure in one of the integration tests. If we wanted to re-enable this particular test, we should figure out how to fix the test, rather than adding a new test configuration. As a final benefit, I’ve also added a m1 macOS-based job. This should help with making sure that people who develop on company-issued laptops can actually be productive, rather than have to tip-toe around a boatload of failing tests. We will have an ability to decide whether we want to block PRs landing on this job in the repository configuration at any point in the future.
nikurt
pushed a commit
that referenced
this pull request
Oct 24, 2023
GHA has many significant benefits compared to buildkite, most major of it is that blindly using GHA is not going to by default allow untrusted contributors to run arbitrary code – instead GitHub will present reviewers a button they can click after reviewing the PR. It also makes maintenance of the CI infrastructure somewhat easier, which given our soon-to-be-stretched-very-thin infrastructure team is a huge benefit. In process of implementing this PR I ended up simplifying a lot of the Rust testing as well. In particular instead of running half a dozen of different combinations on just Linux, we now run just the nightly vs non-nightly versions. This saves CPU time on many different rebuilds… Of note, one feature that no longer gets tested is `mock_node`, as it was causing a failure in one of the integration tests. If we wanted to re-enable this particular test, we should figure out how to fix the test, rather than adding a new test configuration. As a final benefit, I’ve also added a m1 macOS-based job. This should help with making sure that people who develop on company-issued laptops can actually be productive, rather than have to tip-toe around a boatload of failing tests. We will have an ability to decide whether we want to block PRs landing on this job in the repository configuration at any point in the future.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
GHA has many significant benefits compared to buildkite, most major of it is that blindly using GHA is not going to by default allow untrusted contributors to run arbitrary code – instead GitHub will present reviewers a button they can click after reviewing the PR.
It also makes maintenance of the CI infrastructure somewhat easier, which given our soon-to-be-stretched-very-thin infrastructure team is a huge benefit.
In process of implementing this PR I ended up simplifying a lot of the Rust testing as well. In particular instead of running half a dozen of different combinations on just Linux, we now run just the nightly vs non-nightly versions. This saves CPU time on many different rebuilds… Of note, one feature that no longer gets tested is
mock_node, as it was causing a failure in one of the integration tests. If we wanted to re-enable this particular test, we should figure out how to fix the test, rather than adding a new test configuration.As a final benefit, I’ve also added a m1 macOS-based job. This should help with making sure that people who develop on company-issued laptops can actually be productive, rather than have to tip-toe around a boatload of failing tests. We will have an ability to decide whether we want to block PRs landing on this job in the repository configuration at any point in the future.
See also #9608