Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: make docker cache download and docker load time out after 10 minutes #90122

Merged
merged 3 commits into from
Oct 23, 2021
Merged

CI: make docker cache download and docker load time out after 10 minutes #90122

merged 3 commits into from
Oct 23, 2021

Conversation

hkratz
Copy link
Contributor

@hkratz hkratz commented Oct 21, 2021 •
edited
Loading

@rust-highfive
Copy link
Collaborator

r? @Mark-Simulacrum

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 21, 2021
@hkratz
Copy link
Contributor Author

hkratz commented Oct 21, 2021

On second thought it seems much more likely that docker load hangs.

@rust-log-analyzer

This comment has been minimized.

Sign in to view
@Mark-Simulacrum
Copy link
Member

@bors r+

@bors
Copy link
Contributor

bors commented Oct 21, 2021

📌 Commit 838e673 has been approved by Mark-Simulacrum

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 21, 2021
@hkratz
Copy link
Contributor Author

hkratz commented Oct 21, 2021 •
edited
Loading

@Mark-Simulacrum I now strongly suspect that it is docker load. Not sure if we can do anything about it, might be a problem specific to docker in the hosted GA runners.

We could try /usr/bin/timeout -s KILL 600 docker load -i /tmp/rustci_docker_cache (kill after 10 min.), then it would at least fail earlier if stuck just not use the cached docker images.

JohnTitor added a commit to JohnTitor/rust that referenced this pull request Oct 21, 2021
@JohnTitor
Rollup merge of rust-lang#90122 - rusticstuff:ci_curl_max_time, r=Mar…
6717b36 
…k-Simulacrum

CI: make docker cache download attempt time out after 10 minutes

Might help to prevent timeouts we have been seeing:
* https://github.com/rust-lang-ci/rust/runs/3946294286?check_suite_focus=true#step:25:23
* https://github.com/rust-lang-ci/rust/runs/3956799200?check_suite_focus=true#step:25:22
@JohnTitor JohnTitor mentioned this pull request Oct 21, 2021
Closed
@hkratz
Copy link
Contributor Author

hkratz commented Oct 21, 2021

Happened once more: https://github.com/rust-lang-ci/rust/runs/3962928502?check_suite_focus=true#step:25:23

JohnTitor added a commit to JohnTitor/rust that referenced this pull request Oct 21, 2021
@JohnTitor
Rollup merge of rust-lang#90122 - rusticstuff:ci_curl_max_time, r=Mar…
7a2b460 
…k-Simulacrum

CI: make docker cache download attempt time out after 10 minutes

Might help to prevent timeouts we have been seeing:
* https://github.com/rust-lang-ci/rust/runs/3946294286?check_suite_focus=true#step:25:23
* https://github.com/rust-lang-ci/rust/runs/3956799200?check_suite_focus=true#step:25:22
* https://github.com/rust-lang-ci/rust/runs/3962928502?check_suite_focus=true#step:25:23
@JohnTitor JohnTitor mentioned this pull request Oct 21, 2021
Closed
@hkratz hkratz changed the title CI: make docker cache download attempt time out after 10 minutes CI: make docker cache download and docker load time out after 10 minutes Oct 21, 2021
@hkratz
Copy link
Contributor Author

hkratz commented Oct 21, 2021

Adding timeout for docker load as the rollup will time out again anyway so the dl was not to blame: https://github.com/rust-lang-ci/rust/runs/3967892291?check_suite_focus=true

@hkratz
Copy link
Contributor Author

hkratz commented Oct 21, 2021

@rustbot label -S-waiting-on-bors +S-waiting-on-review

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Oct 21, 2021
@Mark-Simulacrum
Copy link
Member

@bors r+ rollup

@bors
Copy link
Contributor

bors commented Oct 23, 2021

📌 Commit 3e6a695 has been approved by Mark-Simulacrum

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 23, 2021
@matthiaskrgr matthiaskrgr mentioned this pull request Oct 23, 2021
Merged
bors added a commit to rust-lang-ci/rust that referenced this pull request Oct 23, 2021
@bors
Auto merge of rust-lang#90188 - matthiaskrgr:rollup-74cwv5c, r=matthi…
cf70855 
…askrgr

Rollup of 11 pull requests

Successful merges:

 - rust-lang#83233 (Implement split_array and split_array_mut)
 - rust-lang#88300 (Stabilise unix_process_wait_more, extra ExitStatusExt methods)
 - rust-lang#89416 (nice_region_error: Include lifetime placeholders in error output)
 - rust-lang#89468 (Report fatal lexer errors in `--cfg` command line arguments)
 - rust-lang#89730 (add feature flag for `type_changing_struct_update`)
 - rust-lang#89920 (Implement -Z location-detail flag)
 - rust-lang#90070 (Add edition configuration to compiletest)
 - rust-lang#90087 (Sync rustfmt subtree)
 - rust-lang#90117 (Make RSplit<T, P>: Clone not require T: Clone)
 - rust-lang#90122 (CI: make docker cache download and `docker load` time out after 10 minutes)
 - rust-lang#90166 (Add comment documenting why we can't use a simpler solution)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 1ea34fb into rust-lang:master Oct 23, 2021
@rustbot rustbot added this to the 1.58.0 milestone Oct 23, 2021
@hkratz hkratz deleted the ci_curl_max_time branch November 6, 2021 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@Mark-Simulacrum Mark-Simulacrum

Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Milestone
1.58.0
Development

Successfully merging this pull request may close these issues.
6 participants
@hkratz @rust-highfive @rust-log-analyzer @Mark-Simulacrum @bors @rustbot