Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(build-and-test-differential): use github runner #6278

Conversation

kminoda
Copy link
Contributor

@kminoda kminoda commented Feb 1, 2024

Description

Use GitHub runner.

The runner has been changed as follows:

Apparently the disk image size has been reduced, and thus we would like to use GitHub Runner again, since current self-hosted runner based CI is too slow to execute multiple PRs.

Test run: https://github.com/HansRobo/autoware.universe/actions/runs/7736637727/job/21094318337
Remaining disk space:

$ dh -a

Filesystem     1K-blocks     Used Available Use% Mounted on
overlay         87204404 75316916  11871104  87% /

du results in the latest full-built Autoware:

$ cd /path/to/pre-built/autoware
$ du -d0
11311632        .  

Tests performed

As shown above

Effects on system behavior

There is a risk in a near future that the CI will fail again due to the disk usage limit as it was before.

Autoware developers need to understand the importance of reducing the file size of both scripts and the generated binaries.

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

  • There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

Signed-off-by: kminoda <koji.minoda@tier4.jp>
@github-actions github-actions bot added the type:ci Continuous Integration (CI) processes and testing. (auto-assigned) label Feb 1, 2024
@kminoda kminoda marked this pull request as ready for review February 1, 2024 06:35
Signed-off-by: kminoda <koji.minoda@tier4.jp>
Copy link
Contributor

@shmpwk shmpwk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shmpwk shmpwk added the run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) label Feb 1, 2024
@miursh miursh merged commit 0262c68 into autowarefoundation:main Feb 1, 2024
25 of 27 checks passed
@kminoda kminoda deleted the ci/build-and-test-dif/use_github_runner branch February 1, 2024 06:58
mitsudome-r added a commit to mitsudome-r/autoware.universe that referenced this pull request Feb 5, 2024
@mitsudome-r mitsudome-r mentioned this pull request Feb 5, 2024
7 tasks
mitsudome-r added a commit that referenced this pull request Feb 5, 2024
…6278)" (#6306)

* Revert "ci(build-and-test-differential): remove prevent-no-label-execution (#6292)"

This reverts commit 76cc954.

* Revert "ci(build-and-test-differential): use github runner (#6278)"

This reverts commit 0262c68.
zulfaqar-azmi-t4 pushed a commit to zulfaqar-azmi-t4/autoware.universe that referenced this pull request Feb 6, 2024
…utowarefoundation#6278)" (autowarefoundation#6306)

* Revert "ci(build-and-test-differential): remove prevent-no-label-execution (autowarefoundation#6292)"

This reverts commit 76cc954.

* Revert "ci(build-and-test-differential): use github runner (autowarefoundation#6278)"

This reverts commit 0262c68.
anhnv3991 pushed a commit to anhnv3991/autoware.universe that referenced this pull request Feb 13, 2024
…utowarefoundation#6278)" (autowarefoundation#6306)

* Revert "ci(build-and-test-differential): remove prevent-no-label-execution (autowarefoundation#6292)"

This reverts commit 76cc954.

* Revert "ci(build-and-test-differential): use github runner (autowarefoundation#6278)"

This reverts commit 0262c68.
@xmfcx
Copy link
Contributor

xmfcx commented Feb 26, 2024

$ dh -a

Filesystem     1K-blocks     Used Available Use% Mounted on
overlay         87204404 75316916  11871104  87% /```

@kminoda it's better to run df -h since it gives disk info in a more readable way.

@mitsudome-r this was never supposed to be merged because even in the PR description, the available space is 11871104 KB which is 11.87 GB.

The available space for public runners was always this size.

This PR was merged, worked a couple of times randomly. Because by nature, the build and test differential workflow can build varying count of packages, changing the size footprint.

And the container size was another blocker, sometimes preventing the job from starting in the first place.

As for the updated workflow:

from the latest job:

Run df -h
Filesystem      Size  Used Avail Use% Mounted on
overlay          73G   61G   12G  85% /

The available space for the containers is still the same.

@xmfcx
Copy link
Contributor

xmfcx commented Feb 26, 2024

I've also browsed some passing jobs that has passed from this PR's merge.

Without CUDA

https://github.com/autowarefoundation/autoware.universe/actions/runs/7757485516/job/21157041781#step:4:6

Filesystem      Size  Used Avail Use% Mounted on
overlay          84G   64G   20G  77% /

With CUDA

https://github.com/autowarefoundation/autoware.universe/actions/runs/7757485516/job/21157042168#step:4:6

Filesystem      Size  Used Avail Use% Mounted on
overlay          84G   74G   11G  88% /

It seems you got some 84G size machines where it worked. But when it failed, it was with 73G machines and CUDA version of the docker image didn't fit in there.

karishma1911 pushed a commit to Interplai/autoware.universe that referenced this pull request Jun 3, 2024
…n#6278)

* ci(build-and-test-differential): use github runner

Signed-off-by: kminoda <koji.minoda@tier4.jp>

* change clang-tidy runner

Signed-off-by: kminoda <koji.minoda@tier4.jp>

---------

Signed-off-by: kminoda <koji.minoda@tier4.jp>
karishma1911 pushed a commit to Interplai/autoware.universe that referenced this pull request Jun 3, 2024
…utowarefoundation#6278)" (autowarefoundation#6306)

* Revert "ci(build-and-test-differential): remove prevent-no-label-execution (autowarefoundation#6292)"

This reverts commit 76cc954.

* Revert "ci(build-and-test-differential): use github runner (autowarefoundation#6278)"

This reverts commit 0262c68.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) type:ci Continuous Integration (CI) processes and testing. (auto-assigned)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants