-
Notifications
You must be signed in to change notification settings - Fork 4.5k
ci: Add cleanup script to manage build agent disk space #381
Conversation
Codecov Report
@@ Coverage Diff @@
## master #381 +/- ##
==========================================
- Coverage 91.06% 90.78% -0.29%
==========================================
Files 36 36
Lines 3493 3493
==========================================
- Hits 3181 3171 -10
- Misses 312 322 +10
Continue to review full report at Codecov.
|
# Regular maintenance performed on a buildkite agent to control disk usage | ||
# | ||
|
||
echo --- Delete all exited containers first |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
prefer printf BUILTIN to echo
( | ||
set -x | ||
exited=$(docker ps -aq --no-trunc --filter "status=exited") | ||
if [[ -n "$exited" ]]; then |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
double quotes unnecessary in bash's builtin [[
set -x | ||
dangling=$(docker images --filter 'dangling=true' -q --no-trunc | sort | uniq) | ||
if [[ -n "$dangling" ]]; then | ||
docker rmi "$dangling" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks to me like dangling is multiple strings... when you pass "$dangling" to docker rmi, you pass a single argument
consider
dangling=( $(docker ...) )
docker rmi "${dangliing[@]}"
echo "--- Delete /tmp files older than 1 day owned by $(whoami)" | ||
( | ||
set -x | ||
find /tmp -maxdepth 1 -user "$(whoami)" -mtime +1 -print0 | xargs -0 rm -rf |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
whoami isn't always around, as such introduces a dependency, consider ${USER} or
…torage (anza-xyz#381) #### Problem The current implementation of TieredStorage::file_size() requires a sys-call to provide the file size. #### Summary of Changes Add len() API to TieredStorageReader, and have HotStorageReader() implement the API using Mmap::len(). #### Test Plan Update existing unit-test to also verify HotStorageReader::len().
Docker containers/images in particular need to be actively pruned to avoid running out of disk