-
Notifications
You must be signed in to change notification settings - Fork 15
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Make
rapids-get-artifact
work locally (#59)
This PR ensures that `rapids-get-artifact` works locally. This request was made by @bdice who was attempting to use the script locally to debug some CUDA 12 issues. I've moved the logic for downloading, extracting, and printing the artifact's path to a new script, `_rapids-download-from-s3`. This script is then called by both `rapids-download-from-s3` and `rapids-get-artifact`. In the future, I would like to rework some of the scripts regarding artifact uploading/downloading, but I think we need to finish reworking the wheel workflows before that will be feasible.
- Loading branch information
1 parent
3303915
commit 17a072b
Showing
3 changed files
with
30 additions
and
21 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
#!/bin/bash | ||
# A utility script that downloads an artifact archive from S3, untars it, | ||
# and prints the location where it was untarred. | ||
# The script can be used locally and in CI. | ||
# Positional Arguments: | ||
# 1) path to s3 artifact | ||
# 2) location to untar it to | ||
set -euo pipefail | ||
source rapids-constants | ||
|
||
s3_dl_path="$1" | ||
untar_dest="$2" | ||
|
||
mkdir -p "${untar_dest}" | ||
|
||
if [ "${CI:-false}" = "false" ]; then | ||
# shellcheck disable=SC2001 | ||
s3_dl_path=$(echo "${s3_dl_path}" | sed "s|s3://${RAPIDS_DOWNLOADS_BUCKET}|https://${RAPIDS_DOWNLOADS_DOMAIN}|") | ||
rapids-echo-stderr "Downloading and decompressing ${s3_dl_path} into ${untar_dest}" | ||
wget -qO- "${s3_dl_path}" | tar xzf - -C "${untar_dest}" | ||
else | ||
rapids-echo-stderr "Downloading and decompressing ${s3_dl_path} into ${untar_dest}" | ||
aws s3 cp --only-show-errors "${s3_dl_path}" - | tar xzf - -C "${untar_dest}" | ||
fi | ||
|
||
# echo path to untarred contents | ||
echo -n "${untar_dest}" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters