retry api.github.com/meta requests #215
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
trxcllnt
added a commit
to trxcllnt/devcontainers
that referenced
this pull request
Feb 5, 2024
* retry api.github.com/meta requests * pipefail
trxcllnt
added a commit
that referenced
this pull request
Mar 7, 2024
* separate make-pip-dependencies and make-conda-dependencies scripts
* fix names
* parse duplicated arguments into a list
* support passing -k|--keys to rapids-make-{conda,pip}-dependencies
* fix cugraph-service-{client,server} paths
* escape rest parameters
* support python cmake, wheel, and install args in manifest.yaml
* generate dependencies per python lib + file_key
* add query-manifest command
* cleanup
* sort and dedupe pip dependencies
* constrain cuda-python and cupy versions to the current CUDA version
* clean compile_commands.json
* intercept the --mode param and only pass it to the python build script
* support defining custom envvars in manifest.yaml python entries
* keep cupy package version, just change the cuda modifier
* clean more setuptools/distutils build dirs
* ignore grep's nonzero exit code
Signed-off-by: ptaylor <paul.e.taylor@me.com>
* add cpp install and cpack scripts
Signed-off-by: ptaylor <paul.e.taylor@me.com>
* add a timing label to build and configure steps
Signed-off-by: ptaylor <paul.e.taylor@me.com>
* install system dependencies for ucxx, cudf_kafka, and cuspatial in pip containers (#209)
Signed-off-by: ptaylor <paul.e.taylor@me.com>
* run rapids-dependency-file-generator in parallel
Signed-off-by: ptaylor <paul.e.taylor@me.com>
* use absolute paths
Signed-off-by: ptaylor <paul.e.taylor@me.com>
* fix find command
* allow passing extra requirements
* add devcontainer-utils-parse-args-from-docstring script
* add bash completion template
* generate completions for rapids-build-utils scripts
* add docstrings for clone-{git,github,gitlab}-repo scripts
* format shebangs
* add docstrings
* narrow debug output by script name
* don't downcase C++ lib names, generate completions
* add docstrings
* add --omit to rapids-list-repos
* install the utils repo from disk, ignore cron.d startup error
* parse args after env name
* fix typo
* fix paths
* make individual <script>-all templates so they can have docstrings
* only auth with gh/glab if neither --no-fork nor --clone-upstream are passed
* default to -j1 if no value is passed, but assume full parallelism in configure/build script templates
* fetch the branch before checking it out
Signed-off-by: ptaylor <paul.e.taylor@me.com>
* fix typo
Signed-off-by: ptaylor <paul.e.taylor@me.com>
* simplify checkout-same-branch a bit
* add -j option to clone scripts to adjust submodule parallelism, make clone scripts skip github/gitlab API calls unless logged in
* add --parallel flag to checkout-same-branch
* add uninstall scripts
* fix typos
* pass -v to rm
* shift off first arg
* add devcontainer-utils-generate-bash-completion script
* rename script templates, add uninstall scripts templates, use devcontainer-utils-generate-bash-completion script
* ensure newlines in known_hosts (#214)
* bump feature version
* error if curl fails
* bump feature version
* retry api.github.com/meta requests (#215)
* retry api.github.com/meta requests
* pipefail
* inline github fingerprints
* fix shellcheck lint
* fix shellcheck lint, rewrite parse-args to use getopts, add cpack-all and cpack-<repo> scripts
* don't quote flags
* set -E, use local - in functions, cleanup
* DOC v24.04 Updates [skip ci]
* Remove -DNVBench_ENABLE_CUPTI=OFF. (#213)
* fix typo
* fix find args
* fixes for bash v4.4.20 (Ubuntu 18.04)
* don't do local -r in loops
* fix pull-repositories
* don't do -r in loops
* remove set -x
* temporarily allow unbound variables while activating the conda env (#216)
* bump mambaforge feature version
* fix shellcheck lint
* use conda env create in both places
* don't set -E
* more verbose output
* add rapids-post-attach-entrypoint
* make the latest symlink local instead of absolute
* allow cpack'ing components by doing a dummy install and tar'ing everything up ourselves
* fix issue with scripts with no value params, fix handling of unknown short opts like -Wno-dev
* cleanup
* don't add empty strings to arg list
* handle args better
* add missing --strip arg
* clean dot files too
* cpack in parallel with pigz
* support --take and --skip to only parse selected args
* assume each repo has 3 submodules to clone in parallel
* clean up
* bool arguments are the complement of all and value args
* install into CONDA_PREFIX if defined
* remove pip args from docstring
* ensure cpp deps propagate to python scripts
* skip cpack'ing if there's no CMakeCache.txt
* expand with quotes
* use bash-completion-lib to complete directories and filenames
* simplify devcontainer-utils clone scripts, configure remote.upstream.fetch during git clone
* handle initial -- in rest args
* don't pass -c remote.upstream.fetch='^refs/heads/pull-request/*' because of git clone bug
* Fix mambaforge shell history (#219)
* disable history when restoring shell options
* Update devcontainer-feature.json
* Add cugraph-equivariant (#218)
* Fix buildx setup on github runners (#220)
* Fix buildx setup on github runners
* change conditional to `env.RUNNER_ENV == 'self-hosted'
* bump versions
* ensure user bin dirs don't end up on the path multiple times
* login to dockerhub so we don't hit rate limits
* pass in runner_env as input
* check runner.name again
* fix quoting
* use ssh URLs in fallback
* use runner.environment
* add devcontainer-utils-post-create-command that uses socat to proxy to SSH_AUTH_SOCK
* fix counting in add_etc_profile_d_script
* make build dirs per python package manager/cuda version, and build type
* fix yq install when tags are ahead of releases
* remove duplicate --log-level=VERBOSE arg
* fix devcontainer-utils-vault-s3-init for sccache v0.7.7
* prefix variables
* fix parse-cmake-args, add parse-pip-{install,wheel}-args
* fix vault-s3-init for sccache v0.7.7
* cleanup
* fix bad merge
* fix typo
* ignore stderr
* always build with --pull, add files.watcherExclude and search.exclude, install more system-level dependencies in pip devcontainers
* Regnerate creds every 8 hours and echo the timestamp to the cron log
* consolidate xtrace logic into debug-output.sh
* consolidate xtrace logic into debug-output.sh
* add underscores to function names in parse-args-from-docstring
* update docstrings to inline from other commands
* add rapids-generate-docstrings to inline docstrings from other commands
* clean up feature install scripts
* accept multiple commands in devcontainer-utils-generate-bash-completion
* remove --pull option from devcontainers
* remove unused variable
* guard against devcontainer-utils feature not being installed when running rapids-build-utils test.sh
* make hierarchy of latest symlinks
* make hierarchy of latest symlinks point at each other, update latest symlinks in post-attach
* fix completion generation in install.sh
* fix checkout-same-branch when branch names have slashes
* ensure directory permissions are set all the way down
* support --remote and --default-branch flags in rapids-pull-repositories
* use free mem not available mem
* handle case where github and gitlab API calls fail to return the remote URLs
* pass additional arguments to devcontainer build
* install devcontainer-utils features
* add tasks-shell-input extension, files and search excludes for CUDA 12.2 devcontainers
* pass --quiet to silence git clone progress, --single-branch and --shallow-submodules
* don't make an env from an empty file
* only pass in the name of a cloned repo to generate_all_script
* fix unbound variable error
* clean up manifest.yaml and python build script templates
* set no_build_isolation=true if defined in manifest.yaml
* test pip_args instead of pip_args_
* set SETUPTOOLS_ENABLE_FEATURES=legacy-editable for dask_cudf and custreamz
* do SETUPTOOLS_ENABLE_FEATURES=legacy-editable for more cugraph python projects
* revert to always set legacy-editable if not using scikit-build-core
* allow making empty envs
* pass -j in clone-all
* revert
* fix free mem calc
* use gibibytes (powers of 1024) instead of gigabytes
* recreate the env from scratch after cloning
* source the rapids-make-{conda,pip}-env call
* revert recent changes
* add --max-archs envvar default, remove short arg for --archs and --max-device-obj-memory-usage, count archs better
* only proxy ssh_auth_sock if it's not already proxied
* guard against -u
* remove reference to old -m flag
* Make generated scripts inherit `-D<repo>_ROOT=` CMake args from their dependencies, clean up manifest.yaml
* cleanup rapids-get-build-dir docstring
* Parse and quote value options that have spaces
* pass --archs 1 in repo clone template
* don't error if dirs don't exist
* pass --archs 1 in cpp.cpack template
* handle pairs of --component and --out-dir
* add rapids-maybe-clean-build-dir, don't unconditionally reconfigure in cpp.build template, support generators other than Ninja
* Add python.install template, update python.build.editable template to call it with --editable flag
* only release features after images are pushed
* support ssh_url and https_url in manifest.yaml
* only modify cupy-cuda and cuda-python if they're at the beginning of a line
* Pass -DUSE_LIBARROW_FROM_PYARROW=ON to cuDF if PYTHON_PACKAGE_MANAGER is pip
* pass CMAKE_CUDA_ARCHTECTURES to the projects that ignore the CUDAARCHS envvar
* define CPM_SOURCE_CACHE
* set libcudf_kernel_cache_path to more specific build dir
* add --max-total-system-memory option to limit total amount of system memory to consume for C++ and device compilations
* make ~/.local/share dir owned by the non-root user
* support --archs 0 to omit device object memory in n_jobs calculation
* filter comments from pip requirements
* randomize sccache_server_port on container creation
* tee all vault-s3-creds-{generate,schedule} logs to the logfile, regenerate creds 5 minutes before they expire
* thread tty through xargs
* thread tty through xargs better
* use tty exit code in shell-is-interactive check
* use read -rsp everywhere
* remove update-content-command, move rapids-generate-scripts into post-start-command
* merge compile_commands.json into one file in the home dir for clangd users not using the workspace
* move everything in rapids-post-attach-command to rapids-post-start-command
* ensure latest symlink exists if not created
* parse bools as arrays whose contents are the full optarg
* add -f flag to clone-git-repo, always fetch all upstreams, add /tag/ in the name of the local branch created from a tag
* don't use API calls to get the upstream and origin names if --no-fork or --clone-upstream are passed
* always call clone-git{hub,lab}-repo in the repo clone template
* cleanup docstrings
* fix reconfigure check
* actually fix reconfigure check
* always do _parse_args at the top
* don't overwrite the existing install_manifest.txts when doing the install for cpack
* support uninstalling from the cpack'd tgz's if the build dir doesn't have install_manifest.txt
* determine parallelism so args from manifest.yaml can use it, but ignore `-h` text on stderr
* remove -j from options list
* bump feature versions
* remove rapids-post-attach-command
* guard against verbose[0] being null now that everything's an array again
* clean up boolean arg handling
* add stub rapids-post-attach-command for backwards-compatibility
* remove CPM_SOURCE_CACHE
* guard against grep failures
* log to /var/log/devcontainer-utils/ so we can mount over that dir for testing
---------
Signed-off-by: ptaylor <paul.e.taylor@me.com>
Co-authored-by: Ray Douglass <ray@raydouglass.com>
Co-authored-by: Bradley Dice <bdice@bradleydice.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.