-
Notifications
You must be signed in to change notification settings - Fork 153
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* In progress. * Add manual docker instructions. * document make_devcontainers.sh * Finalized devcontainer README. * Split of contributing guide to another PR. * Modularize launch script. * Add docs for launch script. * Explain weird vscode launch logic. * Try HTML for clickable URI. * Try using a reference link. * GitHub markdown won't render vscode URI. * Remove old notes. * Remove stale. * Intro. * Fix bullets. * [skip ci] Fix launch script path. * [skip ci] Add reference to rapidsai/devcontainers. * [skip ci] Stale comment. * Update CI overview. * Apply suggestions from code review Co-authored-by: Bradley Dice <bdice@bradleydice.com> Co-authored-by: Vyas Ramasubramani <vyas.ramasubramani@gmail.com> Co-authored-by: Allard Hendriksen <allard@allardhendriksen.nl> * Heavily simplified devcontainer guide. * Add section about codespaces. * [skip ci] Minor wording changes * [skip ci] Update launch script. * [skip ci] Update docs for Docker launch. * [skip ci] Add codespace badge to top level readme. * [skip ci] Update link. * [skip ci] Add links. * [skip ci] Add section on devcontainer customization. * [skip ci] Remove extra space. * Apply suggestions from code review Co-authored-by: Christopher Harris <xixonia@gmail.com> --------- Co-authored-by: Bradley Dice <bdice@bradleydice.com> Co-authored-by: Vyas Ramasubramani <vyas.ramasubramani@gmail.com> Co-authored-by: Allard Hendriksen <allard@allardhendriksen.nl> Co-authored-by: Christopher Harris <xixonia@gmail.com>
- Loading branch information
1 parent
865bcf8
commit af59bb6
Showing
9 changed files
with
267 additions
and
59 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,116 @@ | ||
> **Note** | ||
> The instructions in this README are specific to Linux development environments. Instructions for Windows are coming soon! | ||
[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/NVIDIA/cccl?quickstart=1&devcontainer_path=.devcontainer%2Fdevcontainer.json) | ||
|
||
# CCCL Dev Containers | ||
|
||
CCCL uses [Development Containers](https://containers.dev/) to provide consistent and convenient development environments for both local development and for CI. This guide covers setup in [Visual Studio Code](#quickstart-vscode-recommended) and [Docker](#quickstart-docker-manual-approach). | ||
|
||
## Quickstart: VSCode (Recommended) | ||
|
||
### Prerequisites | ||
- [Visual Studio Code](https://code.visualstudio.com/) | ||
- [Remote - Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) | ||
|
||
### Steps | ||
|
||
1. Clone the Repository | ||
```bash | ||
git clone https://github.com/nvidia/cccl.git | ||
``` | ||
2. Open the cloned directory in VSCode | ||
|
||
3. Launch a Dev Container by clicking the prompt suggesting to "Reopen in Container" | ||
|
||
![Shows "Reopen in Container" prompt when opening the cccl directory in VScode.](./img/reopen_in_container.png) | ||
|
||
- Alternatively, use the Command Palette to start a Dev Container. Press `Ctrl+Shift+P` to open the Command Palette. Type "Remote-Containers: Reopen in Container" and select it. | ||
|
||
![Shows "Reopen in Container" in command pallete.](./img/open_in_container_manual.png) | ||
|
||
4. Select an environment with the desired CTK and host compiler from the list: | ||
|
||
![Shows list of available container environments.](./img/container_list.png) | ||
|
||
5. VSCode will initialize the selected Dev Container. This can take a few minutes the first time. | ||
|
||
6. Once initialized, the local `cccl/` directory is mirrored into the container to ensure any changes are persistent. | ||
|
||
7. Done! See the [contributing guide](../CONTRIBUTING.md#building-and-testing) for instructions on how to build and run tests. | ||
|
||
### (Optional) Authenticate with GitHub for `sccache` | ||
|
||
After starting the container, there will be a prompt to authenticate with GitHub. This grants access to a [`sccache`](https://github.com/mozilla/sccache) server shared with CI and greatly accelerates local build times. This is currently limited to NVIDIA employees belonging to the `NVIDIA` or `rapidsai` GitHub organizations. | ||
|
||
Without authentication to the remote server, `sccache` will still accelerate local builds by using a filesystem cache. | ||
|
||
Follow the instructions in the prompt as below and enter the one-time code at https://github.com/login/device | ||
|
||
![Shows authentication with GitHub to access sccache bucket.](./img/github_auth.png) | ||
|
||
To manually trigger this authentication, execute the `devcontainer-utils-vault-s3-init` script within the container. | ||
|
||
For more information about the sccache configuration and authentication, see the documentation at [`rapidsai/devcontainers`](https://github.com/rapidsai/devcontainers/blob/branch-23.10/USAGE.md#build-caching-with-sccache). | ||
|
||
## Quickstart: Docker (Manual Approach) | ||
|
||
### Prerequisites | ||
- [Docker](https://docs.docker.com/desktop/install/linux-install/) | ||
|
||
### Steps | ||
1. Clone the repository and use the [`launch.sh`](./launch.sh) script to launch the default container environment | ||
```bash | ||
git clone https://github.com/nvidia/cccl.git | ||
cd cccl | ||
./.devcontainer/launch.sh --docker | ||
``` | ||
This script starts an interactive shell as the `coder` user inside the container with the local `cccl/` directory mirrored into `/home/coder/cccl`. | ||
|
||
For specific environments, use the `--cuda` and `--host` options: | ||
```bassh | ||
./.devcontainer/launch.sh --docker --cuda 12.2 --host gcc10 | ||
``` | ||
See `./.devcontainer/launch.sh --help` for more information. | ||
|
||
2. Done. See the [contributing guide](../CONTRIBUTING.md#building-and-testing) for instructions on how to build and run tests. | ||
|
||
## Available Environments | ||
|
||
CCCL provides environments for both the oldest and newest supported CUDA versions with all compatible host compilers. | ||
|
||
Look in the [`.devcontainer/`](.) directory to see the available configurations. The top-level [`devcontainer.json`](./devcontainer.json) serves as the default environment. All `devcontainer.json` files in the `cuda<CTK_VERSION>-<HOST-COMPILER>` sub-directories are variations on this top-level file, with different base images for the different CUDA and host compiler versions. | ||
|
||
## VSCode Customization | ||
|
||
By default, CCCL's Dev Containers come with certain VSCode settings and extensions configured by default, as can be seen in the [`devcontainer.json`](./devcontainer.json) file. This can be further customized by users without needing to modify the `devcontainer.json` file directly. | ||
For extensions, the [`dev.containers.defaultExtensions` setting](https://code.visualstudio.com/docs/devcontainers/containers#_always-installed-extensions) allows listing extensions that will always be installed. | ||
For more general customizations, VSCode allows using a dotfile repository. See the [VSCode documentation](https://code.visualstudio.com/docs/devcontainers/containers#_personalizing-with-dotfile-repositories) for more information. | ||
## GitHub Codespaces | ||
[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/NVIDIA/cccl?quickstart=1&devcontainer_path=.devcontainer%2Fdevcontainer.json) | ||
One of the benefits of Dev Containers is that they integrate natively with [GitHub Codespaces](https://github.com/features/codespaces). Codespaces provide a VSCode development environment right in your browser running on a machine in the cloud. This provides a truly one-click, turnkey development environment where you can develop, build, and test with no other setup required. | ||
Click the badge above or [click here](https://codespaces.new/NVIDIA/cccl?quickstart=1&devcontainer_path=.devcontainer%2Fdevcontainer.json) to get started with CCCL's Dev Containers on Codespaces. This will start the default Dev Container environment. [Click here](https://github.com/codespaces/new?hide_repo_select=true&ref=main&repo=296416761&skip_quickstart=true) to start a Codespace with a particular environment and hardware configuration as shown: | ||
|
||
![Shows configuring a Codespace with a custom environment](../docs/images/codespaces.png) | ||
|
||
## For Maintainers: The `make_devcontainers.sh` Script | ||
|
||
### Overview | ||
|
||
[`make_devcontainers.sh`](./make_devcontainers.sh) generates devcontainer configurations for the unique combinations of CUDA Toolkit (CTK) versions and host compilers in [`ci/matrix.yaml`](../ci/matrix.yaml). | ||
|
||
### How It Works: | ||
|
||
1. Parses the matrix from `ci/matrix.yaml`. | ||
2. Use the top-level [`.devcontainer/devcontainer.json`](./devcontainer.json) as a template. For each unique combination of CTK version and host compiler, generate a corresponding `devcontainer.json` configuration, adjusting only the base Docker image to match the desired environment. | ||
3. Place the generated configurations in the `.devcontainer` directory, organizing them into subdirectories following the naming convention `cuda<CTK_VERSION>-<COMPILER_VERSION>`. | ||
|
||
For more information, see the `.devcontainer/make_devcontainers.sh --help` message. | ||
|
||
**Note**: When adding or updating supported environments, modify `matrix.yaml` and then rerun this script to synchronize the `devcontainer` configurations. |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,50 +1,130 @@ | ||
#!/usr/bin/env bash | ||
|
||
set -euo pipefail | ||
|
||
launch_devcontainer() { | ||
# Ensure the script is being executed in the cccl/ root | ||
cd "$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )/.."; | ||
|
||
if [[ -z $1 ]] || [[ -z $2 ]]; then | ||
echo "Usage: $0 [CUDA version] [Host compiler]" | ||
echo "Example: $0 12.2 gcc12" | ||
return 1 | ||
fi | ||
set -euo pipefail | ||
# Ensure we're in the repo root | ||
cd "$( cd "$( dirname "$(realpath -m "${BASH_SOURCE[0]}")" )" && pwd )/.."; | ||
|
||
|
||
local cuda_version="$1" | ||
local host_compiler="$2" | ||
local workspace="$(basename "$(pwd)")"; | ||
local tmpdir="$(mktemp -d)/${workspace}"; | ||
local path="$(pwd)/.devcontainer/cuda${cuda_version}-${host_compiler}"; | ||
if [[ ! -f ${path}/devcontainer.json ]]; then | ||
echo "Unknown CUDA [${cuda_version}] compiler [${host_compiler}] combination" | ||
echo "Requested devcontainer ${path}/devcontainer.json does not exist" | ||
return 1 | ||
print_help() { | ||
echo "Usage: $0 [-c|--cuda <CUDA version>] [-H|--host <Host compiler>] [-d|--docker]" | ||
echo "Launch a development container. If no CUDA version or Host compiler are specified," | ||
echo "the top-level devcontainer in .devcontainer/devcontainer.json will be used." | ||
echo "" | ||
echo "Options:" | ||
echo " -c, --cuda Specify the CUDA version. E.g., 12.2" | ||
echo " -H, --host Specify the host compiler. E.g., gcc12" | ||
echo " -d, --docker Launch the development environment in Docker directly without using VSCode." | ||
echo " -h, --help Display this help message and exit." | ||
} | ||
|
||
parse_options() { | ||
local OPTIONS=c:H:dh | ||
local LONG_OPTIONS=cuda:,host:,docker,help | ||
local PARSED_OPTIONS=$(getopt -n "$0" -o "${OPTIONS}" --long "${LONG_OPTIONS}" -- "$@") | ||
|
||
if [[ $? -ne 0 ]]; then | ||
exit 1 | ||
fi | ||
mkdir -p "${tmpdir}"; | ||
mkdir -p "${tmpdir}/.devcontainer"; | ||
cp -arL "$path/devcontainer.json" "${tmpdir}/.devcontainer"; | ||
sed -i "s@\${localWorkspaceFolder}@$(pwd)@g" "${tmpdir}/.devcontainer/devcontainer.json"; | ||
path="${tmpdir}"; | ||
|
||
local hash="$(echo -n "${path}" | xxd -pu - | tr -d '[:space:]')"; | ||
local url="vscode://vscode-remote/dev-container+${hash}/home/coder/cccl"; | ||
eval set -- "${PARSED_OPTIONS}" | ||
|
||
while true; do | ||
case "$1" in | ||
-c|--cuda) | ||
cuda_version="$2" | ||
shift 2 | ||
;; | ||
-H|--host) | ||
host_compiler="$2" | ||
shift 2 | ||
;; | ||
-d|--docker) | ||
docker_mode=true | ||
shift | ||
;; | ||
-h|--help) | ||
print_help | ||
exit 0 | ||
;; | ||
--) | ||
shift | ||
break | ||
;; | ||
*) | ||
echo "Invalid option: $1" | ||
print_help | ||
exit 1 | ||
;; | ||
esac | ||
done | ||
} | ||
|
||
launch_docker() { | ||
DOCKER_IMAGE=$(grep "image" "${path}/devcontainer.json" | sed 's/.*: "\(.*\)",/\1/') | ||
echo "Found image: ${DOCKER_IMAGE}" | ||
docker pull ${DOCKER_IMAGE} | ||
docker run \ | ||
-it --rm \ | ||
--user coder \ | ||
--workdir /home/coder/cccl \ | ||
--mount type=bind,src="$(pwd)",dst='/home/coder/cccl' \ | ||
${DOCKER_IMAGE} \ | ||
/bin/bash | ||
} | ||
|
||
echo "devcontainer URL: ${url}"; | ||
launch_vscode() { | ||
# Since Visual Studio Code allows only one instance per `devcontainer.json`, | ||
# this code prepares a unique temporary directory structure for each launch of a devcontainer. | ||
# By doing so, it ensures that multiple instances of the same environment can be run | ||
# simultaneously. The script replicates the `devcontainer.json` from the desired CUDA | ||
# and compiler environment into this temporary directory, adjusting paths to ensure the | ||
# correct workspace is loaded. A special URL is then generated to instruct VSCode to | ||
# launch the development container using this temporary configuration. | ||
local workspace="$(basename "$(pwd)")" | ||
local tmpdir="$(mktemp -d)/${workspace}" | ||
mkdir -p "${tmpdir}" | ||
mkdir -p "${tmpdir}/.devcontainer" | ||
cp -arL "${path}/devcontainer.json" "${tmpdir}/.devcontainer" | ||
sed -i 's@\\${localWorkspaceFolder}@$(pwd)@g' "${tmpdir}/.devcontainer/devcontainer.json" | ||
local path="${tmpdir}" | ||
local hash="$(echo -n "${path}" | xxd -pu - | tr -d '[:space:]')" | ||
local url="vscode://vscode-remote/dev-container+${hash}/home/coder/cccl" | ||
|
||
local launch=""; | ||
local launch="" | ||
if type open >/dev/null 2>&1; then | ||
launch="open"; | ||
launch="open" | ||
elif type xdg-open >/dev/null 2>&1; then | ||
launch="xdg-open"; | ||
launch="xdg-open" | ||
fi | ||
|
||
if [ -n "${launch}" ]; then | ||
code --new-window "${tmpdir}"; | ||
exec "${launch}" "${url}" >/dev/null 2>&1; | ||
echo "Launching VSCode Dev Container URL: ${url}" | ||
code --new-window "${tmpdir}" | ||
exec "${launch}" "${url}" >/dev/null 2>&1 | ||
fi | ||
} | ||
|
||
launch_devcontainer "$@"; | ||
main() { | ||
parse_options "$@" | ||
|
||
# If no CTK/Host compiler are provided, just use the default environment | ||
if [[ -z ${cuda_version:-} ]] && [[ -z ${host_compiler:-} ]]; then | ||
path=".devcontainer" | ||
else | ||
path=".devcontainer/cuda${cuda_version}-${host_compiler}" | ||
if [[ ! -f "${path}/devcontainer.json" ]]; then | ||
echo "Unknown CUDA [${cuda_version}] compiler [${host_compiler}] combination" | ||
echo "Requested devcontainer ${path}/devcontainer.json does not exist" | ||
exit 1 | ||
fi | ||
fi | ||
|
||
if ${docker_mode:-'false'}; then | ||
launch_docker | ||
else | ||
launch_vscode | ||
fi | ||
} | ||
|
||
main "$@" | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.