Skip to content

Commit

Permalink
Add devcontainer readme (#481)
Browse files Browse the repository at this point in the history
* In progress.

* Add manual docker instructions.

* document make_devcontainers.sh

* Finalized devcontainer README.

* Split of contributing guide to another PR.

* Modularize launch script.

* Add docs for launch script.

* Explain weird vscode launch logic.

* Try HTML for clickable URI.

* Try using a reference link.

* GitHub markdown won't render vscode URI.

* Remove old notes.

* Remove stale.

* Intro.

* Fix bullets.

* [skip ci] Fix launch script path.

* [skip ci] Add reference to rapidsai/devcontainers.

* [skip ci] Stale comment.

* Update CI overview.

* Apply suggestions from code review

Co-authored-by: Bradley Dice <bdice@bradleydice.com>
Co-authored-by: Vyas Ramasubramani <vyas.ramasubramani@gmail.com>
Co-authored-by: Allard Hendriksen <allard@allardhendriksen.nl>

* Heavily simplified devcontainer guide.

* Add section about codespaces.

* [skip ci] Minor wording changes

* [skip ci] Update launch script.

* [skip ci] Update docs for Docker launch.

* [skip ci] Add codespace badge to top level readme.

* [skip ci] Update link.

* [skip ci] Add links.

* [skip ci] Add section on devcontainer customization.

* [skip ci] Remove extra space.

* Apply suggestions from code review

Co-authored-by: Christopher Harris <xixonia@gmail.com>

---------

Co-authored-by: Bradley Dice <bdice@bradleydice.com>
Co-authored-by: Vyas Ramasubramani <vyas.ramasubramani@gmail.com>
Co-authored-by: Allard Hendriksen <allard@allardhendriksen.nl>
Co-authored-by: Christopher Harris <xixonia@gmail.com>
  • Loading branch information
5 people authored Oct 3, 2023
1 parent 865bcf8 commit af59bb6
Show file tree
Hide file tree
Showing 9 changed files with 267 additions and 59 deletions.
116 changes: 116 additions & 0 deletions .devcontainer/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
> **Note**
> The instructions in this README are specific to Linux development environments. Instructions for Windows are coming soon!
[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/NVIDIA/cccl?quickstart=1&devcontainer_path=.devcontainer%2Fdevcontainer.json)

# CCCL Dev Containers

CCCL uses [Development Containers](https://containers.dev/) to provide consistent and convenient development environments for both local development and for CI. This guide covers setup in [Visual Studio Code](#quickstart-vscode-recommended) and [Docker](#quickstart-docker-manual-approach).

## Quickstart: VSCode (Recommended)

### Prerequisites
- [Visual Studio Code](https://code.visualstudio.com/)
- [Remote - Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers)

### Steps

1. Clone the Repository
```bash
git clone https://github.com/nvidia/cccl.git
```
2. Open the cloned directory in VSCode

3. Launch a Dev Container by clicking the prompt suggesting to "Reopen in Container"

![Shows "Reopen in Container" prompt when opening the cccl directory in VScode.](./img/reopen_in_container.png)

- Alternatively, use the Command Palette to start a Dev Container. Press `Ctrl+Shift+P` to open the Command Palette. Type "Remote-Containers: Reopen in Container" and select it.

![Shows "Reopen in Container" in command pallete.](./img/open_in_container_manual.png)

4. Select an environment with the desired CTK and host compiler from the list:

![Shows list of available container environments.](./img/container_list.png)

5. VSCode will initialize the selected Dev Container. This can take a few minutes the first time.

6. Once initialized, the local `cccl/` directory is mirrored into the container to ensure any changes are persistent.

7. Done! See the [contributing guide](../CONTRIBUTING.md#building-and-testing) for instructions on how to build and run tests.

### (Optional) Authenticate with GitHub for `sccache`

After starting the container, there will be a prompt to authenticate with GitHub. This grants access to a [`sccache`](https://github.com/mozilla/sccache) server shared with CI and greatly accelerates local build times. This is currently limited to NVIDIA employees belonging to the `NVIDIA` or `rapidsai` GitHub organizations.

Without authentication to the remote server, `sccache` will still accelerate local builds by using a filesystem cache.

Follow the instructions in the prompt as below and enter the one-time code at https://github.com/login/device

![Shows authentication with GitHub to access sccache bucket.](./img/github_auth.png)

To manually trigger this authentication, execute the `devcontainer-utils-vault-s3-init` script within the container.

For more information about the sccache configuration and authentication, see the documentation at [`rapidsai/devcontainers`](https://github.com/rapidsai/devcontainers/blob/branch-23.10/USAGE.md#build-caching-with-sccache).

## Quickstart: Docker (Manual Approach)

### Prerequisites
- [Docker](https://docs.docker.com/desktop/install/linux-install/)

### Steps
1. Clone the repository and use the [`launch.sh`](./launch.sh) script to launch the default container environment
```bash
git clone https://github.com/nvidia/cccl.git
cd cccl
./.devcontainer/launch.sh --docker
```
This script starts an interactive shell as the `coder` user inside the container with the local `cccl/` directory mirrored into `/home/coder/cccl`.

For specific environments, use the `--cuda` and `--host` options:
```bassh
./.devcontainer/launch.sh --docker --cuda 12.2 --host gcc10
```
See `./.devcontainer/launch.sh --help` for more information.

2. Done. See the [contributing guide](../CONTRIBUTING.md#building-and-testing) for instructions on how to build and run tests.

## Available Environments

CCCL provides environments for both the oldest and newest supported CUDA versions with all compatible host compilers.

Look in the [`.devcontainer/`](.) directory to see the available configurations. The top-level [`devcontainer.json`](./devcontainer.json) serves as the default environment. All `devcontainer.json` files in the `cuda<CTK_VERSION>-<HOST-COMPILER>` sub-directories are variations on this top-level file, with different base images for the different CUDA and host compiler versions.

## VSCode Customization

By default, CCCL's Dev Containers come with certain VSCode settings and extensions configured by default, as can be seen in the [`devcontainer.json`](./devcontainer.json) file. This can be further customized by users without needing to modify the `devcontainer.json` file directly.
For extensions, the [`dev.containers.defaultExtensions` setting](https://code.visualstudio.com/docs/devcontainers/containers#_always-installed-extensions) allows listing extensions that will always be installed.
For more general customizations, VSCode allows using a dotfile repository. See the [VSCode documentation](https://code.visualstudio.com/docs/devcontainers/containers#_personalizing-with-dotfile-repositories) for more information.
## GitHub Codespaces
[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/NVIDIA/cccl?quickstart=1&devcontainer_path=.devcontainer%2Fdevcontainer.json)
One of the benefits of Dev Containers is that they integrate natively with [GitHub Codespaces](https://github.com/features/codespaces). Codespaces provide a VSCode development environment right in your browser running on a machine in the cloud. This provides a truly one-click, turnkey development environment where you can develop, build, and test with no other setup required.
Click the badge above or [click here](https://codespaces.new/NVIDIA/cccl?quickstart=1&devcontainer_path=.devcontainer%2Fdevcontainer.json) to get started with CCCL's Dev Containers on Codespaces. This will start the default Dev Container environment. [Click here](https://github.com/codespaces/new?hide_repo_select=true&ref=main&repo=296416761&skip_quickstart=true) to start a Codespace with a particular environment and hardware configuration as shown:

![Shows configuring a Codespace with a custom environment](../docs/images/codespaces.png)

## For Maintainers: The `make_devcontainers.sh` Script

### Overview

[`make_devcontainers.sh`](./make_devcontainers.sh) generates devcontainer configurations for the unique combinations of CUDA Toolkit (CTK) versions and host compilers in [`ci/matrix.yaml`](../ci/matrix.yaml).

### How It Works:

1. Parses the matrix from `ci/matrix.yaml`.
2. Use the top-level [`.devcontainer/devcontainer.json`](./devcontainer.json) as a template. For each unique combination of CTK version and host compiler, generate a corresponding `devcontainer.json` configuration, adjusting only the base Docker image to match the desired environment.
3. Place the generated configurations in the `.devcontainer` directory, organizing them into subdirectories following the naming convention `cuda<CTK_VERSION>-<COMPILER_VERSION>`.

For more information, see the `.devcontainer/make_devcontainers.sh --help` message.

**Note**: When adding or updating supported environments, modify `matrix.yaml` and then rerun this script to synchronize the `devcontainer` configurations.
Binary file added .devcontainer/img/container_list.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .devcontainer/img/github_auth.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .devcontainer/img/open_in_container_manual.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added .devcontainer/img/reopen_in_container.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
148 changes: 114 additions & 34 deletions .devcontainer/launch.sh
Original file line number Diff line number Diff line change
@@ -1,50 +1,130 @@
#!/usr/bin/env bash

set -euo pipefail

launch_devcontainer() {
# Ensure the script is being executed in the cccl/ root
cd "$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )/..";

if [[ -z $1 ]] || [[ -z $2 ]]; then
echo "Usage: $0 [CUDA version] [Host compiler]"
echo "Example: $0 12.2 gcc12"
return 1
fi
set -euo pipefail
# Ensure we're in the repo root
cd "$( cd "$( dirname "$(realpath -m "${BASH_SOURCE[0]}")" )" && pwd )/..";


local cuda_version="$1"
local host_compiler="$2"
local workspace="$(basename "$(pwd)")";
local tmpdir="$(mktemp -d)/${workspace}";
local path="$(pwd)/.devcontainer/cuda${cuda_version}-${host_compiler}";
if [[ ! -f ${path}/devcontainer.json ]]; then
echo "Unknown CUDA [${cuda_version}] compiler [${host_compiler}] combination"
echo "Requested devcontainer ${path}/devcontainer.json does not exist"
return 1
print_help() {
echo "Usage: $0 [-c|--cuda <CUDA version>] [-H|--host <Host compiler>] [-d|--docker]"
echo "Launch a development container. If no CUDA version or Host compiler are specified,"
echo "the top-level devcontainer in .devcontainer/devcontainer.json will be used."
echo ""
echo "Options:"
echo " -c, --cuda Specify the CUDA version. E.g., 12.2"
echo " -H, --host Specify the host compiler. E.g., gcc12"
echo " -d, --docker Launch the development environment in Docker directly without using VSCode."
echo " -h, --help Display this help message and exit."
}

parse_options() {
local OPTIONS=c:H:dh
local LONG_OPTIONS=cuda:,host:,docker,help
local PARSED_OPTIONS=$(getopt -n "$0" -o "${OPTIONS}" --long "${LONG_OPTIONS}" -- "$@")

if [[ $? -ne 0 ]]; then
exit 1
fi
mkdir -p "${tmpdir}";
mkdir -p "${tmpdir}/.devcontainer";
cp -arL "$path/devcontainer.json" "${tmpdir}/.devcontainer";
sed -i "s@\${localWorkspaceFolder}@$(pwd)@g" "${tmpdir}/.devcontainer/devcontainer.json";
path="${tmpdir}";

local hash="$(echo -n "${path}" | xxd -pu - | tr -d '[:space:]')";
local url="vscode://vscode-remote/dev-container+${hash}/home/coder/cccl";
eval set -- "${PARSED_OPTIONS}"

while true; do
case "$1" in
-c|--cuda)
cuda_version="$2"
shift 2
;;
-H|--host)
host_compiler="$2"
shift 2
;;
-d|--docker)
docker_mode=true
shift
;;
-h|--help)
print_help
exit 0
;;
--)
shift
break
;;
*)
echo "Invalid option: $1"
print_help
exit 1
;;
esac
done
}

launch_docker() {
DOCKER_IMAGE=$(grep "image" "${path}/devcontainer.json" | sed 's/.*: "\(.*\)",/\1/')
echo "Found image: ${DOCKER_IMAGE}"
docker pull ${DOCKER_IMAGE}
docker run \
-it --rm \
--user coder \
--workdir /home/coder/cccl \
--mount type=bind,src="$(pwd)",dst='/home/coder/cccl' \
${DOCKER_IMAGE} \
/bin/bash
}

echo "devcontainer URL: ${url}";
launch_vscode() {
# Since Visual Studio Code allows only one instance per `devcontainer.json`,
# this code prepares a unique temporary directory structure for each launch of a devcontainer.
# By doing so, it ensures that multiple instances of the same environment can be run
# simultaneously. The script replicates the `devcontainer.json` from the desired CUDA
# and compiler environment into this temporary directory, adjusting paths to ensure the
# correct workspace is loaded. A special URL is then generated to instruct VSCode to
# launch the development container using this temporary configuration.
local workspace="$(basename "$(pwd)")"
local tmpdir="$(mktemp -d)/${workspace}"
mkdir -p "${tmpdir}"
mkdir -p "${tmpdir}/.devcontainer"
cp -arL "${path}/devcontainer.json" "${tmpdir}/.devcontainer"
sed -i 's@\\${localWorkspaceFolder}@$(pwd)@g' "${tmpdir}/.devcontainer/devcontainer.json"
local path="${tmpdir}"
local hash="$(echo -n "${path}" | xxd -pu - | tr -d '[:space:]')"
local url="vscode://vscode-remote/dev-container+${hash}/home/coder/cccl"

local launch="";
local launch=""
if type open >/dev/null 2>&1; then
launch="open";
launch="open"
elif type xdg-open >/dev/null 2>&1; then
launch="xdg-open";
launch="xdg-open"
fi

if [ -n "${launch}" ]; then
code --new-window "${tmpdir}";
exec "${launch}" "${url}" >/dev/null 2>&1;
echo "Launching VSCode Dev Container URL: ${url}"
code --new-window "${tmpdir}"
exec "${launch}" "${url}" >/dev/null 2>&1
fi
}

launch_devcontainer "$@";
main() {
parse_options "$@"

# If no CTK/Host compiler are provided, just use the default environment
if [[ -z ${cuda_version:-} ]] && [[ -z ${host_compiler:-} ]]; then
path=".devcontainer"
else
path=".devcontainer/cuda${cuda_version}-${host_compiler}"
if [[ ! -f "${path}/devcontainer.json" ]]; then
echo "Unknown CUDA [${cuda_version}] compiler [${host_compiler}] combination"
echo "Requested devcontainer ${path}/devcontainer.json does not exist"
exit 1
fi
fi

if ${docker_mode:-'false'}; then
launch_docker
else
launch_vscode
fi
}

main "$@"

2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/NVIDIA/cccl?quickstart=1&devcontainer_path=.devcontainer%2Fdevcontainer.json)

# CUDA C++ Core Libraries (CCCL)

Welcome to the CUDA C++ Core Libraries (CCCL) where our mission is to make CUDA C++ more delightful.
Expand Down
Loading

0 comments on commit af59bb6

Please sign in to comment.