Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs][KubeRay] Update KubeRay + Kueue guides to use newer versions of Kueue #48564

Merged
merged 1 commit into from
Nov 5, 2024
Merged

[Docs][KubeRay] Update KubeRay + Kueue guides to use newer versions of Kueue #48564

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 6 additions & 10 deletions doc/source/cluster/kubernetes/examples/rayjob-kueue-gang-scheduling.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,16 +37,16 @@ Create a GKE cluster with the `enable-autoscaling` option:
```bash
gcloud container clusters create kuberay-gpu-cluster \
--num-nodes=1 --min-nodes 0 --max-nodes 1 --enable-autoscaling \
--zone=us-west1-b --machine-type e2-standard-4 --cluster-version 1.29
--zone=us-east4-c --machine-type e2-standard-4
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I switched the zone just based on personal experience, this zone has less stockouts for L4 GPUs

```

Create a GPU node pool with the `enable-queued-provisioning` option enabled:
```bash
gcloud beta container node-pools create gpu-node-pool \
gcloud container node-pools create gpu-node-pool \
--accelerator type=nvidia-l4,count=1,gpu-driver-version=latest \
--enable-queued-provisioning \
--reservation-affinity=none \
--zone us-west1-b \
--zone us-east4-c \
--cluster kuberay-gpu-cluster \
--num-nodes 0 \
--min-nodes 0 \
Expand All @@ -55,14 +55,10 @@ gcloud beta container node-pools create gpu-node-pool \
--machine-type g2-standard-4
```

This command creates a node pool which initially has zero nodes. Use the `gcloud beta` command because some of the flags have beta status.
This command creates a node pool which initially has zero nodes.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing comma:
This command creates a node pool, which initially has zero nodes.

The `--enable-queued-provisioning` flag enables "queued provisioning" in the Kubernetes node autoscaler using the ProvisioningRequest API. More details are below.
You need to use the `--reservation-affinity=none` flag because GKE doesn't support Node Reservations with ProvisioningRequest.

:::{note}
"enable-queued-provisioning" is only available on versions 1.28+ with the `gcloud beta` command
:::


## Install the KubeRay operator

Expand All @@ -71,9 +67,9 @@ The KubeRay operator Pod must be on the CPU node if you set up the taint for the

## Install Kueue

Install Kueue with the ProvisioningRequest API enabled.
Install the latest released version of Kueue.
```
kubectl apply --server-side -f https://github.com/kubernetes-sigs/kueue/releases/download/v0.6.0/manifests-alpha-enabled.yaml
kubectl apply --server-side -f https://github.com/kubernetes-sigs/kueue/releases/download/v0.8.2/manifests.yaml
```

See [Kueue Installation](https://kueue.sigs.k8s.io/docs/installation/#install-a-released-version) for more details on installing Kueue.
Expand Down
2 changes: 1 addition & 1 deletion doc/source/cluster/kubernetes/examples/rayjob-kueue-priority-scheduling.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ The KubeRay operator Pod must be on the CPU node if you set up the taint for the
## Step 2: Install Kueue

```bash
VERSION=v0.6.0
VERSION=v0.8.2
kubectl apply --server-side -f https://github.com/kubernetes-sigs/kueue/releases/download/$VERSION/manifests.yaml
```

Expand Down
Loading