Skip to content

Commit

Permalink
Add GPU watch
Browse files Browse the repository at this point in the history
  • Loading branch information
jdeschamps committed May 31, 2023
1 parent be267bc commit 50bacfc
Show file tree
Hide file tree
Showing 4 changed files with 35 additions and 2 deletions.
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ repos:
rev: v0.33.0
hooks:
- id: markdownlint
args: [--fix, --disable, MD013, --]
args: [--fix, --disable, MD013, MD024, --]

# notebooks
- repo: https://github.com/kynan/nbstripout
Expand Down
2 changes: 1 addition & 1 deletion docs/n2v_hpc/start_jupyter.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ module load Jupyter-Notebook/6.4.3
Then, we want to submit a job on a GPU node :fire::

```bash title="Submit GPU job"
srun --time=04:00:00 --job-name=<your.name> -p neubias --gres=gpu:1 --qos=gpu jupyter-notebook --no-browser --ip='0.0.0.0'
srun --time=04:00:00 --job-name=<your.name> -p neubias --gres=gpu:1 --qos=gpu --mem=16G jupyter-notebook --no-browser --ip='0.0.0.0'
```

You should now have a `jupyter` server running on a GPU node.
Expand Down
32 changes: 32 additions & 0 deletions docs/n2v_hpc/watch_gpu.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
tags:
- help
---
# Watch GPU consumption

In this section, we demonstrate how to watch the GPU consumption of your
jupyter job on the compute node.

## Check the compute node

```bash title=""
squeue | grep <your-job-name>
```

You should see something like `maestro-####`, write down the `####` numbers.

## Connect to the compute node

Open a new terminal and ssh into the compute node

```bash
ssh m####
```

where `####` are the numbers from the previous section

## Watch GPU consumption

```bash
watch nvidia-smi
```
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ nav:
- n2v_hpc/activate_env.md
- n2v_hpc/data.md
- n2v_hpc/start_jupyter.md
- n2v_hpc/watch_gpu.md
- n2v_hpc/train.md
- n2v_hpc/bonus.md
- napari.md
Expand Down

0 comments on commit 50bacfc

Please sign in to comment.