Skip to content

Commit

Permalink
feedback: jacob review items
Browse files Browse the repository at this point in the history
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
  • Loading branch information
vsoch committed Jul 27, 2024
1 parent 9e7c64b commit 835cd68
Show file tree
Hide file tree
Showing 10 changed files with 268 additions and 161 deletions.
394 changes: 238 additions & 156 deletions 2024-RADIUSS-AWS/JupyterNotebook/tutorial/01_flux_tutorial.ipynb

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@
"3. Examples `flux kvs` that powers a lot of higher level commands\n",
"4. Advanced job specification interaction with flux job\n",
"\n",
"<br>\n",
"\n",
"## The structure of Flux instances\n",
"\n",
"As mentioned in [Chapter 2](./01_flux_tutorial.ipynb), a Flux instance is comprised of one or more Flux brokers. A high-level depiction of the design of a Flux broker is shown in the figure below.\n",
Expand All @@ -29,7 +31,7 @@
"</figure>\n",
"\n",
"Each broker is a program built on top of the ∅MQ networking library. The broker contains two main components. First, the broker implements Flux-specific networking abstractions over ∅MQ, such as remote-proceedure call (RPC) and publication-subscription (pub-sub). Second, the broker contains several core services, such as PMI (for MPI support), run control support (for enabling automatic startup of other services), and, most importantly, broker module management. The remainder of a Flux broker's functionality comes from broker modules: specially designed services that the broker can deploy in independent OS threads. Some examples of broker modules provided by Flux include:\n",
"* Job scheduling (both [traditional and hierarchical](./02_flux_scheduling.ipynb))\n",
"* Job scheduling (both traditional and hierarchical)\n",
"* [Fluxion](https://github.com/flux-framework/flux-sched) (Flux's advanced graph-based scheduler)\n",
"* Banks and accounting (for system-wide deployments of Flux)\n",
"* [PMIx](https://github.com/openpmix/openpmix) (for OpenMPI)\n",
Expand Down Expand Up @@ -58,6 +60,8 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>\n",
"\n",
"## Flux Modules\n",
"\n",
"To manage and query modules, Flux provides the `flux module` command. The sub-commands provided by `flux module` can be seen by running the cell below."
Expand Down Expand Up @@ -99,7 +103,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"While going through [Module 2](./02_flux_scheduling.ipynb), we've already encountered some built-in services provided by Flux, such as:\n",
"Some examples of Flux modules include:\n",
"* `job-ingest` (used by Flux submission commands like `flux batch` and `flux run`)\n",
"* `job-list` (used by `flux jobs`)\n",
"* `sched-fluxion-qmanager` (used by `flux tree`)\n",
Expand Down Expand Up @@ -289,6 +293,8 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>\n",
"\n",
"## flux jobspec generation\n",
"\n",
"Underlying much interaction with jobs is the creation of job specifications. When you use the command line or Python SDK and submit from a command or script, under the hood (back to that plumbing reference) we are creating a job specification \"Jobspec\" that is passed further through Flux. The command `flux submit` makes it possible to provide a similar command, but instead of running it, to generate the jobspec. Let's do that now. We will generate and view a Jobspec for a simple \"hello world\" job. We do that by adding `--dry-run`."
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
"\n",
"> Where can I learn to set this up on my own?\n",
"\n",
"If you're interested in installing Flux on your cluster, take a look at the [system instance instructions](https://flux-framework.readthedocs.io/en/latest/adminguide.html). If you are interested in running Flux on Kubernetes, check out the [Flux Operator](https://github.com/flux-framework/flux-operator). \n",
"If you're interested in installing Flux on your cluster, take a look at the [system instance instructions](https://flux-framework.readthedocs.io/projects/flux-core/en/latest/guide/admin.html). If you are interested in running Flux on Kubernetes, check out the [Flux Operator](https://github.com/flux-framework/flux-operator). \n",
"\n",
"> How can I run this tutorial on my own?\n",
"\n",
Expand Down Expand Up @@ -60,8 +60,6 @@
"* [DYAD's ReadTheDocs page](https://dyad.readthedocs.io/en/latest/)\n",
"* [DYAD's GitHub repository](https://github.com/flux-framework/dyad)\n",
"* [eScience 2022 Short Paper](https://dyad.readthedocs.io/en/latest/_downloads/27090817b034a89b76e5538e148fea9e/ShortPaper_2022_eScience_LLNL.pdf)\n",
"* [SC 2023 ACM Student Research Competition Extended Abstract](https://github.com/flux-framework/dyad/blob/main/docs/_static/ExtendedAbstract_2023_SC_ACM_SRC_DYAD.pdf)\n",
"* [IPDPS 2024 HiCOMB Workshop Paper](https://github.com/flux-framework/dyad/blob/main/docs/_static/Paper_2024_IPDPS_HiCOMB_DYAD.pdf)\n",
"\n",
"And, of course, you can always reach out to us on any of our [project repositories](https://flux-framework.org) and ask any questions that you have. We'd love your contribution to code, documentation, or just saying hello!\n",
"\n",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash

echo "Once upon a time... 📗️"
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash

echo "There was a little duck 🦆️"
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash

echo "Her name was pizzaquack 🍕️"
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash

echo "She was very fond of cheese 🧀️"
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash

echo "And running Flux 🌀️"
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash

echo "And so she ran Flux, while she ate her cheese 😋️"
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash

echo "And was so happy! The end. 🌈️"

0 comments on commit 835cd68

Please sign in to comment.