Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation: Flows, FireWorks, Dynamic Flows, Makers #338

Merged
merged 22 commits into from
Jun 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,5 +127,6 @@ Jobflow was designed by Alex Ganose, Anubhav Jain, Gian-Marco Rignanese, David W
[defining-jobs]: https://materialsproject.github.io/jobflow/tutorials/3-defining-jobs.html
[creating-flows]: https://materialsproject.github.io/jobflow/tutorials/4-creating-flows.html
[dynamic-flows]: https://materialsproject.github.io/jobflow/tutorials/5-dynamic-flows.html
[jobflow-database]: https://materialsproject.github.io/jobflow/tutorials/6-jobflow-database.html
[jobflow-fireworks]: https://materialsproject.github.io/jobflow/tutorials/7-fireworks.html
[makers]: https://materialsproject.github.io/jobflow/tutorials/6-makers.html
[generalized-makers]: https://materialsproject.github.io/jobflow/tutorials/7-generalized-makers.html
[jobflow-fireworks]: https://materialsproject.github.io/jobflow/tutorials/8-fireworks.html
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

Introduction <self>
Install jobflow <install>
Install FireWorks (Optional) <install_fireworks>
Tutorials <tutorials>

.. toctree::
Expand Down
100 changes: 100 additions & 0 deletions docs/install_fireworks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# FireWorks Setup

## Introduction

[FireWorks](https://github.com/materialsproject/fireworks) can be used to dispatch and monitor workflows, particularly those made using Jobflow. Follow the instructions below for how to use FireWorks with Jobflow. For additional details, refer to the full [Fireworks documentation](https://github.com/materialsproject/fireworks).

## Installation

To install FireWorks, run `pip install fireworks`.

## FireWorks DB Setup

If you plan to use FireWorks to dispatch your Jobflow workflows, you will also need to make a few configuration files: `FW_config.yaml`, `my_fworker.yaml`, `my_launchpad.yaml`, and `my_qadapter.yaml`. To begin, make a directory called `fw_config` where you will store the four files described in greater detail below. The directory structure will look like the following:

```text
fw_config
├── FW_config.yaml
├── my_fworker.yaml
├── my_launchpad.yaml
└── my_qadapter.yaml
```

### FW Config File

For the `FW_config.yaml`, you can use the following template. Make sure to update the path to the `fw_config` folder where the file resides.

```yaml
CONFIG_FILE_DIR: </path/to/fw_config>
QUEUE_UPDATE_INTERVAL: 2
```

### FWorker

For the `my_fworker.yaml`, you can use the following template. You do not need to make any modifications.

```yaml
name: my_fworker
category: ""
query: "{}"
```

### Launchpad

For the `my_launchpad.yaml`, you can use the following template. Replace the entries in `<>` with the appropriate values for your Mongo database.

```yaml
host: <host name>
port: 27017
name: <database name>
username: <username>
password: <password>
logdir: null
Istrm_lvl: DEBUG
user_indices: []
wf_user_indices: []
```

If you are accessing your MongoDB via a URI (e.g. as with MongoDB Atlas), then you will use the following `my_launchpad.yaml` template instead.

```yaml
host: <URI>
port: 27017
name: <database name>
uri_store: true
logdir: null
Istrm_lvl: DEBUG
user_indices: []
wf_user_indices: []
```

### QAdapter

Assuming you plan to use a queuing system for your compute jobs, you will need to make a `my_qadapter.yaml` file. For this, you will need to follow the instructions in the [FireWorks documentation](https://materialsproject.github.io/fireworks/qadapter_programming.html) for your specific job scheduling system. An example `my_qadapter.yaml` file is shown below for SLURM.

```yaml
_fw_name: CommonAdapter
_fw_q_type: SLURM
rocket_launch: rlaunch -w /path/to/fw_config/my_fworker.yaml singleshot
nodes: 2
walltime: 00:30:00
account: <account>
job_name: my_firework
qos: regular
pre_rocket: |
module load vasp
export ATOMATE2_VASP_CMD="srun -N 2 --ntasks-per-node=24"
```

In the above example, you would need to change the path in the `rocket_launch` field to the correct path to your `my_fworker.yaml`. The nodes, walltime, account, and qos are the corresponding parameters for your queuing system. Finally, anything in the `pre_rocket` field will be executed before the job begins running. It is a good place to load modules and set environment variables. A representative example has been provided above.

Finally, you will need to define a `FW_CONFIG_FILE` environment variable pointing to the `FW_config.yaml` file you made. For instance, in your `~/.bashrc` file, add the following line:
`export FW_CONFIG_FILE="/path/to/config/fw_config/FW_config.yaml"`.

### Database Initialization

```{warning}
Running `lpad reset` will clear your FireWorks launchpad, so only use this command if you are a new user.
```

To check that everything is working right with FireWorks, run `lpad reset` to ensure there is a connection to the database.
6 changes: 5 additions & 1 deletion docs/tutorials.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,8 @@ Tutorials
tutorials/1-quickstart
tutorials/2-introduction
tutorials/3-defining-jobs
tutorials/4-generalized-makers
tutorials/4-creating-flows
tutorials/5-dynamic-flows
tutorials/6-makers
tutorials/7-generalized-makers
tutorials/8-fireworks
Loading