Skip to content

Commit

Permalink
beginning changes to tutorial
Browse files Browse the repository at this point in the history
finish updating step 0

add optional tags and fix code block for redis server

add which merlin checks and link to configuration docs

added minor clarifications

fix headers

add a snippet about the no docker version of this tutorial

add link to the contribute page

fix comment that was appearing in the docs

added a make install section and testing section

fix command to list examples

fix typo

fix command to list examples

add squash commits step in PR checklist

fix styling issues

add docs changes to CHANGELOG

Modify install instructions for docs

fix faq launcher formatting issues
  • Loading branch information
bgunnar5 committed Nov 15, 2022
1 parent d10510d commit d2249c6
Show file tree
Hide file tree
Showing 16 changed files with 198 additions and 59 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Added additional argument in test definitions to allow for a "cleanup" command
- Capability for non-user block in yaml
- .readthedocs.yaml and requirements.txt files for docs
- Small modifications to the Tutorial, Getting Started, Command Line, and Contributing pages in the docs

### Changed
- Rename lgtm.yml to .lgtm.yml
Expand Down
3 changes: 3 additions & 0 deletions docs/source/_static/custom.css
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,6 @@ td {
top: 0px;
}
}
.underline {
text-decoration: underline;
}
5 changes: 1 addition & 4 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
# 'sphinx.ext.autodoc',
# 'sphinx.ext.intersphinx',
# ]
extensions = []
extensions = ['sphinx.ext.autodoc']

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
Expand Down Expand Up @@ -103,7 +103,6 @@

html_css_files = ['custom.css']

#html_context = {"css_files": ["_static/custom.css"]}
# Custom sidebar templates, must be a dictionary that maps document names
# to template names.
#
Expand Down Expand Up @@ -185,8 +184,6 @@ def setup(app):
try:
app.add_javascript("custom.js")
app.add_javascript("https://cdn.jsdelivr.net/npm/clipboard@1/dist/clipboard.min.js")
#app.add_stylesheet('custom.css')
except AttributeError:
#app.add_css_file('custom.css')
app.add_js_file("custom.js")
app.add_js_file("https://cdn.jsdelivr.net/npm/clipboard@1/dist/clipboard.min.js")
47 changes: 30 additions & 17 deletions docs/source/faq.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
.. _faq:

.. role:: underline
:class: underline

FAQ
===
.. contents:: Frequently Asked Questions
Expand Down Expand Up @@ -100,7 +103,7 @@ Where are some example workflows?

.. code:: bash
$ merlin example --help
$ merlin example list
How do I launch a workflow?
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -185,7 +188,7 @@ Each step is ultimately designated as:
Normally this happens behinds the scenes, so you don't need to worry about it.
To hard-code this into your step logic, use a shell command such as ``exit $(MERLIN_HARD_FAIL)``.

.. note:: ``$(MERLIN_HARD_FAIL)``
.. note::
The ``$(MERLIN_HARD_FAIL)`` exit code will shutdown all workers connected to the queue associated
with the failed step. To shutdown *all* workers use the ``$(MERLIN_STOP_WORKERS)`` exit code

Expand Down Expand Up @@ -403,25 +406,35 @@ Do something like this:
nodes: 1
procs: 3
The arguments the LAUNCHER syntax will use:
:underline:`The arguments the LAUNCHER syntax will use`:

``procs``: The total number of MPI tasks

``nodes``: The total number of MPI nodes

``walltime``: The total walltime of the run (hh:mm:ss or mm:ss or ss) (not available in lsf)

``cores per task``: The number of hardware threads per MPI task

``gpus per task``: The number of GPUs per MPI task

:underline:`SLURM specific run flags`:

``slurm``: Verbatim flags only for the srun parallel launch (srun -n <nodes> -n <procs> <slurm>)

:underline:`FLUX specific run flags`:

``flux``: Verbatim flags for the flux parallel launch (flux mini run <flux>)

:underline:`LSF specific run flags`:

procs: The total number of MPI tasks
nodes: The total number of MPI nodes
walltime: The total walltime of the run (hh:mm:ss or mm:ss or ss) (not available in lsf)
cores per task: The number of hardware threads per MPI task
gpus per task: The number of GPUs per MPI task
``bind``: Flag for MPI binding of tasks on a node (default: -b rs)

SLURM specific run flags:
slurm: Verbatim flags only for the srun parallel launch (srun -n <nodes> -n <procs> <slurm>)
``num resource set``: Number of resource sets

FLUX specific run flags:
flux: Verbatim flags for the flux parallel launch (flux mini run <flux>)
``launch_distribution``: The distribution of resources (default: plane:{procs/nodes})

LSF specific run flags:
bind: Flag for MPI binding of tasks on a node (default: -b rs)
num resource set: Number of resource sets
launch_distribution : The distribution of resources (default: plane:{procs/nodes})
lsf: Verbatim flags only for the lsf parallel launch (jsrun ... <lsf>)
``lsf``: Verbatim flags only for the lsf parallel launch (jsrun ... <lsf>)

What is level_max_dirs?
~~~~~~~~~~~~~~~~~~~~~~~
Expand Down
62 changes: 61 additions & 1 deletion docs/source/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,13 @@ Check out the :doc:`Tutorial<./tutorial>`!

Developer Setup
++++++++++++++++++
The developer setup can be done via pip or via make. This section will cover how to do both.

Additionally, there is an alternative method to setup merlin on supercomputers. See the :doc:`Spack <./spack>` section for more details.

Pip Setup
******************

To install with the additional developer dependencies, use::

pip3 install "merlin[dev]"
Expand All @@ -21,8 +28,35 @@ or::

pip3 install -e "git+https://github.com/LLNL/merlin.git@develop#egg=merlin[dev]"

See the :doc:`Spack <./spack>` section for an alternative method to setup merlin on supercomputers.
Make Setup
*******************

Visit the `Merlin repository <https://github.com/LLNL/merlin/>`_ on github. `Create a fork of the repo <https://docs.github.com/en/get-started/quickstart/fork-a-repo>`_ and `clone it <https://docs.github.com/en/get-started/quickstart/fork-a-repo#cloning-your-forked-repository>`_ onto your system.

Change directories into the merlin repo:

.. code-block:: bash
$ cd merlin/
Install Merlin with the developer dependencies:

.. code-block:: bash
$ make install-dev
This will create a virtualenv, start it, and install Merlin and it's dependencies for you.

More documentation about using Virtualenvs with Merlin can be found at
:doc:`Using Virtualenvs with Merlin <./virtualenv>`.

We can make sure it's installed by running:

.. code-block:: bash
$ merlin --version
If you don't see a version number, you may need to restart your virtualenv and try again.

Configuring Merlin
*******************
Expand All @@ -32,6 +66,32 @@ Documentation for merlin configuration is in the :doc:`Configuring Merlin <./mer

That's it. To start running Merlin see the :doc:`Merlin Workflows. <./merlin_workflows>`

(Optional) Testing Merlin
*************************

.. warning::

With python 3.6 you may see some tests fail and a unicode error presented. To fix this, you need to reset the LC_ALL environment variable to en_US.utf8.

If you have ``make`` installed and the `Merlin repository <https://github.com/LLNL/merlin/>`_ cloned, you can run the test suite provided in the Makefile by running:

.. code-block:: bash
$ make tests
This will run both the unit tests suite and the end-to-end tests suite.

If you'd just like to run the unit tests you can run:

.. code-block:: bash
$ make unit-tests
Similarly, if you'd just like to run the end-to-end tests you can run:

.. code-block:: bash
$ make e2e-tests
Custom Setup
+++++++++++++
Expand Down
2 changes: 1 addition & 1 deletion docs/source/merlin_commands.rst
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ If you want to run an example workflow, use Merlin's ``merlin example``:

.. code:: bash
$ merlin example --help
$ merlin example list
This will list the available example workflows and a description for each one. To
select one:
Expand Down
6 changes: 4 additions & 2 deletions docs/source/merlin_developer.rst
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,11 @@ To expedite review, please ensure that pull requests

- Are from a meaningful branch name (e.g. ``feature/my_name/cool_thing``)

- Into the `appropriate branch <https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow>`_
- Are being merged into the `appropriate branch <https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow>`_

- Include testing for any new features

- unit tests in ``tests/*``
- unit tests in ``tests/unit``
- integration tests in ``tests/integration``

- Include descriptions of the changes
Expand All @@ -64,6 +64,8 @@ To expedite review, please ensure that pull requests
- in ``CHANGELOG.md``
- in ``merlin.__init__.py``

- Have `squashed <https://github.com/LLNL/merlin/wiki/Squash-commits>`_ commits

Testing
+++++++

Expand Down
4 changes: 2 additions & 2 deletions docs/source/merlin_server.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,11 +62,11 @@ merlin server. A detailed list of commands can be found in the `Merlin Server Co
Note: Running "merlin server init" again will NOT override any exisiting configuration
that the users might have set or edited. By running this command again any missing files
will be created for the users with exisiting defaults. HOWEVER it is highly advised that
users back up their configuration in case an error occurs where configuration files are override.
users back up their configuration in case an error occurs where configuration files are overriden.

.. toctree::
:maxdepth: 1
:caption: Merlin Server Settings:

server/configuration
server/commands
server/commands
2 changes: 1 addition & 1 deletion docs/source/merlin_workflows.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ provides documentation on running these Merlin workflow examples.
Overview
--------

List the built-in Merlin workflow examples with ``merlin example --help``.
List the built-in Merlin workflow examples with ``merlin example list``.

The Merlin team is working on adding a more diverse array of example workflows
like these.
Expand Down
31 changes: 24 additions & 7 deletions docs/source/modules/before.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,40 @@ start the tutorial modules:

__ https://www.python.org/downloads/release/python-360/

* Make sure you have `pip`__ version 22.3 or newer.

__ https://www.pypi.org/project/pip/

* You can upgrade pip to the latest version with:

.. code-block:: bash
pip install --upgrade pip
* OR you can upgrade to a specific version with:

.. code-block:: bash
pip install --upgrade pip==x.y.z
* Make sure you have `GNU make tools`__ and `compilers`__.

__ https://www.gnu.org/software/make/
__ https://gcc.gnu.org/

* Install `docker`__.
* (OPTIONAL) Install `docker`__.

__ https://docs.docker.com/install/

* Download OpenFOAM image with:
* Download OpenFOAM image with:

.. code-block:: bash
.. code-block:: bash
docker pull cfdengine/openfoam
docker pull cfdengine/openfoam
* Download redis image with:
* Download redis image with:

.. code-block:: bash
.. code-block:: bash
docker pull redis
docker pull redis
2 changes: 2 additions & 0 deletions docs/source/modules/contribute.rst
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,5 @@ Contributing to Merlin is easy! Just `send us a pull request <https://github.com
Merlin uses a rough approximation of the Git Flow branching model. The develop branch contains the latest contributions, and main is always tagged and points to the latest stable release.

If you're a contributor, try to test and run on develop. That's where all the magic is happening (and where we hope bugs stop).

More detailed information on contributing can be found on the :doc:`Contributing page<../merlin_developer>`.
Loading

0 comments on commit d2249c6

Please sign in to comment.