Docs/install changes

.readthedocs.yaml

@@ -0,0 +1,13 @@
+version: 2
+
+build:
+  os: "ubuntu-20.04"
+  tools:
+    python: "3.8"
+
+sphinx:
+  configuration: docs/source/conf.py
+
+python:
+  install:
+    - requirements: docs/requirements.txt

CHANGELOG.md

@@ -16,6 +16,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   ranks 0 and 1 of a multi-rank allocation
 - Added additional argument in test definitions to allow for a "cleanup" command
 - Capability for non-user block in yaml
+- .readthedocs.yaml and requirements.txt files for docs
+- Small modifications to the Tutorial, Getting Started, Command Line, and Contributing pages in the docs
 
 ### Changed
 - Rename lgtm.yml to .lgtm.yml
@@ -24,13 +26,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Update requirements to require redis 4.3.4 for acl user channel support
 - Updated tutorial documentation to use merlin server over manual redis installation.
 - Added ssl to the broker and results backend server checks when "merlin info" is called
+- Removed theme_override.css from docs/_static/ since it is no longer needed with the updated version of sphinx
+- Updated docs/Makefile to include a pip install for requirements and a clean command
 
 ### Fixed
 - Fixed return values from scripts with main() to fix testing errors. 
 - CI test for CHANGELOG modifcations
 - Fix the cert_req typo in the merlin config docs, it should read cert_reqs
 - Removed emoji from issue templates that were breaking doc builds
 - Including .temp template files in MANIFEST
+- Styling in the footer for docs
+- Horizontal scroll overlap in the variables page of the docs
 
 ## [1.8.5]
 ### Added

docs/Makefile

@@ -23,6 +23,9 @@ view: code-docs html
 # Catch-all target: route all unknown targets to Sphinx using the new
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile
+	pip install -r requirements.txt
 	echo $(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
 	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
 
+clean:
+	rm -rf build/

docs/requirements.txt

@@ -0,0 +1,56 @@
+#
+# This file is autogenerated by pip-compile with python 3.8
+# To update, run:
+#
+#    pip-compile requirements.in
+#
+alabaster==0.7.12
+    # via sphinx
+babel==2.10.3
+    # via sphinx
+certifi==2022.9.24
+    # via requests
+charset-normalizer==2.1.1
+    # via requests
+docutils==0.17.1
+    # via sphinx
+idna==3.4
+    # via requests
+imagesize==1.4.1
+    # via sphinx
+importlib-metadata==5.0.0
+    # via sphinx
+jinja2==3.0.3
+    # via sphinx
+markupsafe==2.1.1
+    # via jinja2
+packaging==21.3
+    # via sphinx
+pygments==2.13.0
+    # via sphinx
+pyparsing==3.0.9
+    # via packaging
+pytz==2022.5
+    # via babel
+requests==2.28.1
+    # via sphinx
+snowballstemmer==2.2.0
+    # via sphinx
+sphinx==5.3.0
+    # via -r requirements.in
+sphinxcontrib-applehelp==1.0.2
+    # via sphinx
+sphinxcontrib-devhelp==1.0.2
+    # via sphinx
+sphinxcontrib-htmlhelp==2.0.0
+    # via sphinx
+sphinxcontrib-jsmath==1.0.1
+    # via sphinx
+sphinxcontrib-qthelp==1.0.3
+    # via sphinx
+sphinxcontrib-serializinghtml==1.1.5
+    # via sphinx
+urllib3==1.26.12
+    # via requests
+zipp==3.10.0
+    # via importlib-metadata

docs/source/_static/custom.css

@@ -22,3 +22,18 @@ div.sphinxsidebar {
     max-height: 100%;
     overflow-y: auto;
 }
+td {
+	max-width: 300px;
+}
+@media screen and (min-width: 875px) {
+	.sphinxsidebar {
+		background-color: #fff;
+		margin-left: 0;
+		z-index: 1;
+		height: 100vh;
+		top: 0px;
+	}
+}
+.underline {
+    text-decoration: underline;
+}

docs/source/conf.py

@@ -47,7 +47,7 @@
 #    'sphinx.ext.autodoc',
 #    'sphinx.ext.intersphinx',
 # ]
-extensions = []
+extensions = ['sphinx.ext.autodoc']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
@@ -101,11 +101,7 @@
 # so a file named "default.css" will overwrite the builtin "default.css".
 html_static_path = ['_static']
 
-html_context = {
-    'css_files': [
-        '_static/theme_overrides.css',  # override wide tables in RTD theme
-    ],
-}
+html_css_files = ['custom.css']
 
 # Custom sidebar templates, must be a dictionary that maps document names
 # to template names.
@@ -188,8 +184,6 @@ def setup(app):
     try:
         app.add_javascript("custom.js")
         app.add_javascript("https://cdn.jsdelivr.net/npm/clipboard@1/dist/clipboard.min.js")
-        app.add_stylesheet('custom.css')
     except AttributeError:
-        app.add_css_file('custom.css')
         app.add_js_file("custom.js")
         app.add_js_file("https://cdn.jsdelivr.net/npm/clipboard@1/dist/clipboard.min.js")

docs/source/faq.rst

@@ -1,5 +1,8 @@
 .. _faq:
 
+.. role:: underline
+   :class: underline
+
 FAQ
 ===
 .. contents:: Frequently Asked Questions
@@ -100,7 +103,7 @@ Where are some example workflows?
 
 .. code:: bash
 
-   $ merlin example --help
+   $ merlin example list
 
 How do I launch a workflow?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -185,7 +188,7 @@ Each step is ultimately designated as:
 Normally this happens behinds the scenes, so you don't need to worry about it.
 To hard-code this into your step logic, use a shell command such as ``exit $(MERLIN_HARD_FAIL)``.
 
-.. note:: ``$(MERLIN_HARD_FAIL)``
+.. note::
    The ``$(MERLIN_HARD_FAIL)`` exit code will shutdown all workers connected to the queue associated
    with the failed step. To shutdown *all* workers use the ``$(MERLIN_STOP_WORKERS)`` exit code
 
@@ -403,25 +406,35 @@ Do something like this:
         nodes: 1
         procs: 3
 
-The arguments the LAUNCHER syntax will use:
+:underline:`The arguments the LAUNCHER syntax will use`:
+
+``procs``: The total number of MPI tasks
+
+``nodes``: The total number of MPI nodes
+
+``walltime``: The total walltime of the run (hh:mm:ss or mm:ss or ss) (not available in lsf)
+
+``cores per task``: The number of hardware threads per MPI task
+
+``gpus per task``: The number of GPUs per MPI task
+
+:underline:`SLURM specific run flags`:
+
+``slurm``: Verbatim flags only for the srun parallel launch (srun -n <nodes> -n <procs> <slurm>)
+
+:underline:`FLUX specific run flags`:
+
+``flux``: Verbatim flags for the flux parallel launch (flux mini run <flux>)
+
+:underline:`LSF specific run flags`:
 
-procs: The total number of MPI tasks
-nodes: The total number of MPI nodes
-walltime: The total walltime of the run (hh:mm:ss or mm:ss or ss) (not available in lsf)
-cores per task: The number of hardware threads per MPI task
-gpus per task: The number of GPUs per MPI task
+``bind``: Flag for MPI binding of tasks on a node (default: -b rs)
 
-SLURM specific run flags:
-slurm: Verbatim flags only for the srun parallel launch (srun -n <nodes> -n <procs> <slurm>)
+``num resource set``: Number of resource sets
 
-FLUX specific run flags:
-flux: Verbatim flags for the flux parallel launch (flux mini run <flux>)
+``launch_distribution``: The distribution of resources (default: plane:{procs/nodes})
 
-LSF specific run flags:
-bind: Flag for MPI binding of tasks on a node (default: -b rs)
-num resource set: Number of resource sets
-launch_distribution : The distribution of resources (default: plane:{procs/nodes})
-lsf: Verbatim flags only for the lsf parallel launch (jsrun ... <lsf>)
+``lsf``: Verbatim flags only for the lsf parallel launch (jsrun ... <lsf>)
 
 What is level_max_dirs?
 ~~~~~~~~~~~~~~~~~~~~~~~

docs/source/getting_started.rst

@@ -13,6 +13,13 @@ Check out the :doc:`Tutorial<./tutorial>`!
 
 Developer Setup
 ++++++++++++++++++
+The developer setup can be done via pip or via make. This section will cover how to do both.
+
+Additionally, there is an alternative method to setup merlin on supercomputers. See the :doc:`Spack <./spack>` section for more details.
+
+Pip Setup
+******************
+
 To install with the additional developer dependencies, use::
 
     pip3 install "merlin[dev]"
@@ -21,8 +28,35 @@ or::
 
     pip3 install -e "git+https://github.com/LLNL/merlin.git@develop#egg=merlin[dev]"
 
-See the :doc:`Spack <./spack>` section for an alternative method to setup merlin on supercomputers.
+Make Setup
+*******************
+
+Visit the `Merlin repository <https://github.com/LLNL/merlin/>`_ on github. `Create a fork of the repo <https://docs.github.com/en/get-started/quickstart/fork-a-repo>`_ and `clone it <https://docs.github.com/en/get-started/quickstart/fork-a-repo#cloning-your-forked-repository>`_ onto your system.
+
+Change directories into the merlin repo:
+
+.. code-block:: bash
+
+    $ cd merlin/
+
+Install Merlin with the developer dependencies:
+
+.. code-block:: bash
+
+    $ make install-dev
+
+This will create a virtualenv, start it, and install Merlin and it's dependencies for you.
+
+More documentation about using Virtualenvs with Merlin can be found at
+:doc:`Using Virtualenvs with Merlin <./virtualenv>`.
+
+We can make sure it's installed by running:
+
+.. code-block:: bash
 
+    $ merlin --version
+
+If you don't see a version number, you may need to restart your virtualenv and try again. 
 
 Configuring Merlin
 *******************
@@ -32,6 +66,32 @@ Documentation for merlin configuration is in the :doc:`Configuring Merlin <./mer
 
 That's it. To start running Merlin see the :doc:`Merlin Workflows. <./merlin_workflows>`
 
+(Optional) Testing Merlin
+*************************
+
+.. warning::
+
+    With python 3.6 you may see some tests fail and a unicode error presented. To fix this, you need to reset the LC_ALL environment variable to en_US.utf8.
+
+If you have ``make`` installed and the `Merlin repository <https://github.com/LLNL/merlin/>`_ cloned, you can run the test suite provided in the Makefile by running:
+
+.. code-block:: bash
+
+    $ make tests
+
+This will run both the unit tests suite and the end-to-end tests suite.
+
+If you'd just like to run the unit tests you can run:
+
+.. code-block:: bash
+
+    $ make unit-tests
+
+Similarly, if you'd just like to run the end-to-end tests you can run:
+
+.. code-block:: bash
+
+    $ make e2e-tests
 
 Custom Setup
 +++++++++++++

docs/source/merlin_commands.rst

@@ -71,7 +71,7 @@ If you want to run an example workflow, use Merlin's ``merlin example``:
 
 .. code:: bash
 
-    $ merlin example --help
+    $ merlin example list
 
 This will list the available example workflows and a description for each one. To
 select one:

docs/source/merlin_developer.rst

@@ -43,11 +43,11 @@ To expedite review, please ensure that pull requests
 
 - Are from a meaningful branch name (e.g. ``feature/my_name/cool_thing``)
 
-- Into the `appropriate branch <https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow>`_
+- Are being merged into the `appropriate branch <https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow>`_
 
 - Include testing for any new features
 
-  - unit tests in ``tests/*``
+  - unit tests in ``tests/unit``
   - integration tests in ``tests/integration``
 
 - Include descriptions of the changes
@@ -64,6 +64,8 @@ To expedite review, please ensure that pull requests
   - in ``CHANGELOG.md``
   - in ``merlin.__init__.py``
 
+- Have `squashed <https://github.com/LLNL/merlin/wiki/Squash-commits>`_ commits
+
 Testing
 +++++++
 

docs/source/merlin_server.rst

@@ -62,11 +62,11 @@ merlin server. A detailed list of commands can be found in the `Merlin Server Co
 Note: Running "merlin server init" again will NOT override any exisiting configuration
 that the users might have set or edited. By running this command again any missing files 
 will be created for the users with exisiting defaults. HOWEVER it is highly advised that 
-users back up their configuration in case an error occurs where configuration files are override.
+users back up their configuration in case an error occurs where configuration files are overriden.
 
 .. toctree::
    :maxdepth: 1
    :caption: Merlin Server Settings:
 
    server/configuration
-   server/commands
+   server/commands