1.1.10 Release (#432)

1.1.10 Release (#432)

* Sync up read the docs config with dev environments using poetry (#399)
* Print usage on command line when no args are provided (#404)
* Add sacct fallback to slurm adapter to improve robustness of job tracking (#405)
* Update Flurm Job State mappings for flux versions >= 0.26 (#407)
* Bump certifi from 2021.10.8 to 2022.12.7 to address security issue (#409)
* Bump cryptography from 37.0.1 to 38.0.3 to address security issue (#410)
* Add missing shbang in unscheduled scripts from lsf adapter (#411)
* Update poetry lockfile to address dependabot flagged security issues (#412)
* Fix for Dockerfile smell DL3006 (#418)
* Port Maestro documentation to mkdocs and expand coverage of features and tutorials (#403)
* Update version info to be driven from pyproject.toml exclusively, and hook up to command line (#419)
* Pin mermaid to < 10.x due to api change (#422)
* Bump lock file certifi from 2022.12.7 to 2023.7.22 to address security issue (#426)
* Refactor flux adapter to avoid using pickle to talk to flux brokers installed in external environments (#415)
   Also adds flux integration tests to exercise against real flux brokers
* Add pager functionality to status command (#420)
* Patch broken flux job cancellation (#428)
* Insulate slurm adapters from user customization of squeue and sacct output formats (#431)
   Also adds live unit and integration tests for slurm adapter

---------

Co-authored-by: Francesco Di Natale <frank.dinatale1988@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Bruno P. Kinoshita <kinow@users.noreply.github.com>
Co-authored-by: Charles Doutriaux <doutriaux1@llnl.gov>
Co-authored-by: Giovanni Rosa <grosa23@yahoo.com>
Co-authored-by: Brian Gunnarson <49216024+bgunnar5@users.noreply.github.com>

.github/workflows/main.yml

@@ -24,53 +24,55 @@ jobs:
       run: |
         # stop the build if there are Python syntax errors or undefined names
         flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
-        
+
         # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
         flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
-        
+
   test-install:
     needs: [ linting ]
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ["3.7", "3.8", "3.9", "3.10"]
+        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
     name: Python ${{ matrix.python-version }} with pip
 
     steps:
     - name: Check out
       uses: actions/checkout@v2
-      
+
     - name: Setup Python ${{ matrix.python-version }}
       uses: actions/setup-python@v2
       with:
         python-version: ${{ matrix.python-version }}
         architecture: x64
-        
+
     - name: Test editable install
       run: |
         python -m pip install --upgrade pip
         python -m pip install -e .
-        
+
   tests:
     needs: [ linting ]
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ['3.7', '3.8', '3.9', '3.10']
+        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
     name: Python ${{ matrix.python-version }} with poetry
-    
+
     steps:
     - name: Check out
       uses: actions/checkout@v2
-      
+
     - name: Setup Python ${{ matrix.python-version }}
       uses: actions/setup-python@v2
       with:
         python-version: ${{ matrix.python-version }}
         architecture: x64
-        
-    - name: Install Poetry 
+
+    - name: Install Poetry
       uses: snok/install-poetry@v1
+      with:
+        version: 1.4.2
 
     - name: Install Dependencies using Poetry
       run: poetry install

.readthedocs.yaml

@@ -0,0 +1,21 @@
+version: 2
+
+build:
+  os: ubuntu-20.04
+  tools:
+    python: '3.10'
+  jobs:
+    post_install:
+      - pip install "poetry>=1.4,<1.5"
+      - poetry config virtualenvs.create false
+      - poetry install
+
+mkdocs:
+  # configuration: docs/mkdocs.yml
+  # Temporarily allow warnings
+  fail_on_warning: false
+
+# # Optionally declare the Python requirements required to build your docs
+# python:
+#    install:
+#    - requirements: docs/requirements.txt

Dockerfile

@@ -1,4 +1,4 @@
-FROM ubuntu
+FROM ubuntu:22.04
 LABEL maintainer="Francesco Di Natale dinatale3@llnl.gov"
 
 RUN apt-get update && apt-get install -y \

README.md

@@ -131,9 +131,9 @@ There are many definitions of workflow, so we try to keep it simple and define t
 A set of high level tasks to be executed in some order, with or without dependencies on each other.
 ```
 
-> *PHILOSOPHY*: Maestro believes in the principle of a clearly defined process, specified as a list of tasks, that are self-documenting and clear in their intent.
+We have designed Maestro around the core concept of what we call a "study". A study is defined as a set of steps that are executed (a workflow) over a set of parameters. A study in Maestro's context is analogous to an actual tangible scientific experiment, which has a set of clearly defined and repeatable steps which are repeated over multiple specimen.
 
-Running the `hello_world` study is as simple as...
+Maestro's core tenets are defined as follows:
 
 ##### Repeatability
 
@@ -147,33 +147,7 @@ Studies should be consistently documented and able to be run in a consistent fas
 
 Documentation is important in computational studies as much as it is in physical science. The YAML specification defined by Maestro provides a few required key encouraging human-readable documentation. Even further, the specification itself is a documentation of a complete workflow.
 
-``` yaml
-description:
-    name: hello_planet
-    description: A simple study to say hello to planets (and Pluto)
-
-study:
-    - name: say-hello
-      description: Say hello to a planet!
-      run:
-          cmd: |
-            echo "Hello, $(PLANET)!" > hello_$(PLANET).txt
-
-global.parameters:
-    PLANET:
-        values: [Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus, Neptune, Pluto]
-        label: PLANET.%%
-```
-
-> *PHILOSOPHY*: Maestro believes that a workflow should be easily parameterized with minimal modifications to the core process.
-
-Maestro will automatically expand each parameter into its own isolated workspace, generate a script for each parameter, and automatically monitor execution of each task.
-
-And, running the study is still as simple as:
-
-``` bash
-    maestro run study.yaml
-```
+----------------
 
 ## Setting up your Python Environment