Update Develop-Ref after #1649 (#1651)

Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com>
Co-authored-by: Julie Prestopnik <jpresto@seneca.rap.ucar.edu>
Co-authored-by: TaraJensen <jensen@ucar.edu>
Co-authored-by: Mrinal Biswas <biswas@ucar.edu>
Co-authored-by: Hank Fisher <fisherh@kiowa.rap.ucar.edu>
Co-authored-by: Minna Win <minnawin@kiowa.rap.ucar.edu>
Co-authored-by: bikegeek <3753118+bikegeek@users.noreply.github.com>
Co-authored-by: Christina Kalb <kalb@kiowa.rap.ucar.edu>
Co-authored-by: johnhg <johnhg@ucar.edu>
Co-authored-by: Hank Fisher <fisherh@seneca.rap.ucar.edu>
Co-authored-by: Mrinal Biswas <biswas@seneca.rap.ucar.edu>
Co-authored-by: jprestop <jpresto@ucar.edu>
Co-authored-by: j-opatz <jopatz@ucar.edu>
Co-authored-by: Kathryn Newman <knewman@seneca.rap.ucar.edu>
Co-authored-by: j-opatz <59586397+j-opatz@users.noreply.github.com>
Co-authored-by: Lisa Goodrich <lisag@ucar.edu>
Co-authored-by: Julie Prestopnik <jpresto@ucar.edu>
Co-authored-by: Christina Kalb <kalb@mohawk.rap.ucar.edu>
Co-authored-by: Christina Kalb <kalb@ucar.edu>
Co-authored-by: lisagoodrich <33230218+lisagoodrich@users.noreply.github.com>
Co-authored-by: bikegeek <minnawin@ucar.edu>
Co-authored-by: Hank Fisher <fisherh@ucar.edu>
Co-authored-by: Howard Soh <hsoh@seneca.rap.ucar.edu>
Co-authored-by: Molly Smith <molly.b.smith@noaa.gov>
Co-authored-by: hsoh-u <hsoh@ucar.edu>

.github/ISSUE_TEMPLATE/bug_report.md

@@ -70,15 +70,15 @@ Branch name: `bugfix_<Issue Number>_main_<Version>_<Description>`
 - [ ] Submit a pull request to merge into **main_\<Version>**.
 Pull request: `bugfix <Issue Number> main_<Version> <Description>`
 - [ ] Define the pull request metadata, as permissions allow.
-Select: **Reviewer(s)** and **Linked issues**
+Select: **Reviewer(s)** and **Development** issues
 Select: **Organization** level software support **Project** for the current coordinated release
 Select: **Milestone** as the next bugfix version
 - [ ] Iterate until the reviewer(s) accept and merge your changes.
 - [ ] Delete your fork or branch.
 - [ ] Complete the steps above to fix the bug on the **develop** branch.
 Branch name:  `bugfix_<Issue Number>_develop_<Description>`
 Pull request: `bugfix <Issue Number> develop <Description>`
-Select: **Reviewer(s)** and **Linked issues**
+Select: **Reviewer(s)** and **Development** issues
 Select: **Repository** level development cycle **Project** for the next official release
 Select: **Milestone** as the next official version
 - [ ] Close this issue.

.github/ISSUE_TEMPLATE/enhancement_request.md

@@ -59,7 +59,7 @@ Branch name: `feature_<Issue Number>_<Description>`
 - [ ] Submit a pull request to merge into **develop**.
 Pull request: `feature <Issue Number> <Description>`
 - [ ] Define the pull request metadata, as permissions allow.
-Select: **Reviewer(s)** and **Linked issues**
+Select: **Reviewer(s)** and **Development** issues
 Select: **Repository** level development cycle **Project** for the next official release
 Select: **Milestone** as the next official version
 - [ ] Iterate until the reviewer(s) accept and merge your changes.

.github/ISSUE_TEMPLATE/new_feature_request.md

@@ -63,7 +63,7 @@ Branch name: `feature_<Issue Number>_<Description>`
 - [ ] Submit a pull request to merge into **develop**.
 Pull request: `feature <Issue Number> <Description>`
 - [ ] Define the pull request metadata, as permissions allow.
-Select: **Reviewer(s)** and **Linked issues**
+Select: **Reviewer(s)** and **Development** issues
 Select: **Repository** level development cycle **Project** for the next official release
 Select: **Milestone** as the next official version
 - [ ] Iterate until the reviewer(s) accept and merge your changes.

.github/ISSUE_TEMPLATE/new_use_case.md

@@ -72,7 +72,7 @@ Branch name: `feature_<Issue Number>_<Description>`
 - [ ] Submit a pull request to merge into **develop**.
 Pull request: `feature <Issue Number> <Description>`
 - [ ] Define the pull request metadata, as permissions allow.
-Select: **Reviewer(s)** and **Linked issues**
+Select: **Reviewer(s)** and **Development** issues
 Select: **Repository** level development cycle **Project** for the next official release
 Select: **Milestone** as the next official version
 - [ ] Iterate until the reviewer(s) accept your changes. Merge branch into **develop**.

.github/ISSUE_TEMPLATE/task.md

@@ -59,7 +59,7 @@ Branch name: `feature_<Issue Number>_<Description>`
 - [ ] Submit a pull request to merge into **develop**.
 Pull request: `feature <Issue Number> <Description>`
 - [ ] Define the pull request metadata, as permissions allow.
-Select: **Reviewer(s)** and **Linked issues**
+Select: **Reviewer(s)** and **Development** issues
 Select: **Repository** level development cycle **Project** for the next official release
 Select: **Milestone** as the next official version
 - [ ] Iterate until the reviewer(s) accept and merge your changes.

.github/actions/run_tests/Dockerfile

@@ -5,6 +5,6 @@ RUN chmod +x /entrypoint.sh
 
 RUN apk add --update --no-cache docker python3 py3-pip bash
 
-RUN pip3 install python-dateutil
+RUN pip3 install python-dateutil requests
 
 ENTRYPOINT ["/entrypoint.sh"]

.github/parm/use_case_groups.json

@@ -49,6 +49,11 @@
     "index_list": "8",
     "run": false
   },
+  {
+    "category": "convection_allowing_models",
+    "index_list": "9",
+    "run": false
+  },
   {
     "category": "data_assimilation",
     "index_list": "0",

.github/pull_request_template.md

@@ -23,6 +23,6 @@ See the [METplus Workflow](https://metplus.readthedocs.io/en/latest/Contributors
 Select: **Reviewer(s)**
 Select: **Organization** level software support **Project** or **Repository** level development cycle **Project**
 Select: **Milestone** as the version that will include these changes
-- [ ] After submitting the PR, select **Linked issues** with the original issue number.
+- [ ] After submitting the PR, select **Development** issue with the original issue number.
 - [ ] After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
 - [ ] Close the linked issue and delete your feature or bugfix branch from GitHub.

docs/Contributors_Guide/conda_env.rst

@@ -189,3 +189,68 @@ The results are sent to stdout (screen) and three files are created:
      * A list of Python packages on the host system but with different
        version than what is used by METplus
 
+
+
+Link Conda Directory to Data Disk (RAL Linux Machines)
+______________________________________________________
+
+By default, Conda environments are stored in a directory called ".conda" that is found in the user's home directory, i.e. /home/user/.conda (Note that the dot at the beginning of the directory name is a hidden directory that does not always show up in a directory listing). Conda environments can take up a lot of disk space which can quickly fill up the /home disk. It is recommended that you create a directory on a data disk that has more disk space and create a symbolic link from the .conda directory so the environments will be stored on the data disk. Keep in mind that deleting the directory on the data disk will delete all of your conda environments and they cannot be easily recovered.
+
+For all of the following commands, replace "user" with username.
+
+1.  Check if the .conda directory already exists.
+
+    Run the following command:
+
+    .. code-block:: ini
+
+      ls -al ~/ | grep .conda
+
+    If nothing is output to the screen, then the directory does not exist and can proceed to step #3.
+    If the screen output contains .conda followed by an arrow (->) followed by the path to a directory on a data disk, then the link has already been set up properly and no further action is needed. Example:
+
+    .. code-block:: ini
+
+      .conda -> /d1/personal/user/conda
+
+    If there is no arrow, then the contents of this directory need to be moved to the data disk.
+
+2.  Copy Existing Conda files to Data Disk.
+
+    Determine the data disk location where personal files are stored. Examples:
+
+    /d1/personal/user
+    /d1/user
+
+    Create a directory called conda in data directory. The name of this directory can be something else if desired. Modify the following commands as needed. Example:
+
+    .. code-block:: ini
+
+      mkdir -p /d1/personal/$USER/conda
+
+    Copy the contents of .conda in the home directory to this new directory:
+
+    .. code-block:: ini
+
+      cp -r ~/.conda/* /d1/personal/$USER/conda/
+
+    Now remove .conda from the home directory:
+
+    .. code-block:: ini
+
+      rmdir ~/.conda
+
+3.  Create a symbolic link from .conda in the home directory to the directory that was just created.
+
+    Example:
+
+    .. code-block:: ini
+
+      ln -s /d1/personal/$USER/conda ~/.conda
+
+4. Run the ls command again to confirm that there is now an arrow pointing to the new directory on the data disk.
+
+   .. code-block:: ini
+
+     ls -al ~/ | grep .conda
+

docs/Release_Guide/metexpress_official.rst

@@ -11,7 +11,9 @@ Create a new vX.Y.Z official release from the develop branch.
 .. include:: release_steps/metexpress/update_dates_and_versions.rst
 .. include:: release_steps/metexpress/checkout_main_branch.rst
 .. include:: release_steps/metexpress/merge_development_to_main.rst
-.. include:: release_steps/metexpress/roll_app_versions_and_build.rst
+.. include:: release_steps/metexpress/tag_app_version_and_build.rst
+.. include:: release_steps/metexpress/deploy_beta_apps.rst
+.. include:: release_steps/metexpress/deploy_production_apps.rst
 .. include:: release_steps/metexpress/deploy_AWS_apps.rst
 .. include:: release_steps/metexpress/pull_changes_and_merge_to_dev.rst
 .. include:: release_steps/update_docs_official.rst

docs/Release_Guide/release_steps/common/update_dtc_website.rst

@@ -54,7 +54,29 @@ Update DTC Website
 
     * For an official release, remove the corresponding development
       releases.
+
+  * Create or edit the "Coordinated METplus Version X.Y" software release.
+
+    * For a beta1 development release, create the "Coordinated METplus
+      Version X.Y" release entry if it doesn't already exist.  Ensure the
+      "Release Type" is set to *Development*. Add links for the |projectRepo|
+      X.Y.Z beta1, the METplus "Documentation", the METplus "Existing Builds
+      and Docker" page, and the "Release Notes". Add to the Release Notes a link to the
+      |projectRepo| Release Notes.
 
+    * For a beta2 or beyond development release, update the existing link and text in
+      the "Coordinated METplus Version X.Y" release section with the updated
+      development release information.
+
+    * For an official release, update the existing link and text in
+      the "Coordinated METplus Version X.Y" release section with the X.Y.Z
+      information. Change the "Release Type" from *Development* to
+      *Recommended*. Consider changing the "Release Type" of previous coordinated releases from *Recommended* to *Other*.
+
+    * For a bugfix release, update the existing link and text in
+      the "Coordinated METplus Version X.Y" release section with the
+      X.Y.Z+1 information.
+
   * |otherWebsiteUpdates|
 
   * Click on "Save".

docs/Release_Guide/release_steps/metexpress/deploy_AWS_apps.rst

@@ -1,7 +1,7 @@
 Deploy AWS apps
 -------------------
 
-* On metexpress.nws.noaa.gov (permissions required), go to the container_deployment directory.
+* On metexpress.nws.noaa.gov (VPN and permissions required), go to the container_deployment directory.
 
 .. parsed-literal::
 
@@ -21,6 +21,6 @@ Deploy AWS apps
 
 .. parsed-literal::
 
-  bin/down metexpress
-  docker system prune -af
-  bin/up metexpress
+  bin/redeploy metexpress
+
+* Sometimes the apps take a while to start up on AWS. Be patient.

docs/Release_Guide/release_steps/metexpress/deploy_beta_apps.rst

@@ -0,0 +1,27 @@
+Deploy Beta Apps
+-----------------
+
+* After the apps build, which usually takes about 20 minutes, they should be installed on the dedicated beta server (mats-docker-int) for third party testing.
+
+* On mats-docker-int.gsd.esrl.noaa.gov (the beta server, VPN and permissions required), go to the container_deployment directory.
+
+.. parsed-literal::
+
+    ssh www-data@mats-docker-int.gsd.esrl.noaa.gov
+    cd container_deployment
+
+* Add any new apps to docker_compose.yml and web/applist.json. Update the container version numbers in docker_compose.yml, and any settings files in settings/
+
+.. parsed-literal::
+
+    vi docker_compose.yml
+    vi web/applist.json
+    vi settings/appname/settings.json
+
+* Redeploy the apps.
+
+.. parsed-literal::
+
+    bin/redeploy production
+
+* Contact your dedicated beta tester to ensure the apps work as expected.

docs/Release_Guide/release_steps/metexpress/deploy_production_apps.rst

@@ -0,0 +1,25 @@
+Deploy Production Apps
+-----------------------
+
+* If the beta tester approves the apps, deploy them to the production server (mats-docker).
+
+* On mats-docker.gsd.esrl.noaa.gov (the production server, VPN and permissions required), go to the container_deployment directory.
+
+.. parsed-literal::
+
+  ssh www-data@mats-docker.gsd.esrl.noaa.gov
+  cd container_deployment
+
+* Add any new apps to docker_compose.yml and web/applist.json. Update the container version numbers in docker_compose.yml, and any settings files in settings/
+
+.. parsed-literal::
+
+  vi docker_compose.yml
+  vi web/applist.json
+  vi settings/appname/settings.json
+
+* Redeploy the apps.
+
+.. parsed-literal::
+
+  bin/redeploy production

docs/Release_Guide/release_steps/metexpress/pull_changes_and_merge_to_dev.rst

@@ -14,21 +14,21 @@ Pull Changes, Create Release Branch, And Merge To Development
     git pull
     cd ../..
 
-* Create a release branch of the format main_vX.X.X. using your release version.
+* Create a release branch of the format main_vX.Y.Z. using your release version.
 
 .. parsed-literal::
 
-    git checkout -b main_vX.X.X
-    git push -u origin main_vX.X.X
+    git checkout -b main_vX.Y.Z
+    git push -u origin main_vX.Y.Z
     cd MATScommon
-    git checkout -b main_vX.X.X
-    git push -u origin main_vX.X.X
+    git checkout -b main_vX.Y.Z
+    git push -u origin main_vX.Y.Z
     cd ../METexpress
-    git checkout -b main_vX.X.X
-    git push -u origin main_vX.X.X
+    git checkout -b main_vX.Y.Z
+    git push -u origin main_vX.Y.Z
     cd MATScommon
-    git checkout -b main_vX.X.X
-    git push -u origin main_vX.X.X
+    git checkout -b main_vX.Y.Z
+    git push -u origin main_vX.Y.Z
     cd ../..
 
 * Checkout development and merge any build-related changes.