Dev almalinux9 (#147)

* use vagrant with almalinux-9 * clean up common rh packages * update vagrant * added supervisor config * update to almalinux * replaced include by include_tasks * added atlas v1 * use geerlinguy postgres role * udpate postgres * update locales * update port * added slurm role * update munge slurm * update slurm * update hostname * update gitignore * update pywps handler * update restart of supervisor * fix restart script * update slurm drmaa version * use munge role * update logrotate * update readme with slurm * using minimal munge role * update albatross * updated config for redhat 8 * update on redhat 7 * update supervisor * added shearwater * update vagrant example with emu * use logratote template instead of role * update logrotate package * update shearwater config * added selinux * update slurm config * added owl * added duck * removed role ssl-certs * addes ssl vars * use https * cleaned up docs --------- Co-authored-by: root <root@wps.local>
bird-house · Apr 15, 2024 · 63a0dd7 · 63a0dd7
1 parent c05e539
commit 63a0dd7
Show file tree

Hide file tree

Showing 33 changed files with 376 additions and 501 deletions.
diff --git a/.gitignore b/.gitignore
@@ -6,10 +6,13 @@ etc/custom*.yml
 roles/andrewrothstein.*
 roles/geerlingguy.*
 roles/ANXS.*
+roles/anxs.*
 roles/ansible-role-supervisor
 roles/jdauphant.ssl-certs
 roles/nginxinc.nginx
 roles/arillso.logrotate
+roles/galaxyproject.*
+roles/grycap.*
 
 # ansible collections
 ansible_collections/

diff --git a/README.rst b/README.rst
@@ -28,9 +28,10 @@ Introduction
 PyWPS Ansible Playbook can completely provision a remote server to run the full stack of PyWPS_, including:
 
 * Conda_ to manage application dependencies.
-* `Nginx <http://wiki.nginx.org/Main>`_ as Web-Server.
-* `Supervisor <http://supervisord.org/>`_ to start/stop and monitor services.
+* Nginx_ as Web-Server.
+* Supervisor_ to start/stop and monitor services.
 * PostgreSQL_ optional database used for job logging.
+* Slurm_ optional workload manager for jobs.
 
 It will install a PyWPS application on a single host.
 Nginx, Supervisor and miniconda are installed on the system.
@@ -44,4 +45,49 @@ See the ``docs`` subdirectory or `readthedocs <http://ansible-wps-playbook.readt
 .. _Ansible: https://www.ansible.com/
 .. _Vagrant: https://www.vagrantup.com/
 .. _Conda: https://conda.io/miniconda.html
+.. _Nginx: https://www.nginx.com/
+.. _Supervisor: http://supervisord.org/
 .. _PostgreSQL: https://www.postgresql.org/
+.. _Slurm: https://slurm.schedmd.com/
+
+Testing with Vagrant
+====================
+
+Use Vagrant to test the installation:
+
+.. code-block:: sh
+
+    vagrant up
+
+Login in to VM:
+
+.. code-block:: sh
+
+    vagrant ssh
+
+Become root:
+
+.. code-block:: sh
+
+    sudo -i 
+
+Install ansible:
+
+.. code-block:: sh
+
+    dnf install epel-release
+    dnf install ansible
+
+
+Change to the /vagrant folder:
+
+.. code-block:: sh
+  
+    cd /vagrant
+
+Run the playbook:
+
+.. code-block:: sh
+
+    cp etc/sample-vagrant.yml custom.yml
+    make play
diff --git a/Vagrantfile b/Vagrantfile
@@ -1,8 +1,8 @@
 # -*- mode: ruby -*-
 # vi: set ft=ruby :
 
-# This guide is optimized for Vagrant 1.7 and above.
-Vagrant.require_version ">= 1.7.0"
+# This guide is optimized for Vagrant 2.4 and above.
+Vagrant.require_version ">= 2.4.0"
 
 Vagrant.configure("2") do |config|
 
@@ -17,22 +17,19 @@ Vagrant.configure("2") do |config|
   config.ssh.private_key_path = '~/.vagrant.d/insecure_private_key'
 
   config.vm.define "wps" do |wps|
-    # wps.vm.box = "bento/ubuntu-18.04"
-    # wps.vm.box = "bento/ubuntu-16.04"
-    # wps.vm.box = "bento/debian-9"
-    wps.vm.box = "bento/centos-7"
-    # wps.vm.box = "bento/centos-6"
-    # wps.vm.box = "bento/fedora-27"
+    # wps.vm.box = "bento/centos-7"
+    # wps.vm.box = "bento/almalinux-8"
+    wps.vm.box = "bento/almalinux-9"
     wps.vm.hostname = "wps.local"
     wps.vm.network "private_network", ip: "192.168.128.100"
-    wps.vm.provision 'ansible' do |ansible|
-      ansible.playbook = 'playbook.yml'
-      ansible.verbose = "v"
-      ansible.host_key_checking = false
-      ansible.groups = {
-        "web" => ["wps"],
-        "worker"  => []
-      }
-    end
+    # wps.vm.provision 'ansible' do |ansible|
+    #   ansible.playbook = 'playbook.yml'
+    #   ansible.verbose = "v"
+    #   ansible.host_key_checking = false
+    #   ansible.groups = {
+    #     "web" => ["wps"],
+    #     "worker"  => []
+    #   }
+    # end
   end
 end
diff --git a/docs/source/appendix.rst b/docs/source/appendix.rst
diff --git a/docs/source/changes.rst b/docs/source/changes.rst
diff --git a/docs/source/deploy.rst b/docs/source/deploy.rst
@@ -7,7 +7,7 @@ Deploy a PyWPS Application
 
 .. note::
 
-    You can safely try the installation using Vagrant_ or Docker. See :ref:`testing`.
+    You can safely try the installation using Vagrant_.
 
 Prepare
 -------
@@ -19,7 +19,6 @@ You need a recent Ansible_ version (`>=2.7`) on your local client:
   $ ansible --version
   ansible 2.7.2
 
-If you don't have Ansible installed, then follow these instructions: :ref:`requirements`.
 
 Get the Playbook
 ----------------
@@ -53,7 +52,7 @@ Running your playbook locally
   Make sure your Ansible directory is not world-readable, otherwise the `ansible.cfg` file will not be read.
   See `Ansible Documentation <https://docs.ansible.com/ansible/devel/reference_appendices/config.html#cfg-in-world-writable-dir>`_.
 
-If not already done (see :ref:`requirements`), fetch required roles/recipes from `ansible-galaxy`:
+If not already done, fetch required roles/recipes from `ansible-galaxy`:
 
 .. code-block:: console
 
@@ -68,5 +67,3 @@ Run your playbook locally:
 .. note:: You can also use the shortcut to run both::
 
     $ make play
-
-For remote deployment follow the instructions in the next section :ref:`live`.
diff --git a/docs/source/examples.rst b/docs/source/examples.rst
@@ -48,81 +48,3 @@ Run "hello" in sync mode:
 http://localhost:5000/wps?service=WPS&version=1.0.0&request=Execute&identifier=hello&DataInputs=name=Stranger
 
 
-Install Slurm cluster
----------------------
-
-Use this slurm playbook:
-
-https://github.com/roocs/slurm-playbook
-
-Run playbook with Vagrant::
-
-  $ vagrant up
-  $ ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -u vagrant --private-key=~/.vagrant.d/insecure_private_key -i inventories/vagrant-cluster.yml playbook.yml
-
-
-Login to slurm master node::
-
-  $ vagrant ssh slurmmaster
-
-Run slurm test::
-
-  > sudo -i # become root
-  > squeue # to view the queue
-  > sbatch /root/hostname.sh # to run a job
-  > squeue # to see if it is running
-
-
-Install Emu/PyWPS on cluster
-----------------------------
-
-Use the WPS playbook:
-
-https://github.com/bird-house/ansible-wps-playbook
-
-Install PyWPS into the same cluster as slurm::
-
-  $ cp etc/vagrant-cluster.yml custom.yml
-  $ ansible-playbook -u vagrant --private-key=~/.vagrant.d/insecure_private_key -i inventory/vagrant-cluster.yml playbook.yml
-
-This example is using the Emu WPS with simple test processes:
-
-https://github.com/bird-house/emu
-
-Test Emu WPS service
-++++++++++++++++++++
-
-Test connection:
-
-http://192.168.50.44:5000/wps?service=WPS&version=1.0.0&request=GetCapabilities
-
-Run "sleep" in async mode with scheduler:
-
-http://192.168.50.44:5000/wps?service=WPS&version=1.0.0&request=Execute&identifier=sleep&storeExecuteResponse=true&status=true&DataInputs=delay=2
-
-Install the rook/PyWPS for subsetting on cluster
-------------------------------------------------
-
-This example is using the rook WPS with subsetting processes on climate model data:
-
-https://github.com/roocs/rook
-
-Installation is like before with slurm cluster but using a different config file::
-
-  $ cp etc/vagrant-cluster-with-rook.yml custom.yml
-  $ ansible-playbook -u vagrant --private-key=~/.vagrant.d/insecure_private_key -i inventory/vagrant-cluster.yml playbook.yml
-
-In this example demo data is installed in a shared Vagrant folder ``.local/data``:
-
-https://github.com/roocs/mini-esgf-data
-
-Test rook WPS service
-+++++++++++++++++++++
-
-Test connection:
-
-http://192.168.50.44:5000/wps?service=WPS&version=1.0.0&request=GetCapabilities
-
-Run "subset" in async mode with scheduler and default values:
-
-http://192.168.50.44:5000/wps?service=WPS&version=1.0.0&request=Execute&identifier=subset&storeExecuteResponse=true&status=true
diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -6,11 +6,6 @@ Getting started
 .. toctree::
    :maxdepth: 2
 
-   preparing
    deploy
-   live
    configuration
-   testing
    examples
-   changes
-   appendix
diff --git a/docs/source/live.rst b/docs/source/live.rst