Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Irods and davrods: bigger update #3 #612

Merged
merged 23 commits into from
Sep 2, 2022
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
0128616
iRODS: davrods limit to TLS >= 1.2
scimerman Aug 1, 2022
89347e0
Merge branch 'develop' of https://github.com/rug-cit-hpc/league-of-ro…
scimerman Aug 1, 2022
f73b94f
Merge branch 'develop' of https://github.com/rug-cit-hpc/league-of-ro…
scimerman Aug 5, 2022
80d3b8d
irods & davrods v3: irods updated to v4.3.0, replaced certificate, up…
scimerman Aug 9, 2022
eb44e45
irods: renamed Vault variables, added davrods docker unused image pru…
scimerman Aug 9, 2022
15124ad
irods: typo
scimerman Aug 9, 2022
e570f97
irods: sql database testing fix, unified remote and local calls
scimerman Aug 10, 2022
4e27332
irods: remove unneeded backup of /etc/init.d/irods file
scimerman Aug 10, 2022
4f00427
Merge branch 'develop' into irods_and_davrods
pneerincx Aug 11, 2022
c762dbd
irods: version limitation impovement
scimerman Aug 12, 2022
e39614c
Merge branch 'irods_and_davrods' of https://github.com/scimerman/leag…
scimerman Aug 12, 2022
ee82216
Merge branch 'develop' of https://github.com/rug-cit-hpc/league-of-ro…
scimerman Aug 15, 2022
7f8ae02
Merge branch 'develop' into irods_and_davrods
pneerincx Aug 15, 2022
95a2583
Merge branch 'irods_and_davrods' of https://github.com/scimerman/leag…
scimerman Aug 16, 2022
8df9202
Merge branch 'develop' into irods_and_davrods
pneerincx Aug 16, 2022
cda481a
irods: update packages, python3, psql pgpass ...
scimerman Aug 16, 2022
3f0b32b
Merge branch 'develop' of https://github.com/rug-cit-hpc/league-of-ro…
scimerman Aug 17, 2022
9926fd6
irods: new (internal only) postgres test machine
scimerman Aug 18, 2022
8429d50
Cluster packages: changed to variable that can be overwritten
scimerman Aug 18, 2022
578613b
Merge branch 'develop' of https://github.com/rug-cit-hpc/league-of-ro…
scimerman Aug 29, 2022
3bd3b30
Merge branch 'develop' into irods_and_davrods
pneerincx Sep 1, 2022
6444e77
package version limiting, updated installation
scimerman Sep 2, 2022
84dbe57
Merge branch 'irods_and_davrods' of https://github.com/scimerman/leag…
scimerman Sep 2, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 15 additions & 13 deletions group_vars/irods.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,28 +7,30 @@ firewall_allowed_tcp_ports: # list of open ports on iCAT server
- "5432" # PostgreSQL
- "20000:20199" # irods

ir_version: '-4.2.11*' # if defined (empty): version will be installed (must start with '-' and end with '*')
ir_version: '-4.3.0*' # if defined (empty): version will be installed (must start with '-' and end with '*')
scimerman marked this conversation as resolved.
Show resolved Hide resolved
# first install v4.2.11 and then upgrade to 4.3.0
ir_server_type: 'icat' # iRODS Server Type
ir_client_server_policy: 'CS_NEG_REQUIRE' # communicating using SSL (CS_NEG_REQUIRE) or without (CS_NEG_REFUSE

ir_ssl_certificate_chain_file: 'nemi_irods/localhost_and_chain_umcg-icat01.crt' # iRODS server certificate
ir_ssl_certificate_key_file: 'nemi_irods/localhost-umcg01.key' # iRODS server certificate's key
ir_ssl_dh_params_file: 'dhparams.pem' # DHparam filename

ir_zone: 'nlumcg' # default main iRODS zone name
ir_local_res: 'rootResc' # local iRODS resource
ir_vault_path: '/var/lib/irods/Vault' # default path to store files for local resource
ir_default_res: 'surfObjStore' # default resource iRODS uploads to
ir_service_account: 'irods' # linux account under which iRODS runs
ir_service_account: 'irods' # iRODS linux account under which iRODS runs
ir_service_account_home: '/home/{{ ir_service_account }}' # iRODS service account user's home folder
ir_admin_name: 'rods' # iRODS (and zone) account
ir_admin_pwd: '{{ icatV_admin_pwd }}' # iRODS Vaulte main administrator password
ir_admin_home_path: '/{{ ir_zone }}/home/{{ ir_admin_name }}' # iRODS admin's home path
ir_db_user: '{{ ir_service_account }}' # db Username, usually same as irods_service_account
ir_db_server: '{{ icat_db_server }}' # iRODS Database Server
ir_db_name: 'ICAT' # iRODS Database Name
ir_negotiation_key: '{{ icat_negotiation_key }}'
ir_ctrl_plane_key: '{{ icat_ctrl_plane_key }}'
ir_zone_key: '{{ icat_zone_key }}'
ir_salt: '{{ icat_salt }}' # iRODS salt
ir_db_user: '{{ icatV_db_user }}' # iRODS Vaulted database username
ir_db_pwd: '{{ icatV_db_pwd }}' # iRODS Vaulted database password
ir_salt: '{{ icatV_salt }}' # iRODS Vaulted salt
ir_zone: 'nlumcg' # iRODS default main iRODS zone name
ir_zone_key: '{{ icatV_zone_key }}' # iRODS Vaulted zone key
ir_negotiation_key: '{{ icatV_negotiation_key }}' # iRODS Vaulted negotiation key
ir_ctrl_plane_key: '{{ icatV_ctrl_plane_key }}' # iRODS Vaulted control plane key
ir_local_res: 'rootResc' # iRODS local iRODS resource
ir_vault_path: '/var/lib/irods/Vault' # iRODS default path to store files for local resource
ir_default_res: 'surfObjStore' # iRODS default resource iRODS uploads to

davrods_install: true # to install davrods docker
davrods_docker_folder: 'davrods_docker' # davrods docker folder name, relative to the user home directory
Expand Down
201 changes: 101 additions & 100 deletions group_vars/nibbler_cluster/secrets.yml

Large diffs are not rendered by default.

20 changes: 14 additions & 6 deletions roles/docker/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,26 @@
ansible.builtin.yum:
name:
- docker
- python2-pip
- python3-pip
- docker-compose
state: latest
update_cache: true
become: true

- name: Upgrade pip to latest version that still supports Python 2.7
ansible.builtin.command: pip install pip==20.3.4
# - name: Upgrade pip to latest version
# ansible.builtin.command: pip3 install -U pip
# become: true
- name: Upgrade pip3 to latest version
ansible.builtin.pip:
name: pip
state: latest
executable: pip3
become: true
Comment on lines +12 to 20
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will overwrite the version installed with the RPM using ansible.builtin.yum. Hence this cannot be "frozen" using the Pulp repos and could break other Python3 RPMs. Is this necessary? Or is the python3-pip RPM installed with ansible.builtin.yum good enough?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few other Python packages installed with pip3 and without version control:

  1. Install docker-py (supported by python 3)
  2. Install pyodbc with pip3
    Can those be deployed in a python virtual environment for iRODS as opposed to in the default location affecting all Python code on the server?


- name: Install docker-py (supported by python 2.7)
- name: Install docker-py (supported by python 3)
ansible.builtin.pip:
name: docker==4.4.4
name: docker
executable: pip3
become: true
...

...
14 changes: 14 additions & 0 deletions roles/irods/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,20 @@
- preconfigured irods repository
- and administrative privileges on the machine

# Variable naming

`ir_` are all the variables that are used within this playbook

`icatV_` are the variables that are saved in the **Vault**, and can be accessed when
individual `ir_` variable is mapped to the appropriate `icatV_` variable.

Mapping is done with one of following files:
1. `static_inventory/[clustername].yml`
2. `group/irods.yml`
3. `roles/irods/defaults/main.yml`

When the playbook deployed, the variables are over written, 1. will have priority over 2.

# About the iRODS and PostgreSQL database

Role sets up the iRODS iCAT server and (optionally) a local PostgreSQL database.
Expand Down
Loading