Cron (slurm group accounts), sql password (bugfix and changed HC sql root password), delete backup older than 7 days #539
Conversation
scimerman
commented
Feb 14, 2022
scimerman
commented
- cron is executed every 30 minutes, it looks for the subfolders of the /mnt/*/group/ folder. It creates only missing slurm accounts. The output is suppressed if no accounts have been added. This prevents users to get the error when they submit their first job.
- slurm database dump was not working correctly due to the offending characters in the password "<>|". I changed the sql root password for the HC and redeployed the slurm. The quote in the playbook should prevent any similar problems in the future.
- Slurm database backup is keeping all the backups, but we wished to keep only 7 backups.
… accounts, HC fix sql passwords and sql root password change
| login_host: 'localhost' | ||
| login_user: 'root' | ||
| login_password: "{{ MYSQL_ROOT_PASSWORD }}" | ||
| login_unix_socket: "/var/lib/mysql/mysql.sock" | ||
| name: "{{ slurm_database_name }}" | ||
| state: 'present' | ||
| no_log: true | ||
| no_log: false |
There was a problem hiding this comment.
This will log also the password, which is Ok for debugging, but a security risk for production.
There was a problem hiding this comment.
Correct - I forgot to change it back.
| @@ -282,4 +282,20 @@ | |||
| when: functional_admin_group is defined and functional_admin_group | length >= 1 | |||
| become: true | |||
|
|
|||
| - name: 'Deploy script to verify and if necessary fix prmissions of the environment managed by the envsync account.' | |||
There was a problem hiding this comment.
Forgot to update name after copy-paste?
| IFS=$'\n' | ||
| for _each_group in "${_accounts_list}"; do | ||
| _output="$(sacctmgr -i create account ${_each_group} descr=scientists org=various parent=users fairshare=parent 2>&1)" | ||
| if [[ "${?}" -ne "1" ]]; then # suppress the normal output of " Nothing new added." with exit code 1 |
There was a problem hiding this comment.
This will also suppress other errors that will result in the same exit status code "1"...
There was a problem hiding this comment.
You are right. I presumed the errors will be returned under another number (255), and not 1 ... will fix it
| # Search for all existing groups inside the '/groups' folder | ||
| # that are inside /mnt/*/ folder on the SAI machines | ||
| for _each_pfs in /mnt/*; do | ||
| cd ${_each_pfs}/groups |
There was a problem hiding this comment.
This could use an
if [[ -e "${_each_pfs}/groups" ]]; then
...
else
continue
fi
check as not all PFS-ses need to have a groups LFS; some may only have an apps or home LFS.
There was a problem hiding this comment.
Will do. I thought we have it needs to be defined everywhere as I checked the vars of FD, HC and GS.
roles/slurm_management/templates/cron_add_missing_slurm_accounts.sh
Outdated
Show resolved
Hide resolved
| _accounts_list="$(find . -maxdepth 1 -mindepth 1 -type d | sed "s/^\.\///g")" | ||
|
|
||
| IFS=$'\n' | ||
| for _each_group in "${_accounts_list}"; do |
There was a problem hiding this comment.
| _accounts_list="$(find . -maxdepth 1 -mindepth 1 -type d | sed "s/^\.\///g")" | |
| IFS=$'\n' | |
| for _each_group in "${_accounts_list}"; do | |
| readarray -t _groups < <(find . -maxdepth 1 -mindepth 1 -type d | sed "s/^\.\///g") | |
| for _each_group in "${_groups[@]}"; do |
There was a problem hiding this comment.
I was unaware of the readarray. Thanks, I already like it.
…ts.sh Co-authored-by: Pieter Neerincx <pieter.neerincx@gmail.com>
