Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: new data transfer servers #451

Merged
merged 37 commits into from
Aug 30, 2021

Conversation

pneerincx
Copy link
Contributor

@pneerincx pneerincx commented Aug 25, 2021

  • New data_transfer server inventory group:
    • Renamed data_staging inventory group to data_transfer group to prevent confusion with the Slurm Data Staging QoS level and corresponding Slurm ds nodes.
    • Added data transfer server to inventory for Nibbler.
    • Removed data transfer server from inventory for Talos.
    • deploy-os_servers.yml: Added code to create related OpenStack components.
    • Added data_transfer inventory group to several single role playbooks.
    • Added data_transfer inventory group to roles/regular_users/tasks/main.yml.
    • Moved filtering of users/groups that should not be allowed access at all or that should receive only data transfer capabilities from ssh-ldap-wrapper in ldap role to sshd_config in sshd role.
    • Updated mount_volume role to create group folders in /groups on data transfer servers.
    • Added mount_volume role to single role playbook for data transfer servers.
    • Updated logins role: Added login check script to create chrooted fake homes for users in the data_transfer_only_group.
    • Added data_transfer_only_group variable for Gearshift, Talos and Nibbler.
    • Updated sshd role to use data_transfer_only_group variable and use ForceCommand directives to allow only sftp on port 22 and rsync on port 443 on data transfer servers.
    • Updated rsyncd role: added separate configs for regular users versus users in the data_transfer_only_group group. Removed module for regular home dirs as they are not needed for data transfers.
  • Collateral fixes / improvements:
    • Fixed comment in dynamic inventory.
    • online_docs role: Replaced very long lists of hostnames in templates with a variable in roles/online_docs/defaults/main.yml.
    • Added "connection: local" to make tasks work with static inventories too.
    • Updated mount_volume role: Added support for multiple volumes as opposed to a single one and removed machine specific logic from role: differences between machines are now completely handled by group_vars.

…em with a variable in roles/online_docs/defaults/main.yml.
…em with a variable in roles/online_docs/defaults/main.yml.
…sed to a single one and removed machine specific logic from role: differences between machines are now completely handled by group_vars.
…t confusion with the Slurm Data Staging QoS level.
@pneerincx pneerincx changed the title Feature: new data transfer/staging servers Feature: new data transfer servers Aug 26, 2021
@scimerman scimerman merged commit 4bab46e into rug-cit-hpc:develop Aug 30, 2021
@pneerincx pneerincx deleted the feature/new_ds_server branch August 30, 2021 07:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants