Refactoring to Ansible Collection

[mkbrechtel]

Refactoring of the repo to an Ansible Collection.

We now have three roles that basically do the same that the old playbooks did.

The ansible collection is now on top level of the repo.

Also i provided a playbook to recreates the terraform test environment for fast development iteration.

[mkbrechtel]

I will first proceed on putting the 3 playbooks for ceph, ovn and incus into 3 roles named the same. We might split that up later, but i first want to do the refactoring without changing the original logic.

[mkbrechtel]

Ok, this seems to have done it. The setup now runs on my server and from what i can tell everything works.

I think we still need to do some restructuring, but this brings everything in collection form and preserves the original logic.

This is a good point to merge now so we can iterate upon this.

[stgraber]

Hey, just to let you know that I'm not ignoring this :)

I've looked through it a couple of times so far and got a bit overwhelmed by it and have been crazy busy with other work so haven't had the time to start picking things up from this and merging, sorry!

[mkbrechtel]

Hey, just to let you know that I'm not ignoring this :)

I've looked through it a couple of times so far and got a bit overwhelmed by it and have been crazy busy with other work so haven't had the time to start picking things up from this and merging, sorry!

Thank you! :)

I reviewed your latest changes, i am trying on getting the refactoring-branch to include all those.

[mkbrechtel]

Hi @stgraber, i would suggest us having a little chat and make up a plan how we proceed. This is a big change and since the work is ongoing in the repo it would be complex to track all the changes, since the new structure is very much different from the old one. :)

[stgraber]

@mkbrechtel sure, we can arrange a quick chat. You can use https://calendar.app.google/5FghtnkVaYiLn8JK6 to book 30min with me when you have some time.

Keep in mind that I'm quite a noob when it comes to Ansible and effectively never played with collections, so I don't have much in the way of opinion on the matter other than obviously wanting to keep the current capability of this repository to just set some variables in an inventory, run a playbook and get the whole thing developed.

If we can keep things working that way and make it easier for folks to import that logic, or part of it into their own Ansible environment, that'd be great!

[mttjohnson]

I don't know if it would be possible for me to also attend any discussion on this, but I am also interested in helping here.

If not, it would be very helpful to know what ended up being discussed and any decisions that were determined to get a better sense of the direction and plan for incorporating changes into the repo.

[stgraber]

@mttjohnson we'll definitely want to update the PR/issue with whatever we come up with, but I can also let you know here about the time we end up picking and see if you happen to be available.

Given @mkbrechtel is based in Germany, I suspect it'll end up being a morning thing for us on the other side of the pond :)

[mkbrechtel]

@mttjohnson we'll definitely want to update the PR/issue with whatever we come up with, but I can also let you know here about the time we end up picking and see if you happen to be available.

Given @mkbrechtel is based in Germany, I suspect it'll end up being a morning thing for us on the other side of the pond :)

Sorry, i wasn't able to schedule a meeting since i was ill and have a high workload currently. I scheduled the meeting for Monday 2024-12-09 17:00 CET. See you soon!

[stgraber]

@mttjohnson let me know if you happen to be free at that time (11am eastern), if so I can add you to the call.

[mttjohnson]

@mttjohnson let me know if you happen to be free at that time (11am eastern), if so I can add you to the call.

Yes, I can make that time work and would be excited to be added to the call.

[mkbrechtel]

notes from meeting:

• goals of incus-deploy
  • deploy multiple servers with incus
  • also be able to remove nodes
  • support both deployment models: incus-os distro or deploy incus on your existing machines
  • in the future support incus-os as the base and deploy ceph into incus containers
  • also be able to configure additional stuff like idps
  • optionally integrate monitoring?
  • is the goal to get incus, ceph and ovn up and running or also to provide routines to help with operating the incus cluster?
    • problem @stgraber mentioned: you can't really currently remove the leader of an ovn cluster.
      • there might be a solution but it might be difficult
  • focus for now on setting an incus cluster up
• sidetrack: discussion about incus-os reproducible builds in the future maybe :)
• requirements
• directory structure
  • is it a good desicion to put the terraform files into the test folder?
    • no, terraform should be used in the future to deploy on public cloud like AWS for example, so there is a use case for the terraform files besides testing
    • also we can use terraform to deploy additional services like Keycloak for SSO.
• concerns for continuity
  • should not be a big concern, most users did only deploy and did not
• keep the yaml inventory
  • maybe keep multiple versions of the yaml in different variations
• how to ensure that the setup works correctly, especially when developing?
  • Matt and I ran into this issue for checking if ovn and ceph was setup correctly
  • idea: add nagios-plugin type check scripts that are also used to test if the installation in the CI is correctly working, can then be used for monitoring in production too
• support is planned for Debian, Redhat and Ubuntu
• considerations for merging
  • have larger commits/squash the commits to a bigger one – make it easy for Stephané to review the commits.
  • redo the merge request because too much changed already.
  • leave terraform directory alone / only add a routine to deploy via ansible
• split bare-metal and services terraform directories, because they need to be deployed independently from each other anyhow
• ansible playbooks for different deployment stories like setting up a single machine or a cluster – runs one or more roles
• timeline
  • not so many changes planned from @stgraber
    • only an iscsi feature is planned
  • @mkbrechtel wants to try to get it done in December, but might turn out to happen in January.
  • @mttjohnson wants to get rid of ansible host specific programs, mostly ceph-tools
• certificate generation
  • how to handle the certificate generation? on the host or on the servers?
  • idea: ability to define a certificate handling machine in ansible (localhost or a remote machine)
  • idea: use some credential system like vault (maybe some day ;) )