Cloudinit- terraform #432

MalloZup · 2018-09-25T15:33:08Z

Ubuntu 18.04 with cloudinit

TOdos:

after discussion on irc would be nice to have a DISK resize to 200G for ubuntu image.

Requirements of this PR:

we need the 0.5.0 version of terraform-libvirt-provider in sumaform before merging this.

moio · 2018-10-17T08:44:17Z

modules/libvirt/base/main.tf

+# We fetch the latest ubuntu release image from their mirrors
+resource "libvirt_volume" "ubuntu-1804_volume" {
+  name = "${var.name_prefix}ubuntu-1804"
+  source = "https://cloud-images.ubuntu.com/releases/18.04/release/ubuntu-18.04-server-cloudimg-amd64.img"


Why server and not minimal?

http://cloud-images.ubuntu.com/minimal/releases/bionic/release/

i can try that out

That would be awesome 👍

moio · 2018-10-17T08:50:33Z

modules/libvirt/host/cloud_init.cfg

+disable_root: false
+
+# NOTE: This is distro specific.
+apt_update: true


Is this silently ignored on platforms that do not support this flag?

at moment it our SLES images are not using cloudinit, even if we pass it.

To me, i think that when we switch to JEOS/SLES we will need to have 2 different cloudinit files because of the pkg names differs.
Imho this is something that we can postpone and openining an ISSUE as follow-up of this

not only the pkgs name but i think other things differ so we might have 2 files in case or add a different mechanism to create that. But again imho the JEOS SLES is a big step which might provide lot of work for doing that. ( so i would not design for having the support for jeos right now afaik)

moio · 2018-10-17T09:14:18Z

modules/libvirt/host/cloud_init.cfg

+
+
+runcmd:
+ - sed  -i '/PermitRootLogin/s/.*/PermitRootLogin yes/' /etc/ssh/sshd_config 


Isn't this ssh_pwauth + disable_root?

it doesn't work for SLES JEOS.. so we might remove it since is ubuntu specific ( i think should work)

i might also change the name to cloud_init.ubuntu

modules/libvirt/host/main.tf

moio · 2018-10-17T09:25:17Z

salt/default/minimal.sls

@@ -10,7 +10,11 @@ include:
 minimal_package_update:
  pkg.latest:
    - pkgs:
+{%- if grains['os_family'] == 'Debian' %}
+      - salt-common


I have just noticed that salt is a dependency of salt-minion on all RPM distros.

https://build.opensuse.org/package/view_file/openSUSE:Factory/salt/salt.spec?expand=1
https://build.opensuse.org/package/view_file/systemsmanagement:saltstack:products:next/salt/salt.spec?expand=1
https://build.opensuse.org/package/view_file/systemsmanagement:saltstack:products:develop/salt/salt.spec?expand=1

And Ubuntu ones:

https://packages.ubuntu.com/bionic/salt-minion

So I would take the chance to remove this completely and leave only salt-minion

MalloZup · 2018-10-17T10:52:55Z

modules/libvirt/host/main.tf

@@ -23,6 +35,7 @@ resource "libvirt_domain" "domain" {
  count = "${var.count}"
  qemu_agent = true

+  cloudinit = "${libvirt_cloudinit_disk.minimalconf.id}"


@moio this works only for ubuntu if we add a sles client

* module.minssh-sles12sp3.libvirt_domain.domain: Resource 'libvirt_cloudinit_disk.minimalconf' not found for variable 'libvirt_cloudinit_disk.minimalconf.id'

any thoughts how is this condition if exists in HCL ?

this is due because we create it only for ubuntu and it can be that ubuntu hasn't create the disk..

In terraform 0.12 ternary conditions will not evaluate both sides so this becomes a possibility.

For now, I think this will only work if the provider accepts an array, like the network_interface or disk attributes.

on the provider cloudinit is not typelist. https://github.com/dmacvicar/terraform-provider-libvirt/blob/c37707c37dd95b2e941888073192eeb69a24c78f/libvirt/resource_libvirt_domain.go#L98

To me the pragmatic way at this time is that we waste a little space ( creating cloudinit.disk) for each image and for terraform v12 we can change this. ( we open an issue as reminder).

Changing the libvirt-provider for such a feature is not worth and will take more time then expected.. 😄

I think comparing back in the time where we create downladed all qcow for all clients, we can tollorate this and user will even don\t remark that.

Just some numbers here:

-rw-r--r-- 1 qemu qemu 364K Oct 17 17:05 sles12sp3_jeos_cloudinit.iso

slice() or ternary... no if's, it's a declarative language 🙉

I think I will ultimately accept this temporarily (until terraform 0.12, or all images really use cloud-init, whichever comes first).

MalloZup · 2018-10-18T13:38:38Z

I think having some kb more is not a pb @moio

MalloZup · 2018-10-19T06:28:07Z

Ping

moio · 2018-10-19T06:30:19Z

Plan is to have a look today

moio · 2018-10-19T06:30:50Z

Travis is broken BTW, please fix it

MalloZup · 2018-10-19T06:33:53Z

I cannot fix Travis ( and there is no need to fix it, since is more the rieviw at moment important)

MalloZup · 2018-10-19T06:34:47Z

If I fix travis I need to revert , but before we need agreement

Bischoff

Apart from a few nitpicks (see inline), LGTM.

Bischoff · 2018-10-25T13:32:39Z

main.tf.libvirt.example

@@ -46,3 +46,15 @@ module "minion" {
  count = 1
  // see modules/libvirt/minion/variables.tf for possible values
 }
+
+module "min-ubuntu" {


for the other ones, we have the version embedded:

module "min-sles12sp3" { module "min-centos7" {

do the same, for example min-ubuntu1804 ?

Bischoff · 2018-10-25T13:33:52Z

modules/libvirt/base/main.tf

@@ -11,7 +11,7 @@ resource "libvirt_volume" "centos7_volume" {

 resource "libvirt_volume" "opensuse423_volume" {
  name = "${var.name_prefix}opensuse423"
-  source = "https://download.opensuse.org/repositories/systemsmanagement:/sumaform:/images:/libvirt/images/opensuse423.x86_64.qcow2"
+  source = "http://download.opensuse.org/repositories/systemsmanagement:/sumaform:/images:/libvirt/images/opensuse423.x86_64.qcow2"


Why no https ?

If unrelated to ubuntu but needed, maybe in a different PR...

Bischoff · 2018-10-25T13:34:53Z

modules/libvirt/base/variables.tf

@@ -72,6 +72,6 @@ variable "bridge" {

 variable "images" {
  description = "list of images to be uploaded to the libvirt host, leave default for all"
-  default = ["centos7", "opensuse423",  "sles15",  "sles11sp4",  "sles12", "sles12sp1", "sles12sp2", "sles12sp3", "sles-es7"]
+  default = ["centos7", "opensuse423", "ubuntu-1804",  "sles15",  "sles11sp4",  "sles12", "sles12sp1", "sles12sp2", "sles12sp3", "sles-es7"]


Alphabetical order: move it after sles ?

modules/libvirt/host/main.tf

Bischoff · 2018-10-25T13:38:40Z

modules/libvirt/host/main.tf

@@ -23,6 +35,7 @@ resource "libvirt_domain" "domain" {
  count = "${var.count}"
  qemu_agent = true

+  cloudinit = "${libvirt_cloudinit_disk.minimalconf.id}"


slice() or ternary... no if's, it's a declarative language 🙉

Bischoff · 2018-10-25T13:39:26Z

modules/libvirt/host/main.tf

+      type        = "pty"
+      target_type = "virtio"
+      target_port = "1"
+  }


that one in a separate PR if we get it to work... 😆

Just tested on an existing mirror (after the VM has been re-built with this PR code, without changes to the volumes): virsh console works as expected.

moio · 2018-10-29T06:57:56Z

I am not totally happy with this PR, as we currently use different mechanisms for different OSs (custom images without cloud-init and upstream images with cloud-init in libvirt, custom images with cloud-init in openstack, custom images without cloud-init in AWS).

Nevertheless we need this and we have to start somewhere, so I am accepting.

@MalloZup please see the commits I added on top, which among other things address all comments from @Bischoff.

Thanks a lot for getting this off the ground! 🚀

MalloZup force-pushed the ubuntu-cloudinit branch from 803b41c to 3a451ec Compare September 26, 2018 07:02

MalloZup self-assigned this Sep 28, 2018

MalloZup requested a review from moio September 28, 2018 08:19

MalloZup changed the title ~~WIP: Ubuntu cloudinit~~ Cloudinit- terraform Sep 28, 2018

MalloZup changed the title ~~Cloudinit- terraform~~ Cloudinit- terraform Sep 28, 2018

uyuni-project deleted a comment from moio Oct 2, 2018

MalloZup mentioned this pull request Oct 2, 2018

JEOS enablement in sumaform #434

Closed

3 tasks

Bischoff mentioned this pull request Oct 10, 2018

Add support for serial consoles #436

Closed

MalloZup changed the title ~~Cloudinit- terraform~~ Wip: Cloudinit- terraform Oct 15, 2018

MalloZup changed the title ~~Wip: Cloudinit- terraform~~ Cloudinit- terraform Oct 17, 2018

MalloZup requested a review from Bischoff October 17, 2018 07:17

moio reviewed Oct 17, 2018

View reviewed changes

moio suggested changes Oct 17, 2018

View reviewed changes

uyuni-project deleted a comment from moio Oct 17, 2018

MalloZup commented Oct 17, 2018

View reviewed changes

MalloZup mentioned this pull request Oct 19, 2018

FOLLOW-up :resize ubuntu image 200gb with cloudinit #442

Open

Bischoff approved these changes Oct 25, 2018

View reviewed changes

MalloZup added 2 commits October 26, 2018 15:30

add ubuntu1804 volume

fea65c6

Add mininal cloudinit

5115cc8

MalloZup and others added 25 commits October 26, 2018 15:30

add pkg

2104881

add devices

5b95205

PermitRootLogin yes

7d884d5

add cloudinit pkgs

d257cb0

add jeos and locale

0ba0496

add right states

a57b391

fix ubuntu states

0cb0c16

fix openscap ubuntu

5d8b94c

update images to cloud

077999f

add debina hots

0501faa

fix http

ac45e69

fixup

632cd59

restore

f78171e

fixup

3d28804

add minimal image

7f565ab

remove common-pkg

fbffd64

Make cloudinit file ubuntu specific

8deb456

create cloudinit_disk only for ubuntu1804

fcd7b7e

README_ADVANCED: move OS selection here from the basic example

479d6ad

Continue using https

f0e7e59

streamline comments

c66d15c

Keep consistency in image naming

d20ca49

Keep alphabetic order

c37288b

Some light renaming/comment streamlining

fd69078

Indentation correction

ce0d146

moio force-pushed the ubuntu-cloudinit branch from d499a7c to ce0d146 Compare October 29, 2018 06:52

moio approved these changes Oct 29, 2018

View reviewed changes

moio merged commit 50c9a4d into uyuni-project:master Oct 29, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cloudinit- terraform #432

Cloudinit- terraform #432

MalloZup commented Sep 25, 2018 •

edited

Loading

moio Oct 17, 2018

MalloZup Oct 17, 2018

moio Oct 17, 2018

moio Oct 17, 2018

MalloZup Oct 17, 2018 •

edited

Loading

MalloZup Oct 17, 2018 •

edited

Loading

moio Oct 17, 2018

MalloZup Oct 17, 2018

moio Oct 17, 2018

MalloZup Oct 17, 2018

MalloZup Oct 17, 2018 •

edited

Loading

MalloZup Oct 17, 2018

moio Oct 18, 2018

MalloZup Oct 18, 2018

Bischoff Oct 25, 2018

moio Oct 26, 2018

MalloZup commented Oct 18, 2018

MalloZup commented Oct 19, 2018

moio commented Oct 19, 2018

moio commented Oct 19, 2018

MalloZup commented Oct 19, 2018 •

edited

Loading

MalloZup commented Oct 19, 2018

Bischoff left a comment

Bischoff Oct 25, 2018

Bischoff Oct 25, 2018

Bischoff Oct 25, 2018

Bischoff Oct 25, 2018

Bischoff Oct 25, 2018

moio Oct 29, 2018

moio commented Oct 29, 2018



		runcmd:
		- sed -i '/PermitRootLogin/s/.*/PermitRootLogin yes/' /etc/ssh/sshd_config

Cloudinit- terraform #432

Cloudinit- terraform #432

Conversation

MalloZup commented Sep 25, 2018 • edited Loading

TOdos:

Requirements of this PR:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MalloZup Oct 17, 2018 • edited Loading

Choose a reason for hiding this comment

MalloZup Oct 17, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MalloZup Oct 17, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MalloZup commented Oct 18, 2018

MalloZup commented Oct 19, 2018

moio commented Oct 19, 2018

moio commented Oct 19, 2018

MalloZup commented Oct 19, 2018 • edited Loading

MalloZup commented Oct 19, 2018

Bischoff left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

moio commented Oct 29, 2018

MalloZup commented Sep 25, 2018 •

edited

Loading

MalloZup Oct 17, 2018 •

edited

Loading

MalloZup Oct 17, 2018 •

edited

Loading

MalloZup Oct 17, 2018 •

edited

Loading

MalloZup commented Oct 19, 2018 •

edited

Loading