Skip to content
This repository has been archived by the owner on Jun 10, 2019. It is now read-only.

Add LVM as a disk backend #381

Merged
merged 1 commit into from
Jun 8, 2017
Merged

Add LVM as a disk backend #381

merged 1 commit into from
Jun 8, 2017

Conversation

skylt
Copy link
Contributor

@skylt skylt commented May 26, 2017

Enables the use of Logical Volumes as disk backends.
This was needed as we store our VMs through LVM and wanted to build
the image directly onto a lv.

It uses an existing volume group and has no support for creating a new one.
It will not override an existing logical volume and fail gracefully.

The lv is created, activated and then mounted as a loop device.
The boostraping process is then launched on the loop device.
Once the process is completed, the lv is unmounted and desactivated.

The created lv will be deleted should the boostraping process fail.

The lv must be activated before use.

A manifest has been included for testing purposes.

Copy link
Owner

@andsens andsens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, this is awesome. Great adherence to the guidelines and all the right parts of the API are used :-)
I can definitely see this expanding into supporting creation of a VG on the other volume backings at some point, allowing you to boot any machine directly from a LV :-D

I have added some comments that would make the addition a little more robust and expand it so it can support providers other than kvm.

def run(cls, info):
image_path = '/dev/{vg}/{lv}'.format(vg=info.manifest.volume['logicalvolume'].get('vg', None),
lv=info.manifest.volume['logicalvolume'].get('lv', None))
info.volume.create(image_path)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be better to just pass vg and lv directly on to create() and generate the image_path there? Then you don't need to do the path splitting in _before_create().

])
if manifest.volume.get('logicalvolume', []):
taskset.update([logicalvolume.AddRequiredCommands,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of only supporting this on kvm, you could modify volume_group in task_groups.py to be a function and add the commands there. The same would actually work for the loopback tasks as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which providers do you have in mind ?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Come to think of it, there might not be any. This is pretty kvm specific, since the volume is kind of tied to the host machine, correct?
I'm ok with merging this if you squash your commits :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The commits have been squashed.

locale: en_US
timezone: UTC
volume:
backing: lv
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't lvm be more accurate?

timezone: UTC
volume:
backing: lv
logicalvolume:
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there's any need to put vg and lv under another key. Just spell them out directly in volume. So: volume: {logivalvolume: "lvname", volumegroup: "vgname"}

@@ -134,6 +134,7 @@ properties:
type: object
properties:
backing: {type: string}
logicalvolume: {type: object}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For better manifest validation the volume object should be split into a standardvolume definition and logicalvolume definition, like so:

  volume:
    type: object
    oneOf:
      - $ref: '#/definitions/standardvolume'
      - $ref: '#/definitions/logicalvolume'
definitions:
  ...
  standardvolume:
    type: object
    properties:
      backing: {type: string}
      partitions:
        type: object
        oneOf:
          - $ref: '#/definitions/no_partitions'
          - $ref: '#/definitions/partition_table'
    required: [partitions]
    additionalProperties: false
  logicalvolume:
    type: object
    properties:
      backing: lvm
      volumegroup: {type: string}
      logicalvolume: {type: string}
      partitions:
        type: object
        oneOf:
          - $ref: '#/definitions/no_partitions'
          - $ref: '#/definitions/partition_table'
    required: [volumegroup, logicalvolume, partitions]
    additionalProperties: false

@skylt
Copy link
Contributor Author

skylt commented Jun 1, 2017

I have updated the various issues.

Enables the use of Logical Volumes as disk backends.

It uses an existing volume group and has no support for creating a new one.
It will not override an existing logical volume and fail gracefully.

The lv is created, activated and then mounted as a loop device.
The boostraping process is then launched on the loop device.
Once the process is completed, the lv is unmounted and desactivated.

The created lv will be deleted should the boostraping process fail.

The lv must be activated before use.

A manifest has been included for testing purposes.
@andsens andsens merged commit 3e5c94f into andsens:master Jun 8, 2017
@skylt skylt deleted the lvm branch July 6, 2017 23:44
brett-smith added a commit to brett-smith/bootstrap-vz that referenced this pull request Jan 10, 2018
Merge pull request andsens#407 from zmarano/master

Remove the bits that disabled IPv6 for GCE.
pep8: Fix E722 do not use bare except'
ec2: Move cloud-init mount tuning to cloud-init plugin

Fixes andsens#406
Fix bug in Stretch builds. sshd_config no longer contains
PermitRootLogin. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=852781
Merge pull request andsens#372 from zmarano/master

Fix an ordering issue with the expand-root plugin.
Merge pull request andsens#374 from liori/documentation-fixes

fix some typos in documentation
Merge pull request andsens#375 from zmarano/master

Enable SCSI block multiqueue for GCE Stretch images.
Merge pull request andsens#378 from Exy13/loops

Fix loopdevice partitions not being unmapped correctly
Merge pull request andsens#381 from Exy13/lvm

Add LVM as a disk backend
Merge pull request andsens#383 from Exy13/qcow

Add qcow2 as a disk backend for KVM
Merge pull request andsens#384 from zmarano/master

Promote stretch to stable, add buster as testing, jessie to oldstable.
Merge pull request andsens#385 from liori/lone-dpkg

Fix lone minimize_size/dpkg without apt
Merge pull request andsens#386 from liori/minimize-size-bindmount-permissions

minimize_size: make sure the permissions of bindmounts are preserved
Merge pull request andsens#382 from Exy13/existing_commands

Fix unfailing CheckExternalCommands
Make trusted-keys paths in manifest relative to manifest
ansible: Make playbook path relative to manifest
ansible: Check for ansible availability before running
vbox: Make guest additions path relative to manifest
ansible: Change tags/skip-tags to be lists and shorten task
ansible hosts -> groups
ansible: Fix extra_vars so that it actually works

Also remove tmp_cmd var and fix manifest schema
ansible: Remove erroneous whitespace
commands plugin: Run copy files before running commands

This is quite useful in situations where you want to do something
with the assets that you have copied into the image.
One could argue for the opposite case as well, but with the
commands plugin you always have the manual "cp" escape hatch.
Merge pull request andsens#390 from indigoid/readme-escaping-note

brief note about escaping braces in manifests
Merge pull request andsens#391 from NeatNerdPrime/puppet_module_upgrade

Puppet tasks fix commit
Partitions: Fix check for additional partitions on single part.
Merge pull request andsens#392 from gplessis/empty-ami-names

Don’t crash on empty AMI name
Merge pull request andsens#395 from zmarano/master

Add GCE buster builds.
Update Google Compute Engine package and repository structure.
GCE packages are now Debian policy compliant.
Merge pull request andsens#396 from NeatNerdPrime/puppet_update_stretch

Stretch update commit
Update doc
Merge pull request andsens#398 from jmporcelg/issue-397

andsens#397 fix: Installing networking kernel driver using DKMS fails in stretch
Merge pull request andsens#400 from Exy13/doc

Update misleading documentation
Fix tabs vs. spaces in changelog
Merge pull request andsens#414 from CMeza99/netbase

allow networking task when netbase pkg is present
Added security field to packages.
Updated Readme and example manifest for jessie-lvm kmv
Merge pull request andsens#415 from ashegedus/config-security

Added security field to packages.
Merge pull request andsens#417 from ashegedus/password-crypt

Add password-crypt to root password plugin
Merge pull request andsens#422 from CMeza99/fix-intel-url

update intel ixgbevf url
Merge pull request andsens#428 from octivi/systemd_expand_root

Change location of expand-root systemd unit from /lib/... to /etc/...
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants