From 38f314271cbc19a9f5f6df74bf42b9ff028eebdd Mon Sep 17 00:00:00 2001 From: Brett Holman Date: Wed, 6 Dec 2023 11:39:00 -0700 Subject: [PATCH] doc: Set expectations for new datasources (#4670) Some technical and logistical expectations were implied. Be clear about expectations for new datasource inclusions. Also reorganize content for clarity. --- doc/rtd/development/datasource_creation.rst | 137 +++++++++++++++----- 1 file changed, 105 insertions(+), 32 deletions(-) diff --git a/doc/rtd/development/datasource_creation.rst b/doc/rtd/development/datasource_creation.rst index 4b6bda4c0bac..192d7e51678a 100644 --- a/doc/rtd/development/datasource_creation.rst +++ b/doc/rtd/development/datasource_creation.rst @@ -1,27 +1,46 @@ .. _datasource_creation: -Datasource creation -******************* +Adding cloud-init support to your cloud or platform +*************************************************** -There are multiple ways to provide `user data`, `metadata`, and -`vendor data`, and each cloud solution prefers its own way. A datasource -abstract base class defines a single interface to interact with the different -clouds. Each cloud implementation must inherit from this base class to use this -shared functionality and interface. See :file:`cloud-init/sources/__init__.py` -to see this class. +The upstream cloud-init project regularly accepts code contributions for new +platforms that wish to support cloud-init. -If you are interested in adding a new datasource for your cloud platform you -will need to do all of the following: +Ways to add platform support +============================ + +In order to use cloud-init with a new platform, a couple of approaches +are possible: + +1. Provide platform compatibility with one of the existing datasource + definitions, such as DatasourceNoCloud.py. Several platforms, including + `Libvirt`_ and `Proxmox`_ use this approach. +2. Add a new datasource definition to upstream cloud-init. This provides + tighter integration, a better debugging experience, and more control + and flexibility of cloud-init's interaction with the datasource. This + option is more sensible for clouds that have an unique architecture. -Identify a mechanism for positive identification of the platform -================================================================ +Platform requirements +===================== -It is good practice for a cloud platform to positively identify itself to -the guest. This allows the guest to make educated decisions based on the -platform on which it is running. On the x86 and arm64 architectures, many -clouds identify themselves through `DMI`_ data. For example, Oracle's public -cloud provides the string ``'OracleCloud.com'`` in the DMI chassis-asset -field. +There are some technical and logistical requisites that must be met for +cloud-init support. + +Technical Requirements +---------------------- + +Cloud-init requires that a cloud be able to identify itself to cloud-init at +runtime, and that the cloud be able to provide configuration to the +instance. + +A mechanism for self-identification +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Each cloud platform must positively identify itself to the guest. This allows +the guest to make educated decisions based on the platform on which it is +running. On the x86 and arm64 architectures, many clouds identify themselves +through `DMI`_ data. For example, Oracle's public cloud provides the string +``'OracleCloud.com'`` in the DMI chassis-asset field. ``Cloud-init``-enabled images produce a log file with details about the platform. Reading through this log in :file:`/run/cloud-init/ds-identify.log` @@ -30,17 +49,61 @@ If the log is not present, you can generate it by running from source :file:`./tools/ds-identify` or the installed location :file:`/usr/lib/cloud-init/ds-identify`. -The mechanism used to identify the platform will be required for the -``ds-identify`` and datasource module sections below. +The mechanism used to identify the platform will be required for +``ds-identify`` and the datasource module sections below. + +A mechanism for cloud-init to retrieve configuration +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There are a few different methods by which cloud-init may retrieve +cloud-configuration from its platform to configure the instance. Perhaps +the most common method is a webserver providing configuration over a link-local +network. + +Logistical Requirements +----------------------- + +As with any open source project, multiple logistal requirements exist. + +Testing access +^^^^^^^^^^^^^^ + +A platform that isn't available for testing is a platform which cannot be +independently validated. Please provide some means for community members and +upstream developers to test and verify this platform. Code that cannot be used +is not code that can be supported. + +Maintainer support +^^^^^^^^^^^^^^^^^^ + +At times a point of contact is required to answer questions and occasionally +provide testing or maintenance support. Maintainership is relatively informal, +but please note that there is expectation that from time to time upstream may +need to contact a the maintainer with inquiries. Datasources which are believed +to be unmaintained and unused may at some point in the future be considered for +removal. + +Adding cloud-init support +========================= + +There are multiple ways to provide `user data`, `metadata`, and +`vendor data`, and each cloud solution prefers its own way. A datasource +abstract base class defines a single interface to interact with the different +clouds. Each cloud implementation must inherit from this base class to use this +shared functionality and interface. See :file:`cloud-init/sources/__init__.py` +to see this class. + +If you are interested in adding a new datasource for your cloud platform you +will need to do all of the following: Add datasource module cloudinit/sources/DataSource.py -==================================================================== +-------------------------------------------------------------------- We suggest you start by copying one of the simpler datasources such as ``DataSourceHetzner``. Re-run datasource detection ---------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^ While developing a new datasource it may be helpful to manually run datasource detection without rebooting the system. @@ -56,53 +119,63 @@ re-run, then clean up any logs, and finally, re-run ``cloud-init``: sudo cloud-init init Add tests for datasource module -=============================== +------------------------------- Add a new file with some tests for the module to :file:`cloudinit/sources/test_.py`. For example, see :file:`cloudinit/sources/tests/test_oracle.py` Update ``ds-identify`` -====================== +---------------------- In ``systemd`` systems, ``ds-identify`` is used to detect which datasource should be enabled, or if ``cloud-init`` should run at all. You'll need to make changes to :file:`tools/ds-identify`. Add tests for ``ds-identify`` -============================= +----------------------------- Add relevant tests in a new class to :file:`tests/unittests/test_ds_identify.py`. You can use ``TestOracle`` as an example. Add your datasource name to the built-in list of datasources -============================================================ +------------------------------------------------------------ Add your datasource module name to the end of the ``datasource_list`` entry in :file:`cloudinit/settings.py`. Add your cloud platform to apport collection prompts -==================================================== +---------------------------------------------------- Update the list of cloud platforms in :file:`cloudinit/apport.py`. This list will be provided to the user who invokes :command:`ubuntu-bug cloud-init`. Enable datasource by default in Ubuntu packaging branches -========================================================= +--------------------------------------------------------- Ubuntu packaging branches contain a template file, -:file:`debian/cloud-init.templates`, which ultimately sets the default -``datasource_list`` when installed via package. This file needs updating when -the commit gets into a package. +:file:`config/cloud.cfg.tmpl`, which ultimately sets the default +``datasource_list`` that is installed by distros that use the upstream +packaging configuration. Add documentation for your datasource -===================================== +------------------------------------- You should add a new file in :file:`doc/rtd/reference/datasources/.rst` and reference it in :file:`doc/rtd/reference/datasources.rst` +Benefits of including your datasource in upstream cloud-init +============================================================ + +Datasources which are included in upstream cloud-init benefit from ongoing +maintenance, compatibility with the rest of the codebase, and security +fixes by the upstream development team. + + .. _make-mime: https://cloudinit.readthedocs.io/en/latest/explanation/instancedata.html#storage-locations .. _DMI: https://www.dmtf.org/sites/default/files/standards/documents/DSP0005.pdf +.. _Libvirt: https://github.com/virt-manager/virt-manager/blob/main/man/virt-install.rst#--cloud-init +.. _Proxmox: https://pve.proxmox.com/wiki/Cloud-Init_Support