From f356f9709e231ff4129fbcb6eecb247baefb0ab8 Mon Sep 17 00:00:00 2001 From: Brett Holman Date: Mon, 4 Dec 2023 16:40:23 -0700 Subject: [PATCH] doc: Overhaul debugging documentation (#4578) - New page and content describing debugging for users - New page and content documenting cloud-init's status - New page and content documenting cloud-init's exported errors - New page and content documenting cloud-init's failure states - New page and content documenting how to re-run cloud-init - New content documenting how validate user-data - New content documenting how to use cloud-init with libvirt Documents GH-4500 Fixes GH-4608 --- doc/rtd/development/index.rst | 1 - doc/rtd/explanation/exported_errors.rst | 131 ++++++++++++++ doc/rtd/explanation/failure_states.rst | 78 ++++++++ doc/rtd/explanation/index.rst | 2 + doc/rtd/howto/debug_user_data.rst | 50 ++++-- doc/rtd/howto/debugging.rst | 183 +++++++++++++++++++ doc/rtd/howto/index.rst | 7 +- doc/rtd/howto/predeploy_testing.rst | 141 --------------- doc/rtd/howto/rerun_cloud_init.rst | 97 ++++++++++ doc/rtd/howto/run_cloud_init_locally.rst | 217 +++++++++++++++++++++++ doc/rtd/howto/status.rst | 62 +++++++ doc/rtd/reference/faq.rst | 48 ----- 12 files changed, 806 insertions(+), 211 deletions(-) create mode 100644 doc/rtd/explanation/exported_errors.rst create mode 100644 doc/rtd/explanation/failure_states.rst create mode 100644 doc/rtd/howto/debugging.rst delete mode 100644 doc/rtd/howto/predeploy_testing.rst create mode 100644 doc/rtd/howto/rerun_cloud_init.rst create mode 100644 doc/rtd/howto/run_cloud_init_locally.rst create mode 100644 doc/rtd/howto/status.rst diff --git a/doc/rtd/development/index.rst b/doc/rtd/development/index.rst index c61647a0905..73a90247395 100644 --- a/doc/rtd/development/index.rst +++ b/doc/rtd/development/index.rst @@ -73,7 +73,6 @@ Debugging and reporting ../howto/bugs.rst logging.rst - debugging.rst internal_files.rst ../howto/debugging.rst diff --git a/doc/rtd/explanation/exported_errors.rst b/doc/rtd/explanation/exported_errors.rst new file mode 100644 index 00000000000..5febf99ef1a --- /dev/null +++ b/doc/rtd/explanation/exported_errors.rst @@ -0,0 +1,131 @@ +.. _exported_errors: + +Exported errors +=============== + +Cloud-init makes internal errors available to users for debugging. These +errors map to logged errors and may be useful for understanding what +happens when cloud-init doesn't do what you expect. + +Aggregated errors +----------------- + +When a :ref:`recoverable error` occurs, the internal +cloud-init state information is made visible under a top level aggregate key +``recoverable_errors`` with errors sorted by error level: + +.. code-block:: shell-session + :emphasize-lines: 11-19 + + $ cloud-init status --format json + { + "boot_status_code": "enabled-by-generator", + "config": {...}, + "datasource": "", + "detail": "Cloud-init enabled by systemd cloud-init-generator", + "errors": [], + "extended_status": "degraded done", + "init": {...}, + "last_update": "", + "recoverable_errors": + { + "WARNING": [ + "Failed at merging in cloud config part from p-01: empty cloud config", + "No template found in /etc/cloud/templates for template source.deb822", + "No template found in /etc/cloud/templates for template sources.list", + "No template found, not rendering /etc/apt/soures.list.d/ubuntu.source" + ] + }, + "status": "done" + } + + +Reported recoverable error messages are grouped by the level at which +they are logged. Complete list of levels in order of increasing +criticality: + +.. code-block:: shell-session + + WARNING + DEPRECATED + ERROR + CRITICAL + +Each message has a single level. In cloud-init's :ref:`log files`, +the level at which logs are reported is configurable. These messages are +exported via the ``'recoverable_errors'`` key regardless of which level of +logging is configured. + +Per-stage errors +---------------- + +The keys ``errors`` and ``recoverable_errors`` are also exported for each +stage to allow identifying when recoverable and non-recoverable errors +occurred. + +.. code-block:: shell-session + :emphasize-lines: 4-11,16-21 + + $ cloud-init status --format json + { + "boot_status_code": "enabled-by-generator", + "config": + { + "WARNING": [ + "No template found in /etc/cloud/templates for template source.deb822", + "No template found in /etc/cloud/templates for template sources.list", + "No template found, not rendering /etc/apt/soures.list.d/ubuntu.source" + ] + }, + "datasource": "", + "detail": "Cloud-init enabled by systemd cloud-init-generator", + "errors": [], + "extended_status": "degraded done", + "init": + { + "WARNING": [ + "Failed at merging in cloud config part from p-01: empty cloud config", + ] + }, + "last_update": "", + "recoverable_errors": + { + "WARNING": [ + "Failed at merging in cloud config part from p-01: empty cloud config", + "No template found in /etc/cloud/templates for template source.deb822", + "No template found in /etc/cloud/templates for template sources.list", + "No template found, not rendering /etc/apt/soures.list.d/ubuntu.source" + ] + }, + "status": "done" + } + +.. note:: + + Only completed cloud-init stages are listed in the output of + ``cloud-init status --format json``. + +The JSON representation of cloud-init :ref:`boot stages` +(in run order) is: + +.. code-block:: shell-session + + "init-local" + "init" + "modules-config" + "modules-final" + +Limitations of exported errors +------------------------------ + +- Exported recoverable errors represent logged messages, which are not + guaranteed to be stable between releases. The contents of the + ``'errors'`` and ``'recoverable_errors'`` keys are not guaranteed to have + stable output. +- Exported errors and recoverable errors may occur at different stages + since users may reorder configuration modules to run at different + stages via :file:`cloud.cfg`. + +Where to next? +-------------- +See :ref:`here` for a detailed guide to debugging cloud-init. diff --git a/doc/rtd/explanation/failure_states.rst b/doc/rtd/explanation/failure_states.rst new file mode 100644 index 00000000000..0f1680c52e2 --- /dev/null +++ b/doc/rtd/explanation/failure_states.rst @@ -0,0 +1,78 @@ +.. _failure_states: + +Failure states +============== + +Cloud-init has multiple modes of failure. This page describes these +modes and how to gather information about failures. + +.. _critical_failure: + +Critical failure +---------------- + +Critical failures happens when cloud-init experiences a condition that it +cannot safely handle. When this happens, cloud-init may be unable to complete, +and the instance is likely to be in an unknown broken state. + +Cloud-init experiences critical failure when: + +* there is a major problem with the cloud image that is running cloud-init +* there is a severe bug in cloud-init + +When this happens, error messages will be visible in output of +``cloud-init status --long`` within the ``'error'``. + +The same errors will also be located under the key nested under the +module-level keys that store information related to each +:ref:`stage of cloud-init`: ``init-local``, ``init``, +``modules-config``, ``modules-final``. + +.. _recoverable_failure: + +Recoverable failure +------------------- + +In the case that cloud-init is able to complete yet something went wrong, +cloud-init has experienced a "recoverable failure". When this happens, +the service will return with exit code 2, and error messages will be +visible in the output of ``cloud-init status --long`` under the top +level ``recoverable_errors`` and ``error`` keys. + +To identify which stage an error came from, one can check under the +module-level keys: ``init-local``, ``init``, ``modules-config``, +``modules-final`` for the same error keys. + +See :ref:`this more detailed explanation` for to learn how to +use cloud-init's exported errors. + +Cloud-init error codes +---------------------- + +Cloud-init's ``status`` subcommand is useful for understanding which type of +error cloud-init experienced while running. The return code will be one of the +following: + +.. code-block:: shell-session + + 0 - success + 1 - unrecoverable error + 2 - recoverable error + +If ``cloud-init status`` exits with exit code 1, cloud-init experienced +critical failure and was unable to recover. In this case, something is likely +seriously wrong with the system, or cloud-init has experienced a serious bug. +If you believe that you have experienced a serious bug, please file a +:ref:`bug report`. + +If cloud-init exits with exit code 2, cloud-init was able to complete +gracefully, however something went wrong and the user should investigate. + +See :ref:`this more detailed explanation` for more information +on cloud-init's status. + +Where to next? +-------------- + +See :ref:`our more detailed guide` for a detailed guide to +debugging cloud-init. diff --git a/doc/rtd/explanation/index.rst b/doc/rtd/explanation/index.rst index 754318c11de..0dd248c6db6 100644 --- a/doc/rtd/explanation/index.rst +++ b/doc/rtd/explanation/index.rst @@ -20,3 +20,5 @@ knowledge and become better at using and configuring ``cloud-init``. security.rst analyze.rst kernel-cmdline.rst + failure_states.rst + exported_errors.rst diff --git a/doc/rtd/howto/debug_user_data.rst b/doc/rtd/howto/debug_user_data.rst index a29adca3704..0c555c35b97 100644 --- a/doc/rtd/howto/debug_user_data.rst +++ b/doc/rtd/howto/debug_user_data.rst @@ -1,43 +1,55 @@ -How to debug user data -====================== +.. _check_user_data_cloud_config: -Two of the most common issues with cloud config user data are: +How to validate user data cloud config +====================================== + +The two most common issues with cloud config user data are: 1. Incorrectly formatted YAML -2. The first line does not contain ``#cloud-config`` +2. The first line does not start with ``#cloud-config`` Static user data validation --------------------------- -To verify your cloud config is valid YAML you can use `validate-yaml.py`_. - -To ensure the keys and values in your user data are correct, you can run: +Cloud-init is capable of validating cloud config user data directly from +its datasource (i.e. on a running cloud instance). To do this, you can run: .. code-block:: shell-session sudo cloud-init schema --system --annotate -Or, to test YAML in a file: +Or, to test YAML in a specific file: .. code-block:: shell-session cloud-init schema -c test.yml --annotate -Log analysis ------------- +Example output: -If you can log into your system, the best way to debug your system is to -check the contents of the log files :file:`/var/log/cloud-init.log` and -:file:`/var/log/cloud-init-output.log` for warnings, errors, and -tracebacks. Tracebacks are always reportable bugs. +.. code-block:: shell-session -To report any bugs you find, :ref:`refer to this guide `. + $ cloud-init schema --config-file=test.yaml --annotate + #cloud-config + users: + - name: holmanb # E1,E2,E3 + gecos: Brett Holman + primary_group: holmanb + lock_passwd: false + invalid_key: true -Validation service ------------------- + # Errors: ------------- + # E1: Additional properties are not allowed ('invalid_key' was unexpected) + # E2: {'name': 'holmanb', 'gecos': 'Brett Holman', 'primary_group': 'holmanb', 'lock_passwd': False, 'invalid_key': True} is not of type 'array' + # E3: {'name': 'holmanb', 'gecos': 'Brett Holman', 'primary_group': 'holmanb', 'lock_passwd': False, 'invalid_key': True} is not of type 'string' -Another option to is to use the self-hosted HTTP `validation service`_, -refer to its documentation for more info. +Debugging +--------- + +If your user-data cloud config is correct according to the `cloud-init schema` +command, but you are still having issues, then please refer to our +:ref:`debugging guide`. + +To report any bugs you find, :ref:`refer to this guide `. .. LINKS .. _validate-yaml.py: https://github.com/canonical/cloud-init/blob/main/tools/validate-yaml.py diff --git a/doc/rtd/howto/debugging.rst b/doc/rtd/howto/debugging.rst new file mode 100644 index 00000000000..17cdea9e499 --- /dev/null +++ b/doc/rtd/howto/debugging.rst @@ -0,0 +1,183 @@ +.. _how_to_debug: + +How to debug cloud-init +*********************** + +There are several cloud-init :ref:`failure modes` that one may +need to debug. Debugging is specific to the scenario, but the starting points +are often similar: + +* :ref:`I cannot log in` +* :ref:`Cloud-init did not run` +* :ref:`Cloud-init did the unexpected` +* :ref:`Cloud-init never finished running` + +.. _cannot_log_in: + +I can't log in to my instance +============================= + +One of the more challenging scenarios to debug is when you don't have +shell access to your instance. You have a few options: + +1. Acquire log messages from the serial console and check for any errors. + +2. To access instances without SSH available, create a user with password + access (using the user-data) and log in via the cloud serial port console. + This only works if ``cc_users_groups`` successfully ran. + +3. Try running the same user-data locally, such as in one of the + :ref:`tutorials`. Use LXD or QEMU locally to get a shell or + logs then debug with :ref:`these steps`. + +4. Try copying the image to your local system, mount the filesystem locally + and inspect the image logs for clues. + +.. _did_not_run: + +Cloud-init did not run +====================== + +1. Check the output of ``cloud-init status --long`` + + - what is the value of the ``'extended_status'`` key? + - what is the value of the ``'boot_status_code'`` key? + + See :ref:`our reported status explanation` for more + information on the status. + +2. Check the contents of :file:`/run/cloud-init/ds-identify.log` + + This log file is used by the systemd generator and sets cloud-init + to be enabled or disabled. + +3. Check the status of the services + + .. code-block:: + + systemctl status cloud-init-local.service cloud-init.service\ + cloud-config.service cloud-final.service + + Cloud-init may have started to run, but not completed. This shows how many, + and which, cloud-init stages completed. + +.. _did_not_do_the_thing: + +Cloud-init ran, but didn't do what I want it to +=============================================== + +1. If you are using cloud-init's user data + :ref:`cloud config`, make sure + to :ref:`validate your user data cloud config` + +2. Check for errors in ``cloud-init status --long`` + + - what is the value of the ``'errors'`` key? + - what is the value of the ``'recoverable_errors'`` key? + + See :ref:`our guide on exported errors` for more + information on these exported errors. + +3. For more context on errors, check the logs files: + + - :file:`/var/log/cloud-init.log` + - :file:`/var/log/cloud-init-output.log` + + Identify errors in the logs and the lines preceding these errors. + + Ask yourself: + + - According to the log files, what went wrong? + - How does the cloud-init error relate to the configuration provided + to this instance? + - What does the documentation say about the parts of the configuration that + relate to this error? Did a configuration module fail? + - What :ref:`failure state` is cloud-init in? + + +.. _did_not_finish_running: + +Cloud-init never finished running +================================= + +There are many reasons why cloud-init may fail to complete. Some reasons are +internal to cloud-init, but in other cases, cloud-init failure to +complete may be a symptom of failure in other components of the +system, or the result of a user configuration. + +External reasons +---------------- + +- Failed dependent services in the boot. +- Bugs in the kernel or drivers. +- Bugs in external userspace tools that are called by ``cloud-init``. + +Internal reasons +---------------- + +- A command in ``bootcmd`` or ``runcmd`` that never completes (e.g., running + :command:`cloud-init status --wait` will deadlock). +- Configurations that disable timeouts or set extremely high timeout values. + +To start debugging +------------------ + +1. Check ``dmesg`` for errors: + + .. code-block:: + + dmesg -T | grep -i -e warning -e error -e fatal -e exception + +2. Investigate other systemd services that failed + + .. code-block:: + + systemctl --failed + +3. Check the output of ``cloud-init status --long`` + + - what is the value of the ``'extended_status'`` key? + - what is the value of the ``'boot_status_code'`` key? + + See :ref:`our guide on exported errors` for more + information on these exported errors. + +4. Identify which cloud-init :ref:`boot stage` is currently + running: + + .. code-block:: + + systemctl status cloud-init-local.service cloud-init.service\ + cloud-config.service cloud-final.service + + Cloud-init may have started to run, but not completed. This shows how many, + and which, cloud-init stages completed. + +5. Use the PID of the running service to find all running subprocesses. + Any running process that was spawned by cloud-init may be blocking + cloud-init from continuing. + + .. code-block:: + + pstree + + Ask yourself: + + - Which process is still running? + - Why is this process still running? + - How does this process relate to the configuration that I provided? + +6. For more context on errors, check the logs files: + + - :file:`/var/log/cloud-init.log` + - :file:`/var/log/cloud-init-output.log` + + Identify errors in the logs and the lines preceding these errors. + + Ask yourself: + + - According to the log files, what went wrong? + - How does the cloud-init error relate to the configuration provided to this + instance? + - What does the documentation say about the parts of the configuration that + relate to this error? diff --git a/doc/rtd/howto/index.rst b/doc/rtd/howto/index.rst index f8b522d5a0f..b6811fa3133 100644 --- a/doc/rtd/howto/index.rst +++ b/doc/rtd/howto/index.rst @@ -18,9 +18,12 @@ How do I...? .. toctree:: :maxdepth: 1 - Test cloud-init locally before deploying + Run cloud-init locally before deploying + Re-run cloud-init Change how often a module runs - Debug my user data + Validate my user data + Debug cloud-init + Check the status of cloud-init Report a bug Identify my datasource Disable cloud-init diff --git a/doc/rtd/howto/predeploy_testing.rst b/doc/rtd/howto/predeploy_testing.rst deleted file mode 100644 index aec6e7beb69..00000000000 --- a/doc/rtd/howto/predeploy_testing.rst +++ /dev/null @@ -1,141 +0,0 @@ -.. _predeploy_testing: - -How to test ``cloud-init`` locally before deploying -*************************************************** - -It's very likely that you will want to test ``cloud-init`` locally before -deploying it to the cloud. Fortunately, there are several different virtual -machines (VMs) and container tools that are ideal for this sort of local -testing. - -In this guide, we will show how to use three of the most popular tools: -`Multipass`_, `LXD`_ and `QEMU`_. - -Multipass -========= - -Multipass is a cross-platform tool for launching Ubuntu VMs across Linux, -Windows, and macOS. - -When a user launches a Multipass VM, user data can be passed by adding the -``--cloud-init`` flag and the appropriate YAML file containing the user data: - -.. code-block:: shell-session - - $ multipass launch bionic --name test-vm --cloud-init userdata.yaml - -Multipass will validate the YAML syntax of the cloud-config file before -attempting to start the VM! A nice addition which saves time when you're -experimenting and launching instances with various cloud-configs. - -Multipass *only* supports passing user data, and *only* as YAML cloud-config -files. Passing a script, a MIME archive, or any of the other user data formats -``cloud-init`` supports will result in an error from the YAML syntax validator. - -LXD -=== - -LXD offers a streamlined user experience for using Linux system containers. -With LXD, a user can pass: - -* user data, -* vendor data, -* metadata, and -* network configuration. - -The following command initialises a container with user data: - -.. code-block:: shell-session - - $ lxc init ubuntu-daily:bionic test-container - $ lxc config set test-container user.user-data - < userdata.yaml - $ lxc start test-container - -To avoid the extra commands this can also be done at launch: - -.. code-block:: shell-session - - $ lxc launch ubuntu-daily:bionic test-container --config=user.user-data="$(cat userdata.yaml)" - -Finally, a profile can be set up with the specific data if you need to -launch this multiple times: - -.. code-block:: shell-session - - $ lxc profile create dev-user-data - $ lxc profile set dev-user-data user.user-data - < cloud-init-config.yaml - $ lxc launch ubuntu-daily:bionic test-container -p default -p dev-user-data - -The above examples all show how to pass user data. To pass other types of -configuration data use the config option specified below: - -+----------------+---------------------------+ -| Data | Config option | -+================+===========================+ -| user data | cloud-init.user-data | -+----------------+---------------------------+ -| vendor data | cloud-init.vendor-data | -+----------------+---------------------------+ -| network config | cloud-init.network-config | -+----------------+---------------------------+ - -See the LXD `Instance Configuration`_ docs for more info about configuration -values or the LXD `Custom Network Configuration`_ document for more about -custom network config. - -QEMU -==== - -The :command:`cloud-localds` command from the `cloud-utils`_ package generates -a disk with user-supplied data. The ``NoCloud`` datasouce allows users to -provide their own user data, metadata, or network configuration directly to -an instance without running a network service. This is helpful for launching -local cloud images with QEMU, for example. - -The following is an example of creating the local disk using the -:command:`cloud-localds` command: - -.. code-block:: shell-session - - $ cat >user-data <` +format, you might wish to re-run just a single configuration module. +Cloud-init provides the ability to run a single module in isolation and +separately from boot. This command is: + +.. code-block:: shell-session + + $ sudo cloud-init single --name cc_ssh --frequency always + +Example output: + +.. code-block:: + + ... + Generating public/private ed25519 key pair + ... + +This subcommand is not called by the init system. It can be called manually to +load the configured datasource and run a single cloud-config module once, using +the cached user data and metadata after the instance has booted. + +.. note:: + + Each cloud-config module has a module ``FREQUENCY`` configured: ``PER_INSTANCE``, ``PER_BOOT``, ``PER_ONCE`` or ``PER_ALWAYS``. When a module is run by cloud-init, it stores a semaphore file in :file:`/var/lib/cloud/instance/sem/config_.` which marks when the module last successfully ran. Presence of this semaphore file prevents a module from running again if it has already been run. + +Inspect :file:`cloud-init.log` for output of what operations were performed as +a result. + +.. _partially_rerun_cloud_init: + +How to partially re-run cloud-init +================================== + +If the behavior you are testing runs on every boot, there are a couple +of ways to test this behavior. + +Manually run cloud-init stages +------------------------------ + +Note that during normal boot of cloud-init, the init system runs these +stages at specific points during boot. This means that running the code +manually after booting the system may cause the code to interact with +the system in a different way than it does while it boots. + +.. code-block:: shell-session + + cloud-init init --local + cloud-init init + cloud-init modules --mode=config + cloud-init modules --mode=final + +Reboot the instance +------------------- + +Rebooting the instance will take a little bit longer, however it will +make cloud-init stages run at the correct times during boot, so it will +behave more correctly. + +.. code-block:: shell-session + + reboot -h now diff --git a/doc/rtd/howto/run_cloud_init_locally.rst b/doc/rtd/howto/run_cloud_init_locally.rst new file mode 100644 index 00000000000..0111bc1da42 --- /dev/null +++ b/doc/rtd/howto/run_cloud_init_locally.rst @@ -0,0 +1,217 @@ +.. _run_cloud_init_locally: + +How to run ``cloud-init`` locally +********************************* + +It's very likely that you will want to test ``cloud-init`` locally before +deploying it to the cloud. Fortunately, there are several different virtual +machine (VM) and container tools that are ideal for this sort of local +testing. + +* :ref:`boot cloud-init with QEMU ` +* :ref:`boot cloud-init with LXD ` +* :ref:`boot cloud-init with Libvirt ` +* :ref:`boot cloud-init with Multipass ` + +.. _run_with_qemu: + +QEMU +==== + +`QEMU`_ is a general purpose computer hardware emulator that is capable of +running virtual machines with hardware acceleration as well as emulating the +instruction sets of different architectures than the host that you are +running on. + +The ``NoCloud`` datasource allows users to provide their own user data, +metadata, or network configuration directly to an instance without running a +network service. This is helpful for launching local cloud images with QEMU. + +Create your configuration +------------------------- + +We will leave the :file:`network-config` and :file:`meta-data` files empty, but +populate :file:`user-data` with a cloud-init configuration. You may edit the +:file:`network-config` and :file:`meta-data` files if you have a config to +provide. + +.. code-block:: shell-session + + $ touch network-config + $ touch meta-data + $ cat >user-data <user-data <`. + +Cloud-init status +----------------- + +To simplify this, cloud-init provides a tool, ``cloud-init status`` to +report the current status of cloud-init. + +.. code-block:: shell-session + + $ cloud-init status + "done" + +Cloud-init's extended status +---------------------------- + +Cloud-init is also capable of reporting when cloud-init has not been +able to complete the tasks described in a user configuration. If cloud-init +has experienced issues while running, the extended status will include the word +"degraded" in its status. + +Cloud-init can report its internal state via the ``status --format json`` +subcommand under the ``extended_status`` key. + +.. code-block:: shell-session + :emphasize-lines: 7 + + $ cloud-init status --format json + { + "boot_status_code": "enabled-by-generator", + "datasource": "", + "detail": "Cloud-init enabled by systemd cloud-init-generator", + "errors": [], + "extended_status": "degraded done", + "last_update": "", + "recoverable_errors": {}, + "status": "done" + } + +See the list of all possible reported statuses: + +.. code-block:: shell-session + + "not running" + "running" + "done" + "error" + "degraded done" + "degraded running" + "disabled" + +See :ref:`our explanation of failure states` for more +information. diff --git a/doc/rtd/reference/faq.rst b/doc/rtd/reference/faq.rst index 7f358f6aab8..45ec431d910 100644 --- a/doc/rtd/reference/faq.rst +++ b/doc/rtd/reference/faq.rst @@ -15,54 +15,6 @@ Having trouble? We would like to help! - Find a bug? Check out the :ref:`reporting_bugs` topic to find out how to report one -Why did ``cloud-init`` never complete? -====================================== - -To check if ``cloud-init`` is running still, run: - -.. code-block:: shell-session - - cloud-init status - -To wait for ``cloud-init`` to complete, run: - -.. code-block:: shell-session - - cloud-init status --wait - -There are a number of reasons that ``cloud-init`` might never complete. This -list is not exhaustive, but attempts to enumerate potential causes: - -External reasons ----------------- - -- Failed dependent services in the boot. -- Bugs in the kernel or drivers. -- Bugs in external userspace tools that are called by ``cloud-init``. - -Internal reasons ----------------- - -- A command in ``bootcmd`` or ``runcmd`` that never completes (e.g., running - :command:`cloud-init status --wait` will wait forever on itself and never - complete). -- Non-standard configurations that disable timeouts or set extremely high - values ("never" is used in a loose sense here). - -Failing to complete on ``systemd`` ----------------------------------- - -``Cloud-init`` consists of multiple services on ``systemd``. If a service -that ``cloud-init`` depends on stalls, ``cloud-init`` will not continue. -If reporting a bug related to ``cloud-init`` failing to complete on -``systemd``, please make sure to include the following logs. - -.. code-block:: shell-session - - systemd-analyze critical-chain cloud-init.target - journalctl --boot=-1 - systemctl --failed - ``autoinstall``, ``preruncmd``, ``postruncmd`` ==============================================