Cloud-init status always reporting running #5304
Comments
Overmorrow1501
added
bug
|
Thank you for the bug report, but I am unable to reproduce this with the information given. Included in your cloud-init logs are installer logs. To reproduce this, are you running the Ubuntu installer or cloud-init alone? Both your logs and the status output that you've pasted show that only the |
TheRealFalcon
added
incomplete
|
Thanks for the reply! I'm creating a VM from a VMware template on which before the last shutdown the command What's really strange is that each time I use this template to deploy 2 or more new VM at the same run (using Ansible with I've gathered a little more details from one of the VMs which has this problem
$ systemctl | grep "cloud-"
cloud-config.service loaded active exited Apply the settings specified in cloud-config
cloud-final.service loaded active exited Execute cloud user/final scripts
cloud-init-hotplugd.socket loaded active listening cloud-init hotplug hook socket
cloud-config.target loaded active active Cloud-config availability
cloud-init.target loaded active active Cloud-init target
$ cloud-init status --long
status: running
extended_status: running
boot_status_code: enabled-by-generator
last_update: Mon, 20 May 2024 20:17:51 +0000
detail:
Cloud-init enabled by systemd cloud-init-generator
errors: []
recoverable_errors: {}
for I in `systemctl | grep "cloud-" | awk '{print $1}'`
> do
> systemctl status $I
> done
● cloud-config.service - Apply the settings specified in cloud-config
Loaded: loaded (/lib/systemd/system/cloud-config.service; enabled; vendor preset: enabled)
Active: active (exited) since Mon 2024-05-20 22:17:51 CEST; 4min 37s ago
Process: 717 ExecStart=/usr/bin/cloud-init modules --mode=config (code=exited, status=0/SUCCESS)
Main PID: 717 (code=exited, status=0/SUCCESS)
Tasks: 0 (limit: 11751)
Memory: 6.5M
CPU: 243ms
CGroup: /system.slice/cloud-config.service
May 20 22:17:50 oc-ubu-dev02 systemd[1]: Starting Apply the settings specified in cloud-config...
May 20 22:17:51 oc-ubu-dev02 cloud-init[922]: Cloud-init v. 23.4.4-0ubuntu0~22.04.1 running 'modules:config' at Mon, 20 May 2024 20>
May 20 22:17:51 oc-ubu-dev02 cloud-init[922]: No 'config' modules to run under section 'cloud_config_modules'
May 20 22:17:51 oc-ubu-dev02 systemd[1]: Finished Apply the settings specified in cloud-config.
● cloud-final.service - Execute cloud user/final scripts
Loaded: loaded (/lib/systemd/system/cloud-final.service; enabled; vendor preset: enabled)
Active: active (exited) since Mon 2024-05-20 22:17:51 CEST; 4min 41s ago
Process: 924 ExecStart=/usr/bin/cloud-init modules --mode=final (code=exited, status=0/SUCCESS)
Main PID: 924 (code=exited, status=0/SUCCESS)
Tasks: 0
Memory: 40.0K
CPU: 237ms
CGroup: /system.slice/cloud-final.service
May 20 22:17:51 oc-ubu-dev02 systemd[1]: Starting Execute cloud user/final scripts...
May 20 22:17:51 oc-ubu-dev02 cloud-init[930]: Cloud-init v. 23.4.4-0ubuntu0~22.04.1 running 'modules:final' at Mon, 20 May 2024 20:>
May 20 22:17:51 oc-ubu-dev02 cloud-init[930]: No 'final' modules to run under section 'cloud_final_modules'
May 20 22:17:51 oc-ubu-dev02 systemd[1]: Finished Execute cloud user/final scripts.
● cloud-init-hotplugd.socket - cloud-init hotplug hook socket
Loaded: loaded (/lib/systemd/system/cloud-init-hotplugd.socket; enabled; vendor preset: enabled)
Active: active (listening) since Mon 2024-05-20 22:17:50 CEST; 4min 43s ago
Triggers: ● cloud-init-hotplugd.service
Listen: /run/cloud-init/hook-hotplug-cmd (FIFO)
CGroup: /system.slice/cloud-init-hotplugd.socket
May 20 22:17:50 oc-ubu-dev02 systemd[1]: Listening on cloud-init hotplug hook socket.
● cloud-config.target - Cloud-config availability
Loaded: loaded (/lib/systemd/system/cloud-config.target; static)
Active: active since Mon 2024-05-20 22:17:50 CEST; 4min 43s ago
Notice: journal has been rotated since unit was started, output may be incomplete.
● cloud-init.target - Cloud-init target
Loaded: loaded (/lib/systemd/system/cloud-init.target; enabled-runtime; vendor preset: enabled)
Active: active since Mon 2024-05-20 22:17:51 CEST; 4min 42s ago
May 20 22:17:51 oc-ubu-dev02 systemd[1]: Reached target Cloud-init target.
○ cloud-init.service - Initial cloud-init job (metadata service crawler)
Loaded: loaded (/lib/systemd/system/cloud-init.service; enabled; vendor preset: enabled)
Active: inactive (dead)
# systemctl list-dependencies cloud-init.service
cloud-init.service
○ ├─cloud-init-local.service
● ├─ssh.service
○ ├─sshd-keygen.service
● └─system.slice
# systemctl cat cloud-init.service
# /lib/systemd/system/cloud-init.service
[Unit]
Description=Initial cloud-init job (metadata service crawler)
DefaultDependencies=no
Wants=cloud-init-local.service
Wants=sshd-keygen.service
Wants=sshd.service
After=cloud-init-local.service
After=systemd-networkd-wait-online.service
After=networking.service
Before=network-online.target
Before=sshd-keygen.service
Before=sshd.service
Before=sysinit.target
Before=shutdown.target
Conflicts=shutdown.target
Before=systemd-user-sessions.service
ConditionPathExists=!/etc/cloud/cloud-init.disabled
ConditionKernelCommandLine=!cloud-init=disabled
ConditionEnvironment=!KERNEL_CMDLINE=cloud-init=disabled
[Service]
Type=oneshot
ExecStart=/usr/bin/cloud-init init
RemainAfterExit=yes
TimeoutSec=0
# Output needs to appear in instance console output
StandardOutput=journal+console
[Install]
WantedBy=cloud-init.target
# systemctl list-dependencies cloud-init-local.service
cloud-init-local.service
● ├─-.mount
● ├─system.slice
● └─network-pre.target
# systemctl status cloud-init-local.service
○ cloud-init-local.service - Initial cloud-init job (pre-networking)
Loaded: loaded (/lib/systemd/system/cloud-init-local.service; enabled; vendor preset: enabled)
Active: inactive (dead)
# systemctl cat cloud-init-local.service
# /lib/systemd/system/cloud-init-local.service
[Unit]
Description=Initial cloud-init job (pre-networking)
DefaultDependencies=no
Wants=network-pre.target
After=hv_kvp_daemon.service
After=systemd-remount-fs.service
Before=NetworkManager.service
Before=network-pre.target
Before=shutdown.target
Before=sysinit.target
Conflicts=shutdown.target
RequiresMountsFor=/var/lib/cloud
ConditionPathExists=!/etc/cloud/cloud-init.disabled
ConditionKernelCommandLine=!cloud-init=disabled
ConditionEnvironment=!KERNEL_CMDLINE=cloud-init=disabled
[Service]
Type=oneshot
ExecStart=/usr/bin/cloud-init init --local
RemainAfterExit=yes
TimeoutSec=0
# Output needs to appear in instance console output
StandardOutput=journal+console
[Install]
WantedBy=cloud-init.target# who -b
system boot 2024-05-20 22:17
# ls -la /etc/cloud/cloud-init.disabled
ls: cannot access '/etc/cloud/cloud-init.disabled': No such file or directory
# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.15.0-101-generic root=UUID=da565bae-5497-4be7-9409-e4284fa5d6bc ro ipv6.disable=1 net.ifnames=0 biosdevname=0
# ls -dl /var/lib/cloud
drwxr-xr-x 3 root root 18 May 20 22:17 /var/lib/cloud
# dmesg -T | grep cloud-init
[Mon May 20 22:17:49 2024] systemd[1]: sysinit.target: Found ordering cycle on cloud-init.service/start
[Mon May 20 22:17:49 2024] systemd[1]: sysinit.target: Found dependency on cloud-init-local.service/start
[Mon May 20 22:17:49 2024] systemd[1]: sysinit.target: Job cloud-init.service/start deleted to break ordering cycle starting with sysinit.target/start
[Mon May 20 22:17:49 2024] systemd[1]: sysinit.target: Found ordering cycle on cloud-init-local.service/start
[Mon May 20 22:17:49 2024] systemd[1]: sysinit.target: Job cloud-init-local.service/start deleted to break ordering cycle starting with sysinit.target/start |
|
There is your problem: An ordering cycle dropped two of cloud-init's services from the boot order. Long term @Overmorrow1501 can you please share the output of the following commands? systemctl show -p Requires,Wants,Requisite,BindsTo,PartOf,Before,After cloud-init-local.service
systemd-analyze verify sysinit.target |
|
@holmanb: Yeah, i noticed that, but I forgot to put this output in my previous comment... This is the the dmesg full output regarding the I'll add also the mentioned systemd-units |
holmanb
added
priority
|
adding 24.3 milestone, as we don't necessarily want to block 24.2 on this development. |
TheRealFalcon
added a commit
to TheRealFalcon/cloud-init
that referenced
this issue
Aug 2, 2024
|
The original issue here was fixed in #4864 . Cloud-init no longer reports RUNNING but instead reports DONE.
Attempted in #5579 but realized there's no real value now that we use a single process. I see nothing left to do to address here, so I'm going to close the issue. If there is something we have left unresolved, please comment as to why we should reopen this issue, or open a new issue. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
After boot the newly created VM and every task cloud-init should perform is done, cloud-init status still reports status "Running"
cloud-init configuration
/etc/cloud/cloud.cfg:
###Steps to reproduce the problem
$ cloud-init clean --logs --seed --machine-id --reboot $ cloud-init status --long status: running extended_status: running boot_status_code: enabled-by-generator last_update: Sat, 18 May 2024 14:21:26 +0000 detail: Cloud-init enabled by systemd cloud-init-generator errors: [] recoverable_errors: {} $ cloud-init status --format json { "_schema_version": "1", "boot_status_code": "enabled-by-generator", "datasource": null, "detail": "Cloud-init enabled by systemd cloud-init-generator", "errors": [], "extended_status": "running", "init": { "errors": [], "finished": null, "start": null }, "init-local": { "errors": [], "finished": null, "start": null }, "last_update": "Sat, 18 May 2024 14:21:26 +0000", "modules-config": { "errors": [], "finished": 1716042085.1915476, "recoverable_errors": {}, "start": 1716042085.1726842 }, "modules-final": { "errors": [], "finished": 1716042086.0076993, "recoverable_errors": {}, "start": 1716042085.9895425 }, "recoverable_errors": {}, "schemas": { "1": { "boot_status_code": "enabled-by-generator", "datasource": null, "detail": "Cloud-init enabled by systemd cloud-init-generator", "errors": [], "extended_status": "running", "init": { "errors": [], "finished": null, "start": null }, "init-local": { "errors": [], "finished": null, "start": null }, "last_update": "Sat, 18 May 2024 14:21:26 +0000", "modules-config": { "errors": [], "finished": 1716042085.1915476, "recoverable_errors": {}, "start": 1716042085.1726842 }, "modules-final": { "errors": [], "finished": 1716042086.0076993, "recoverable_errors": {}, "start": 1716042085.9895425 }, "recoverable_errors": {}, "stage": null, "status": "running" } }, "stage": null, "status": "running" }Environment details
cloud-init logs
cloud-init.tar.gz
The text was updated successfully, but these errors were encountered: