Duplicated apt sources/list

[simondeziel]

Bug report

For Noble, cloud-init adds /etc/apt/sources.list.d/ubuntu.sources but those are are already defined in /etc/apt/sources.list. This leads apt to emit many warnings about sources being duplicated.

Steps to reproduce the problem

$ lxc launch ubuntu-daily:24.04 c1
$ lxc exec c1 -- cloud-init status --wait
$ lxc exec c1 -- apt-get update
Hit:1 http://security.ubuntu.com/ubuntu noble-security InRelease
Get:2 http://archive.ubuntu.com/ubuntu noble InRelease [240 kB]
Hit:3 http://archive.ubuntu.com/ubuntu noble-updates InRelease
...
Fetched 27.9 MB in 3s (8032 kB/s)                                   
Reading package lists... Done
W: Target Packages (main/binary-amd64/Packages) is configured multiple times in /etc/apt/sources.list:37 and /etc/apt/sources.list.d/ubuntu.sources:2
W: Target Packages (main/binary-all/Packages) is configured multiple times in /etc/apt/sources.list:37 and /etc/apt/sources.list.d/ubuntu.sources:2
W: Target Translations (main/i18n/Translation-en) is configured multiple times in /etc/apt/sources.list:37 and /etc/apt/sources.list.d/ubuntu.sources:2
...

Environment details

• Cloud-init version: 23.4-0ubuntu1
• Operating System Distribution: Ubuntu Noble Numbat (development branch)
• Cloud provider, platform or installer type: LXD

cloud-init logs

cloud-init.tar.gz

[aciba90]

Thanks, @simondeziel, for reporting this and making cloud-init better.

I can confirm this is reproducible in noble. Those warnings are not hard errors and apt works correctly, but having two sources of config could lead to conflicts / unexpected behavior / hard apt errors in the future. In addition to that, the warnings could be a concern for users, so I think this is a bug that has to be fixed.

One possible simple and safe way to solve this would be, if features.APT_DEB822_SOURCE_LIST_FILE and /etc/apt/source.list is present, then do work with the classic sources lists and emit a warning informing the user about the fact that deb822 style has not been used.

Another way would be to parse the info from /etc/apt/sources.list, merge it into the deb822 one and /etc/apt/sources.list.old. This option would be more risky and difficult to maintain, and it would only provide value while images/distro are transitioning from classic to deb822.

[aciba90]

Ubuntu downstream bug: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/2045086.

[blackboxsw]

This seems to be an artifact of the image build process. The file /etc/apt/sources.list predates when cloud-init is run.

root@unique-terrapin:~# cat /etc/cloud/build.info 
build_name: server
serial: 20231213
root@unique-terrapin:~# ls -l  --full-time /etc/apt/sources.list
-rw-r--r-- 1 root root 2403 2023-12-13 08:38:45.000000000 +0000 /etc/apt/sources.list
root@unique-terrapin:~# egrep 'Writing.*/etc'  /var/log/cloud-init.log 
2023-12-14 19:54:04,260 - util.py[DEBUG]: Writing to /etc/hostname - wb: [644] 16 bytes
2023-12-14 19:54:04,363 - util.py[DEBUG]: Writing to /etc/netplan/50-cloud-init.yaml - wb: [600] 389 bytes
2023-12-14 19:54:05,613 - util.py[DEBUG]: Writing to /etc/sudoers.d/90-cloud-init-users - wb: [440] 131 bytes
2023-12-14 19:54:09,640 - util.py[DEBUG]: Writing to /etc/apt/sources.list.d/ubuntu.sources - wb: [644] 3097 bytes
root@unique-terrapin:~# lsb_release -sc
No LSB modules are available.
noble

[simondeziel]

I concur, /etc/apt/sources.list is there before cloud-init is involved hence the cloud-init status --wait part that's needed to reproduce. Without it, you can immediately apt-get update cleanly right after the instance is launched.