Standardize image names and source URLs

[mbuechse]

We want users to be able to upload their own images to Glance, so we mandate that this must be possible (with some quota).

We want to standardize image names and source URLs with the intention that the CSP can provide any subset of these images at will; AND if an image with a name from that list is provided, it has to be from the given source URL. The list should be given by a yaml file, whose schema could be a subset of the one for the image-manager yaml files.

The only mandatory image should be the kubernetes-capi images. These could be listed in the yaml file as well and marked mandatory.

Work on this issue can be done inside #314 -- at least the applicable parts from that PR should be reused.

Definition of done:

• A new standard is written that covers image upload and standard names/URLs (given via a yaml file that is regarded as an annex).
• The yaml file lists the mandatory kubernetes-capi images.
• From the existing image metadata standard the section "Standard images" is removed.
• A test script has been written for the new standard, and it has been added to https://github.com/SovereignCloudStack/standards/blob/issue/267/Tests/testing/scs-compatible-test.yaml.
• The new standard has been added to Entropy test sets up and tears down its network topology #317

[mbuechse]

Documentation about quotas in Glance: https://docs.openstack.org/glance/latest/admin/quotas.html
Team IaaS opined about minima:

maybe start with at least 200 GB (and 40 images)

https://input.scs.community/2023-scs-team-iaas?both#Standardize-image-names-and-source-URLs-standards326

[mbuechse]

The matter of image_source seems to be kind of tricky.

• For instance, Debian 11 can be downloaded from https://cdimage.debian.org/cdimage/cloud/bullseye, but you still need to go into a subfolder.
• The update script from the original(TM) image manager uses the address https://cdimage.debian.org/cdimage/cloud/bullseye/latest, and mirrors the version found there under https://minio.services.osism.tech/openstack-image-manager/debian-11. Now of course, the original source is no longer visible! Unfortunately, the mirror URL is what gets set as image_source.
• The image crawler by PlusServer is better in this regard. It looks up the versions listed under https://cdimage.debian.org/cdimage/cloud/bullseye and creates corresponding version entries for the image manager to use, where the image_source is then correctly, e.g., https://cloud.debian.org/images/cloud/bullseye/20230802-1460/...etc...

So we can't really specify the image_source, but we can specify a prefix, or if we want to get fancy, we can use a regular expression. It would be important though that the update script in the image manager preserve the source URL to be used for the properties regardless of mirroring.

[mbuechse]

The next problem then is whether to use (for the example of Ubuntu 20.04 LTS)

• https://cloud-images.ubuntu.com/releases/focal (release build)
• https://cloud-images.ubuntu.com/focal (daily build)

Or we could allow both, either by (again) using a regular expression, or by providing a list of acceptable sources.

Needless to say, the original image manager uses the daily build, whereas the image crawler prefers release builds.

Oh, also, there is a difference between

• https://cdimage.debian.org/cdimage/cloud/bullseye/
• https://cloud.debian.org/images/cloud/bullseye/

that I totally missed (even in the previous comment)!! Which one should we prescribe?