Increase the OEM partition size

[margamanterola]

Current situation
Currently we are limited to shipping a maximum of 128MB in the OEM partition.

Impact
This means that if we're trying to ship big OEM tools, we can't. Users can run those tools as containers, but it means that running those tools requires an internet connection. This is affecting AWS in particular (#107)

Ideal future situation
We should have a bigger OEM partition so that we are not limited in the tools that we want to ship.

Implementation options
Going for 1GB for the OEM partition sounds sensible given the current state. As the size of the downloaded image only includes actual data, it would only grow in the cases where we would ship more stuff (like AWS). The main difference with the current state would be that after the image gets unpacked and the root disk resized, there would be ~900MB less in the root partition. This also means that the minimum disk size would grow by those same ~900MB.

If we think that's too much, we could go with 512MB, which would still allow us to ship the needed AWS tools and leave some space for growth. However, I'd rather not have to keep doing this every couple of years as things grow, so I'd go with 1GB.

Additional information
As the OEM partition is not currently part of the update payload, a change in the size of the partition wouldn't interfere with updates. When, at some point in the future, we finally implement updating the OEM partition, we will need to take into account the underlying size.

[samm-git]

+1

[margamanterola]

While talking about this, it was raised that for some specific architectures (e.g. nodes running "on the edge"), increasing the partition size if it's going to be empty or almost empty would be wasteful. We might want to consider having different partition sizes depending on whether or not we actually use them.

[samm-git]

@marga-kinvolk its pity that we are not using something like zfs where volume sizes are dynamic. Anyway, i dont think that these days it will make such a big deal, especially because it will be compressed anyway during distribution

[margamanterola]

Increasing the partition size would have no effect on our image tarballs, but it would affect the amount of available disk space on the running instances.

The concern raised was that on machines where the oem partition is not heavily used and use a small disk size (like edge VMs), this could be a significant penalty for little gain.

We think it could be possible (and not too hard) to have 2 OEM partition sizes one small (current size) and one large (1GB) and apply the right partition size according to the platform.

[pothos]

With the btrfs compression for the OEM partition we are able to store the SSM agent's 122 MB binaries into a 37 MB used filesystem.
Now we just use zlib because zstd isn't available yet until we update GRUB to a newer version. But with the current setup we can store around ~400 MB of such Go binaries like the SSM agent's ones in the 128 MB OEM partition if the calculation holds.

I suggest to close this issue. And if the size is really a problem, I think we should rather look into fetching things at runtime through Docker images, which also has some advantages to solve the update problem.