feat(akamai): unzip user content

[guilhem]

This pull request introduces functionality to handle compressed userdata in the Linode Metadata Service and includes the implementation of a utility function to decompress gzip data if needed. The most important changes include modifications to the fetchConfig function and the addition of a new utility function for gzip decompression.

Enhancements to userdata handling:

• internal/providers/akamai/akamai.go: Updated the fetchConfig function to decompress gzip data if needed before parsing the configuration.

New utility function:

• internal/providers/util/unzip.go: Added a new utility function GunzipIfNeeded to handle gzip decompression, including a helper function hasGzipMagicNumber to check for gzip magic numbers.

[eljohnson92]

What do you think about inverting this logic so we can un-nest the bulk of the logic here ?

if !hasGzipMagicNumber(raw) { return raw, nil }
...

[tormath1]

This looks good to me from a functional aspect. It has been tested successfully with Flatcar with both configuration (gzipped and not gzipped):

$ linode-cli linodes create \
    --metadata.user_data "$(cat config.json | gzip | base64 -w0)" \
    --label flatcar \
    ...
$ linode-cli linodes create \
    --metadata.user_data "$(cat config.json | base64 -w0)" \
    --label flatcar \
    ...

On the instance:

Flatcar Container Linux by Kinvolk developer 9999.0.0+tormath1-ignition-akamai for Akamai
core@172-236-99-29 ~ $ sudo cat /var/run/ignition.json
{"ignition":{"config":{"replace":{"verification":{}}},"proxy":{},"security":{"tls":{}},"timeouts":{},"version":"3.5.0-experimental"},"kernelArguments...

• Tested patch here: sys-apps/ignition: pull akamai patch flatcar/scripts#2307
• Dev image available here: http://bincache.flatcar-linux.net/images/amd64/9999.0.0+tormath1-ignition-akamai/flatcar_production_akamai_image.bin.gz (instructions: https://www.flatcar.org/docs/latest/installing/community-platforms/akamai/)

[jlebon]

Doesn't the response include a Content-Encoding header? We can enhance the fetcher such that if we pass a flag via FetchOptions (or e.g. set Compression to a special value), it automatically respects Content-Encoding and decompresses it. It would then probably make sense to set this flag for all providers and not just Akamai.

[tormath1]

Doesn't the response include a Content-Encoding header? We can enhance the fetcher such that if we pass a flag via FetchOptions (or e.g. set Compression to a special value), it automatically respects Content-Encoding and decompresses it. It would then probably make sense to set this flag for all providers and not just Akamai.

At the moment it does not seem that Akamai is sending a Content-Encoding header:

core@172-233-211-243 ~ $ curl --silent -H "Metadata-Token: $TOKEN" http://169.254.169.254/v1/user-data | base64 -d | gzip -c -d -
{"ignition":{"version":"3.3.0"},"passwd":{"users":[{"name":"core","sshAuthorizedKeys":["ssh-rsa ...
core@172-233-211-243 ~ $ curl --head -H "Metadata-Token: $TOKEN" http://169.254.169.254/v1/user-data
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 936
Content-Type: text/html; charset=utf-8
Date: Tue, 17 Sep 2024 08:02:27 GMT
Retry-After: 1
Server: nginx/1.18.0
X-Ratelimit-Limit: 10
X-Ratelimit-Remaining: 9
X-Ratelimit-Reset: 1726560149

@guilhem do you think that's something Akamai could consider?

It would then probably make sense to set this flag for all providers and not just Akamai.

That's discussed here I think: #1933 and from what I understand it's not the first time this topic is raised on Ignition. I'm just concerned about relying entirely on this Content-Encoding header - as it's up to each provider to respect this approach...

[guilhem]

I will speak to the metadata team and ask about this header :)