DigitalOcean: no set hostname

[dustymabe]

When I boot one of our DigitalOcean images following the documentation. I end up with a machine that doesn't have a hostname set. I guess with DigitalOcean this information isn't provided via DHCP and only via the metadata service.

This turns out to be a problem when trying to install OKD, where the install fails in bizarre ways if every node in the cluster has a hostname of localhost. How did this work in the past for CL? Is this something Ignition or Afterburn should handle?

For OKD I was able to workaround with something like this to add a service to the existing ignition config provided by the openshift-installer that would set the hostname based on the metadata:

$ cat fcct-master-withhostname.yaml 
variant: fcos
version: 1.1.0
ignition:
  config:
    merge:
      - local: ./initial/master.ign
systemd:                        
  units:                        
  - name: sethostname.service       
    enabled: true               
    contents: |                 
      [Unit]
      After=NetworkManager-wait-online.service
      [Service]
      Type=oneshot
      ExecStart=/usr/local/bin/run-hostnamectl
      RemainAfterExit=yes
      [Install]
      WantedBy=multi-user.target
storage:
  files:
    - path: /usr/local/bin/run-hostnamectl
      mode: 0755
      contents:
        inline: |
          #!/usr/bin/bash
          hostnamectl set-hostname $(curl -s http://169.254.169.254/metadata/v1/hostname)

[lucab]

I guess with DigitalOcean this information isn't provided via DHCP and only via the metadata service.

Can you please verify this at NM level? (I'm not exactly sure how, but between logs and DBus API there should be some way to know where it is getting the hostname from).

How did this work in the past for CL? Is this something Ignition or Afterburn should handle?

CL images were running on the non-custom non-DHCP infrastructure. Afterburn already knows how to read the properties on DO, but we are currently not enabling the dracut service there. If indeed the DHCP does not provide the hostname to the node, we can patch the Afterburn service to do that.

[dustymabe]

Can you please verify this at NM level? (I'm not exactly sure how, but between logs and DBus API there should be some way to know where it is getting the hostname from).

Here is what I see in the leases file that NM saves:

# tail -n 100 /var/lib/NetworkManager/internal-*lease
==> /var/lib/NetworkManager/internal-94d96937-015f-4b88-8cfa-1400cab728cb-eth0.lease <==
# This is private data. Do not parse.
ADDRESS=159.203.64.40
NETMASK=255.255.240.0
ROUTER=159.203.64.1
SERVER_ADDRESS=169.254.169.253
NEXT_SERVER=169.254.169.253
MTU=1500
T1=43200
T2=75600
LIFETIME=86400
DNS=67.207.67.2 67.207.67.3
CLIENTID=01a6f60160288d

==> /var/lib/NetworkManager/internal-94d96937-015f-4b88-8cfa-1400cab728cb-eth1.lease <==
# This is private data. Do not parse.
ADDRESS=10.132.118.137
NETMASK=255.255.0.0
SERVER_ADDRESS=169.254.169.253
NEXT_SERVER=169.254.169.253
MTU=1500
T1=43200
T2=75600
LIFETIME=86400
CLIENTID=0192116fa956e6

I don't see any information about hostname in there but I can find it in the metadata service viacurl -s http://169.254.169.254/metadata/v1/hostname

[max3903]

@dustymabe Your script above allowed me to install the bootstrap and masters.

[lucab]

@dustymabe great, thanks! Then something like the following should work:

sudo /usr/bin/afterburn --cmdline --hostname=/etc/hostname

If so, merging coreos/afterburn#433 and landing it in an FCOS release should take care of this.

[dustymabe]

[core@localhost ~]$ cat /etc/hostname
cat: /etc/hostname: No such file or directory
[core@localhost ~]$ sudo /usr/bin/afterburn --cmdline --hostname=/etc/hostname
Jun 15 15:53:22.725 INFO Fetching http://169.254.169.254/metadata/v1.json: Attempt #1
Jun 15 15:53:22.779 INFO Fetch successful
[core@localhost ~]$ cat /etc/hostname 
fedoracoreos322020060122digitaloceanx86-64-s-1vcpu-2gb-nyc3-01

Looks good!

[lucab]

fedoracoreos322020060122digitaloceanx86-64-s-1vcpu-2gb-nyc3-01

Is this, uh, awesome hostname expected?

[dustymabe]

ha, yes. It derived the name from the custom image name I selected, which was long. Disregard that, just concentrate on the success!

[lucab]

The fix for this landed in Afterburn 4.4.1.

[dustymabe]

The fix for this went into testing stream release 32.20200726.2.0. Please try out the new release and report issues.

[dustymabe]

The fix for this went into stable stream release 32.20200726.3.1.