Medium: exportfs: Use canonical hostname for monitor op

[nrwahl2]

Issue

If the hostname specified in clientspec does not match the canonical
hostname, the monitor operation fails even though the export exists.

This is caused by the exportfs command's name resolution process. It
queries the hosts DB as configured in /etc/nsswitch.conf (usually
/etc/hosts and/or DNS) to ensure that the given hostname resolves.

If the lookup returns the FQDN and the clientspec uses the short name,
the monitor operation fails. This also occurs in reverse. If the lookup
returns the short name and the clientspec uses the FQDN, the monitor
operation fails.

# # FQDN is configured first in /etc/hosts
# grep fastvm-rhel-7-5-36 /etc/hosts | grep -v ^#
192.168.22.36	fastvm-rhel-7-5-36.localdomain	fastvm-rhel-7-5-36

# # Exported as FQDN regardless of whether short name or FQDN is specified
# exportfs -v fastvm-rhel-7-5-36:/mnt/clusterfs
exportfs: No file systems exported!
exporting fastvm-rhel-7-5-36.localdomain:/mnt/clusterfs
# exportfs -u fastvm-rhel-7-5-36:/mnt/clusterfs
# exportfs -v fastvm-rhel-7-5-36.localdomain:/mnt/clusterfs
exportfs: No file systems exported!
exporting fastvm-rhel-7-5-36.localdomain:/mnt/clusterfs


# # Short name is configured first in /etc/hosts
# grep fastvm-rhel-7-5-36 /etc/hosts | grep -v ^#
192.168.22.36	fastvm-rhel-7-5-36	fastvm-rhel-7-5-36.localdomain

# # Exported as short name regardless of whether short name or FQDN is specified
# exportfs -v fastvm-rhel-7-5-36:/mnt/clusterfs
exportfs: No file systems exported!
exporting fastvm-rhel-7-5-36:/mnt/clusterfs
# exportfs -u fastvm-rhel-7-5-36:/mnt/clusterfs
# exportfs -v fastvm-rhel-7-5-36.localdomain:/mnt/clusterfs
exportfs: No file systems exported!
exporting fastvm-rhel-7-5-36:/mnt/clusterfs

Reproducer

1. Configure /etc/hosts with a record for the intended client of the following format:

    <ip>    <fqdn>    <short name>

For example:

    192.168.22.36	fastvm-rhel-7-5-36.localdomain	fastvm-rhel-7-5-36

2. Create an ocf:heartbeat:exportfs resource using the short name in the clientspec attribute.

    # pcs resource create nfs_export ocf:heartbeat:exportfs clientspec=fastvm-rhel-7-5-36 directory=/mnt/clusterfs fsid=1

3. Observe the resource cycling through successful start -> failed monitor -> successful stop -> successful start in /var/log/messages.

Sep 27 23:08:26 fastvm-rhel-7-5-22 crmd[1421]:  notice: Result of start operation for nfs_export on fastvm-rhel-7-5-22: 0 (ok)
Sep 27 23:08:26 fastvm-rhel-7-5-22 crmd[1421]:  notice: Initiating monitor operation nfs_export_monitor_10000 locally on fastvm-rhel-7-5-22
Sep 27 23:08:26 fastvm-rhel-7-5-22 exportfs(nfs_export)[5101]: INFO: Directory /mnt/clusterfs is not exported to fastvm-rhel-7-5-36 (stopped).
Sep 27 23:08:26 fastvm-rhel-7-5-22 crmd[1421]: warning: Action 5 (nfs_export_monitor_10000) on fastvm-rhel-7-5-22 failed (target: 0 vs. rc: 7): Error
...
Sep 27 23:08:26 fastvm-rhel-7-5-22 crmd[1421]:  notice: Initiating stop operation nfs_export_stop_0 locally on fastvm-rhel-7-5-22
Sep 27 23:08:26 fastvm-rhel-7-5-22 exportfs(nfs_export)[5136]: INFO: Directory /mnt/clusterfs is not exported to fastvm-rhel-7-5-36 (stopped).
Sep 27 23:08:26 fastvm-rhel-7-5-22 crmd[1421]:  notice: Result of stop operation for nfs_export on fastvm-rhel-7-5-22: 0 (ok)
Sep 27 23:08:26 fastvm-rhel-7-5-22 crmd[1421]:  notice: Initiating start operation nfs_export_start_0 locally on fastvm-rhel-7-5-22
Sep 27 23:08:26 fastvm-rhel-7-5-22 exportfs(nfs_export)[5171]: INFO: Directory /mnt/clusterfs is not exported to fastvm-rhel-7-5-36 (stopped).
Sep 27 23:08:26 fastvm-rhel-7-5-22 exportfs(nfs_export)[5171]: INFO: Exporting file system ...
Sep 27 23:08:26 fastvm-rhel-7-5-22 exportfs(nfs_export)[5171]: INFO: exportfs: No file systems exported! exporting fastvm-rhel-7-5-36.localdomain:/mnt/clusterfs
Sep 27 23:08:26 fastvm-rhel-7-5-22 exportfs(nfs_export)[5171]: WARNING: rmtab backup /mnt/clusterfs/.rmtab not found or not readable.
Sep 27 23:08:26 fastvm-rhel-7-5-22 exportfs(nfs_export)[5171]: INFO: File system exported
Sep 27 23:08:26 fastvm-rhel-7-5-22 crmd[1421]:  notice: Result of start operation for nfs_export on fastvm-rhel-7-5-22: 0 (ok)

Resolution

This patch resolves the issue by querying the ahosts database to fetch
the canonical hostname. We use the canonical hostname as the spec. If
the clientspec is an IP address, the fetch simply returns the same IP
address. If the clientspec is of some other format, the fetch returns
nothing and we continue to use the spec as-is.

Using getent ahosts rather than getent hosts eliminates the need to
handle IP addresses specially, thanks to its output formatting. It uses
getaddrinfo rather than gethostbyname2.

Notes

I added the NF == 3 awk filter to match only lines that included a canonical name/address, which will be in the third field. I exited after the first match in case there are multiple. In my testing, the line with the canonical name/address was always the first line, and there was never more than one match. So this may not be necessary. Better safe than sorry.

I added the '$NF != "localhost"' awk filter because of an edge case. An otherwise unrecognized hostname ending in ".localdomain" matches against the localhost record.

# getent ahosts asdf.localdomain 
::1             STREAM localhost
::1             DGRAM  
::1             RAW    
127.0.0.1       STREAM 
127.0.0.1       DGRAM  
127.0.0.1       RAW  

# getent ahosts asdf.localdomain | awk 'NF == 3 { print $3; exit }'
localhost

This broke monitoring for clientspecs with wildcard characters in my testing.

[nrwahl2]

Test cases:
https://gist.github.com/nrwahl2/9bf6b669ca77c24a9d65ed75f6760bf2

[nrwahl2]

RHBZ#1633848

[knet-jenkins]

Can one of the admins check and authorise this run please: https://ci.kronosnet.org/job/resource-agents-pipeline/job/PR-1225/1/input