msmtp: msmtpq broken on darwin (ping)

[dbaynard]

Describe the bug

The msmtp sendmail wrapper script, msmtp, checks that there's a network connection before attempting to send emails. The default uses ping, which results in a (silenced) error on darwin. Looks like #185343 is relevant — I don’t understand why ping is excluded (and this can be fixed without changing fake.external).

`connect_test`, from `msmtpq`, as patched by nixpkgs

## test whether system is connected
## returns t/f (0/1)
##
connect_test() {
  if [ -z "$EMAIL_CONN_TEST" ] || \
     [ "$EMAIL_CONN_TEST" = 'p' ] ; then                       # use ping test (default)
    # verify net connection - ping ip address of debian.org
    # would ping -qnc2 -w4 be better ?
    # would ping -qnc1 -w10 or -w20 be better ?
    #ping -qnc1 -w4 debian.org >/dev/null 2>&1 || return 1
    ping -qnc2 -w10 debian.org >/dev/null 2>&1 || return 1

  elif [ "$EMAIL_CONN_TEST" = 'P' ] ; then                     # use quicker ping test
    # I personally think that including a DNS lookup
    # is a better connection test but some
    # have found the above test too slow
    ping -qnc1 -w4 8.8.8.8 >/dev/null 2>&1 || return 1

  elif [ "$EMAIL_CONN_TEST" = 'n' ] ; then                     # use netcat (nc) test
    # must, of course, have netcat (nc) installed
    /nix/store/9vs7i33gxgrjsj74cknzzfylp5acyx6d-which-2.21/bin/which nc >/dev/null 2>&1 || \
      log -e 1 "msmtpq : can't find netcat executable [ nc ]"  # if not found - complain ; quit
    '/nix/store/mcwlaq5pmg2r87kr9s6idxhv5bjn55i8-netcat-gnu-0.7.1/bin/nc' -vz www.debian.org 80 >/dev/null 2>&1 || return 1

  elif [ "$EMAIL_CONN_TEST" = 's' ] ; then                     # use sh sockets test
    # note that this does not work on debian systems
    #   where bash opened sockets are suppressed for security
    #   reasons on multiuser systems - however, this should be
    #   ok for single user systems (including embedded systems)
    # test whether this is supported on your system before using...
    # thank you to Brian Goose, on the list, for encouraging this
    exec 3<>/dev/udp/debian.org/80 || return 1                 # open socket on site ; use dns
    exec 3<&- ; exec 3>&-                                      # close socket
  fi
  return 0
}

#
## ----------------------------------- functions for queue management
## ----------------------------------- queue maintenance mode - (msmtp-queue)
#

The ping call uses an option that is unavailable on macos ping.

> ping -qnc2 -w10 debian.org
ping: invalid option -- w
usage: ping [-AaDdfnoQqRrv] [-c count] [-G sweepmaxsize]
            [-g sweepminsize] [-h sweepincrsize] [-i wait]
            [-l preload] [-M mask | time] [-m ttl] [-p pattern]
            [-S src_addr] [-s packetsize] [-t timeout][-W waittime]
            [-z tos] host
       ping [-AaDdfLnoQqRrv] [-c count] [-I iface] [-i wait]
            [-l preload] [-M mask | time] [-m ttl] [-p pattern] [-S src_addr]
            [-s packetsize] [-T ttl] [-t timeout] [-W waittime]
            [-z tos] mcast-group
Apple specific options (to be specified before mcast-group or host like all options)
            -b boundif           # bind the socket to the interface
            -k traffic_class     # set traffic class socket option
            -K net_service_type  # set traffic class socket options
            --apple-connect       # call connect(2) in the socket
            --apple-time          # display current time

The nixpkgs version of ping (inetutils) works.

> nix shell 'nixpkgs#inetutils' -c ping -qnc2 -w10 debian.org
copying path '/nix/store/l7jimx9mlabgagqlk0r5gmmhnfxb3y02-inetutils-2.3' from 'https://cache.nixos.org'...
PING debian.org (130.89.148.77): 56 data bytes
--- debian.org ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 24.201/24.520/24.839/0.319 ms

A workaround is to call msmtpq with EMAIL_CONN_TEST set to x, n or s, for no testing, netcat testing (nix-provided netcat) or bash socket testing, respectively.

I think the bug is upstream, in that msmtpq assumes a linux-only ping implementation — it could identify whether to call ping with -w (linux) or -t/--timeout (bsd) for the timeout. Oh, and because the ping call diverts errors to /dev/null there’s no way to observe what’s happening.

This could (and perhaps should) be fixed in nixpkgs, though, by substituting the default ping for the ping provided by inetutils (the only one that seems to work on darwin) or setting EMAIL_CONN_TEST to n or s (I’ve only tried n). I suspect setting EMAIL_CONN_TEST to n in the derivation is the best option, and then there’s no messing with ping.

Steps To Reproduce

Steps to reproduce the behavior:

1. Configure msmtpq — it shouldn't need a working configuration in order to see the bug, but a broken configuration shouldn't be expected to succeed anyway.
2. Attempt to send mail.
3. See the following error.

2022 11 Oct 12:29:25 : mail for [ --read-envelope-from --read-recipients -- ${TO_ADDRESS?} ] : couldn't be sent - host not connected
2022 11 Oct 12:29:25 : enqueued mail as : [ 2022-10-11-12.29.25 ] ( --read-envelope-from --read-recipients -- ${TO_ADDRESS?} ) : successful

Expected behavior

This is the success message.

  mail for [ --read-envelope-from --read-recipients ] : send was successful

Screenshots

Additional context

Notify maintainers

@peterhoeg

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix run 'nixpkgs#nix-info' -- -m
 - system: `"aarch64-darwin"`
 - host os: `Darwin 21.6.0, macOS 12.5.1`
 - multi-user?: `yes`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.11.0`
 - channels(root): `"nixpkgs-21.11pre300649.3e0ce8c5d47"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixpkgs`

[peterhoeg]

This should be fixed upstream, but in the mean-time we can fake it on linux (since we need the suid wrapper) or otherwise use ping from inetutils on darwin.

Do you feel like doing a PR? I don't have any readily available machine running mac for testing.

[dbaynard]

Happy to make the PR, though I didn't quite understand your first sentence. Which option would you prefer?

I've been wrapping msmtpq to use EMAIL_CONN_TEST of n — this has been working fine.

{ makeWrapper, msmtp, runCommandLocal }:
runCommandLocal "wrap-msmtmp"
{
  buildInputs = [ makeWrapper ];
} ''
  mkdir -p "$out/bin"
  makeWrapper "${msmtp}/bin/msmtpq" "$out/bin/msmtpq" \
    --argv0 msmtpq \
    --set EMAIL_CONN_TEST n
''

I could adjust the derivation (probably by patching in EMAIL_CONN_TEST=${EMAIL_CONN_TEST:-n}, on darwin only). Does that seem reasonable?

[peterhoeg]

Sorry about the lack of clarity. Your workaround works of course, but it removes functionality on darwin, so I'd rather we do something like what is done in the referenced PR.

[peterhoeg]

And to answer your question, ping is excluded because on nixos we have to pick up the impure version in /run/wrappers/bin rather than the binary directly from inetutils.