Allow for full managed master.cf services

[ixs]

Currently master.cf only allows for very limited configuration
options mainly focussed on SMTP submission settings.

This is rather limited and does not scale very well for managing
the other services defined in master.cf.

This patch has moved all the service definitions into a jinja file
and generates the master.cf service definition on the fly based on
these defaults.

Defaults can be overridden in a pillar to customize the rendered
master.cf file accordingly to local needs.
Undefined values will be filled with the postfix defaults.

Care has been taken that the previous ways of managing the submission
configuration options are still supported for backwards compatibility
to prevent breakage for existing users of the formula.

[javierbertoli]

Running the postfix.config state as is, in a Debian host, it changes a bunch of defaults, which I'm not sure is a good thing or harmless.

What do you think of changing this default to False, so the master.cf file is managed when explicitely desired?

Also, I'd add a

# This file managed by Salt, do not edit by hand!!

or similar header

[ixs]

@javierbertoli Sure, changing the default to false should be simple and probably makes sense.

What about the changed defaults you mention? Are these changes happening on commented out services or on active ones? I'd be interested in seeing what changed as I based the services defaults on current upstream configuration examples...

[javierbertoli]

@ixs , they happen in both active and inactive ones, here's the output on a kitchen run:

                  +++ 
                  @@ -9,12 +9,12 @@
                   # service type  private unpriv  chroot  wakeup  maxproc command + args
                   #               (yes)   (yes)   (no)    (never) (100)
                   # ==========================================================================
                  -smtp      inet  n       -       y       -       -       smtpd
                  -#smtp      inet  n       -       y       -       1       postscreen
                  -#smtpd     pass  -       -       y       -       -       smtpd
                  -#dnsblog   unix  -       -       y       -       0       dnsblog
                  -#tlsproxy  unix  -       -       y       -       0       tlsproxy
                  -#submission inet n       -       y       -       -       smtpd
                  +smtp      inet  n       -       n       -       -       smtpd
                  +#smtp      inet  n       -       n       -       1       postscreen
                  +#smtpd     pass  -       -       n       -       -       smtpd
                  +#dnsblog   unix  -       -       n       -       0       dnsblog
                  +#tlsproxy  unix  -       -       n       -       0       tlsproxy
                  +#submission inet  n       -       n       -       -       smtpd
                   #  -o syslog_name=postfix/submission
                   #  -o smtpd_tls_security_level=encrypt
                   #  -o smtpd_sasl_auth_enable=yes
                  @@ -22,10 +22,9 @@
                   #  -o smtpd_client_restrictions=$mua_client_restrictions
                   #  -o smtpd_helo_restrictions=$mua_helo_restrictions
                   #  -o smtpd_sender_restrictions=$mua_sender_restrictions
                  -#  -o smtpd_recipient_restrictions=
                  -#  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
                  +#  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
                   #  -o milter_macro_daemon_name=ORIGINATING
                  -#smtps     inet  n       -       y       -       -       smtpd
                  +#smtps     inet  n       -       n       -       -       smtpd
                   #  -o syslog_name=postfix/smtps
                   #  -o smtpd_tls_wrappermode=yes
                   #  -o smtpd_sasl_auth_enable=yes
                  @@ -33,35 +32,34 @@
                   #  -o smtpd_client_restrictions=$mua_client_restrictions
                   #  -o smtpd_helo_restrictions=$mua_helo_restrictions
                   #  -o smtpd_sender_restrictions=$mua_sender_restrictions
                  -#  -o smtpd_recipient_restrictions=
                  -#  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
                  +#  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
                   #  -o milter_macro_daemon_name=ORIGINATING
                  -#628       inet  n       -       y       -       -       qmqpd
                  -pickup    unix  n       -       y       60      1       pickup
                  -cleanup   unix  n       -       y       -       0       cleanup
                  +#628       inet  n       -       n       -       -       qmqpd
                  +pickup    unix  n       -       n       60      1       pickup
                  +cleanup   unix  n       -       n       -       0       cleanup
                   qmgr      unix  n       -       n       300     1       qmgr
                  -#qmgr     unix  n       -       n       300     1       oqmgr
                  -tlsmgr    unix  -       -       y       1000?   1       tlsmgr
                  -rewrite   unix  -       -       y       -       -       trivial-rewrite
                  -bounce    unix  -       -       y       -       0       bounce
                  -defer     unix  -       -       y       -       0       bounce
                  -trace     unix  -       -       y       -       0       bounce
                  -verify    unix  -       -       y       -       1       verify
                  -flush     unix  n       -       y       1000?   0       flush
                  +#qmgr      unix  n       -       n       300     1       oqmgr
                  +tlsmgr    unix  -       -       n       1000?   1       tlsmgr
                  +rewrite   unix  -       -       n       -       -       trivial-rewrite
                  +bounce    unix  -       -       n       -       0       bounce
                  +defer     unix  -       -       n       -       0       bounce
                  +trace     unix  -       -       n       -       0       bounce
                  +verify    unix  -       -       n       -       1       verify
                  +flush     unix  n       -       n       1000?   0       flush
                   proxymap  unix  -       -       n       -       -       proxymap
                  -proxywrite unix -       -       n       -       1       proxymap
                  -smtp      unix  -       -       y       -       -       smtp
                  -relay     unix  -       -       y       -       -       smtp
                  +proxywrite unix  -       -       n       -       1       proxymap
                  +smtp      unix  -       -       n       -       -       smtp
                  +relay     unix  -       -       n       -       -       smtp
                   #       -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
                  -showq     unix  n       -       y       -       -       showq
                  -error     unix  -       -       y       -       -       error
                  -retry     unix  -       -       y       -       -       error
                  -discard   unix  -       -       y       -       -       discard
                  +showq     unix  n       -       n       -       -       showq
                  +error     unix  -       -       n       -       -       error
                  +retry     unix  -       -       n       -       -       error
                  +discard   unix  -       -       n       -       -       discard
                   local     unix  -       n       n       -       -       local
                   virtual   unix  -       n       n       -       -       virtual
                  -lmtp      unix  -       -       y       -       -       lmtp
                  -anvil     unix  -       -       y       -       1       anvil
                  -scache    unix  -       -       y       -       1       scache
                  +lmtp      unix  -       -       n       -       -       lmtp
                  +anvil     unix  -       -       n       -       1       anvil
                  +scache    unix  -       -       n       -       1       scache
                   #
                   # ====================================================================
                   # Interfaces to non-Postfix software. Be sure to examine the manual
                  @@ -75,8 +73,8 @@
                   # maildrop. See the Postfix MAILDROP_README file for details.
                   # Also specify in main.cf: maildrop_destination_recipient_limit=1
                   #
                  -maildrop  unix  -       n       n       -       -       pipe
                  -  flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
                  +#maildrop  unix  -       n       n       -       -       pipe
                  +#  flags=DRhu user=vmail argv=/usr/local/bin/maildrop -d ${recipient}
                   #
                   # ====================================================================
                   #
                  @@ -98,6 +96,7 @@
                   #  user=cyrus argv=/cyrus/bin/deliver -e -r ${sender} -m ${extension} ${user}
                   #
                   # ====================================================================
                  +#
                   # Old example of delivery via Cyrus.
                   #
                   #old-cyrus unix  -       n       n       -       -       pipe
                  @@ -107,18 +106,23 @@
                   #
                   # See the Postfix UUCP_README file for configuration details.
                   #
                  -uucp      unix  -       n       n       -       -       pipe
                  -  flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
                  +#uucp      unix  -       n       n       -       -       pipe
                  +#  flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
                  +#
                  +# ====================================================================
                   #
                   # Other external delivery methods.
                   #
                  -ifmail    unix  -       n       n       -       -       pipe
                  -  flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
                  -bsmtp     unix  -       n       n       -       -       pipe
                  -  flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
                  -scalemail-backend unix	-	n	n	-	2	pipe
                  -  flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
                  -mailman   unix  -       n       n       -       -       pipe
                  -  flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
                  -  ${nexthop} ${user}
                  -
                  +#ifmail    unix  -       n       n       -       -       pipe
                  +#  flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
                  +#
                  +#bsmtp     unix  -       n       n       -       -       pipe
                  +#  flags=Fq. user=bsmtp argv=/usr/local/sbin/bsmtp -f $sender $nexthop $recipient
                  +#
                  +#scalemail-backend unix -       n       n       -       2       pipe
                  +#  flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store
                  +#  ${nexthop} ${user} ${extension}
                  +#
                  +#mailman   unix  -       n       n       -       -       pipe
                  +#  flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
                  +#  ${nexthop} ${user}

It's quite possible the changes are Debian specific (that diverge from upstream). I bet the same happens in other platforms. That's why I always liked formulas to have this logic 😄 :

1. If no config state is added, leave config untouched (so it will respect whatever is already set, by the package, by other means, etc.).
2. If a config state is called, and there are multiple config files, add a manage_* defaulting to false for each of them, so you can manually configure what you want and respect what you don't want to manage.

[javierbertoli]

@ixs, @aboe76, I've been checking this formula and this PR, and found that the current state of the formula fails to configure master.cf: when you set

policyd-spf:
  enabled: true

this section fails with

----------
ID: /etc/postfix/master.cf
  Function: file.managed
  Result: False
  Comment: Unable to manage file: Jinja variable 'xbin_prefix' is undefined

because the xbin_prefix var is used as local, and is not locally set.

The error is still present in this PR because @ixs changes do not touch that section. Changing that piece of code to

  user=nobody argv={{ postfix.xbin_prefix }}/bin/policyd-spf

will fix that error.

With that, I'd say I'm OK with merging this PR as is. After all,

• the settings @ixs declare are upstream's, which is a sane-enough first step for this refactoring,
• the setting of postfix:manage_master_config: True is the current state of affairs (so nothing changes here) and
• adding the "Managed by Salt" header can be done in another PR.

[aboe76]

@javierbertoli will you create a PR for the xbin_prefix issue?

[aboe76]

@ixs and @javierbertoli thanks for testing and reviewing !!

[javierbertoli]

@aboe76 submitted #67