1:N address expansion

[greengekota]

Use case

It it described on wikipedia:

• Role-addresses
  info, sales, postmaster, and similar names[6] can appear to the left of @ in email addresses. An organization may forward messages intended for a given role to the address of the person(s) currently functioning in that role or office.

• Multiple, or discontinued addresses
  When users change their email address, or have several addresses, the user or an administrator may set up forwarding from these addresses, if still valid, to a single current one, in order to avoid losing messages.

Your idea for a solution

I haven't looked at the source code of maddy. It would be nice to store forward rules in a table such as

(1, "root@domain.com", "alice@domain.com")
(2, "root@domain.com", "bob@domain.com")
(3, "root@domain.com", "bob@otherdomain.com")

and so when a mail is received the To header would be checked against "root@domain.com" and automatically forwarded to "alice@domain.com", "bob@domain.com" and "bob@otherdomain.com". The table could be altered (and reloaded?) via the command line or other mechanisms.

I'm not sure how difficult would that be.

• I'm willing to help with the implementation

[foxcpp]

maddy supports forwarding ("address rewriting") but it is currently limited to 1-to-1. That is, root@domain.com could be aliased to alice@domain.com or bob@domain.com or bob@otherdomain.com (requires additional configuration if otherdomain.com is not handled by the same server, https://maddy.email/tutorials/alias-to-remote/).

[greengekota]

Yes my interest is in the other situations, such as 1:n with n greater than one. My main problem right now is that I would like to have mail addressing postmaster being read by more than one person. And for 1:1, in the situation it would be nice to have a copy of the mail both in root@domain.com mail storage and in the one of bob@otherdomain.com .

[angelnu]

I am also interested in the 1:N address expansion and trying to add a PR.

My thinking would be to make RewriteRcpt return a []string instead of a single string so that later for each to in the list we can do:

maddy/internal/msgpipeline/msgpipeline.go

Lines 309 to 364 in 85d3648

	rcptBlock, err := dd.rcptBlockForAddr(ctx, to)
	if err != nil {
	return wrapErr(err)
	}
	if rcptBlock.rejectErr != nil {
	return wrapErr(rcptBlock.rejectErr)
	}
	if err := dd.checkRunner.checkRcpt(ctx, rcptBlock.checks, to); err != nil {
	return wrapErr(err)
	}
	rcptModifiersState, err := dd.getRcptModifiers(ctx, rcptBlock, to)
	if err != nil {
	return wrapErr(err)
	}
	newTo, err = rcptModifiersState.RewriteRcpt(ctx, to)
	if err != nil {
	rcptModifiersState.Close()
	return wrapErr(err)
	}
	dd.log.Debugln("per-rcpt modifiers:", to, "=>", newTo)
	to = newTo
	wrapErr = func(err error) error {
	return exterrors.WithFields(err, map[string]interface{}{
	"effective_rcpt": to,
	})
	}
	if originalTo != to {
	dd.msgMeta.OriginalRcpts[to] = originalTo
	}
	for _, tgt := range rcptBlock.targets {
	// Do not wrap errors coming from nested pipeline target delivery since
	// that pipeline itself will insert effective_rcpt field and could do
	// its own rewriting - we do not want to hide it from the admin in
	// error messages.
	wrapErr := wrapErr
	if _, ok := tgt.(*MsgPipeline); ok {
	wrapErr = func(err error) error { return err }
	}
	delivery, err := dd.getDelivery(ctx, tgt)
	if err != nil {
	return wrapErr(err)
	}
	if err := delivery.AddRcpt(ctx, to); err != nil {
	return wrapErr(err)
	}
	delivery.recipients = append(delivery.recipients, originalTo)
	}

Do you think that I am in the right track?

Also, what separator should we take? A comma separated lists?
root@domain.com: bob@otherdomain.com,bob@otherdomain.com

[foxcpp]

Do you think that I am in the right track?

Yeah, that should work. Assuming we keep current naive handling of partial failures, but I would prefer to leave that can of worms for later.

Also, what separator should we take? A comma separated lists?

Better hope no-one uses a "a,b,c"@example.org email address. The only 100% safe approach for this is to use \t or \n as a separator, both of these are required to be escaped even in quoted email addresses.
I wonder if we should take an opinionated approach and just tell people they cannot do rewriting with addresses with commas in them.

[angelnu]

Thanks for confirming! I will start working on it - I was checking first I could run all testcases so I ensure I do not break anything as I start with the changes.

I am all for the opinionated as it makes the rules easier to read/understand. I proposed , under the impression that it is not a valid character in email addresses (several programs use , as separators) but if another character is better (maybe an space?) we can go with it.

[foxcpp]

I am all for the opinionated as it makes the rules easier to read/understand

Agreed. People with weird addresses can use other data sources (e.g. SQL database) that have no similar problems.

P.S. There is some existing code that you probably want to use, some module.Table implementations have optional LookupMulti method that returns multiple entries. It probably should be used by rewrite_rcpt implementation if available. And then file-based implementation should be adjusted to use , as a separator.

[angelnu]

ok, I will then use LookupMulti when possible and default to , separated addresses otherwise. Just confirming - , is better than as separator?

[greengekota]

Better hope no-one uses a "a,b,c"@example.org email address.

Is it enough to keep reading until the next unescaped ", after encountering a "? In this case a modification to the string split algorithm should suffice.

Also, is it going to be a problem if someone puts something like

a@example.com: a@example.com

or

a@example.com: b@example.com
b@example.com: a@example.com

?

[angelnu]

a@example.com: a@example.com

This is a nop but should not result an infinite loop

a@example.com: b@example.com
b@example.com: a@example.com

This would also be a nop after replacing a->b->a. This is not really related to the 1:N, is it? This type of config is valid today.

We cannot assume " in the email addresses (most will not), so using string splits with , or should work in most cases.

[foxcpp]

ok, I will then use LookupMulti when possible and default to , separated addresses otherwise.

I think if LookupMulti is not defined then you should not do any splitting on singular value returned by Lookup. This will cause problems with all other non-file based table implementations.

Just confirming - , is better than as separator?

I believe , is less likely to appear in an address.

Is it enough to keep reading until the next unescaped ", after encountering a "? In this case a modification to the string split algorithm should suffice.

table.file is also re-used for other values that are not email address, I would prefer to keep parser as content-unaware as possible.

Also, is it going to be a problem if someone puts something like
a@example.com: a@example.com

or

a@example.com: b@example.com
b@example.com: a@example.com
?

P.S. As described in replace_rcpt documentation, it does only one replacement step - aliases are not resolved recursively.

[greengekota]

This is not really related to the 1:N, is it?

Not really, I was wondering about a situation like this (or some faulty version)

postmaster@example.com: root@example.com
hostmaster@example.com: root@example.com
...
root@example.com: admin1@example.com, admin2@example.com

but one could just replace root with the addresses in the config file, so it's not necessary. I was worried about possible loops, since I saw AddRcpt calling itself. Thank you both for your replies.

[angelnu]

I have not implemented the , separated case in file yet but the following should work

root@example.com: admin1@example.com
root@example.com: admin2@example.com

replace_rcpt static {
  entry root@example.com admin1@example.com admin2@example.com
}

replace_rcpt regexp "root@(.+)" "admin1@$1" "admin2@$1"

Update

Implemented the , so it also works with

root@example.com: admin1@example.com, admin2@example.com

So it is ready for review.

[raylee]

Regarding commas in email addresses, the last paragraph of RFC5322 section 3.4 reserves semicolons for this use case (separating email addresses and treating them as a group). eg:

root@example.com: admin1@example.com; admin2@example.com

[angelnu]

@foxcpp - are you ok with switching to semicolons? It is an easy change to do. Anything else that is needed before merging the PR?

Thanks @raylee for the suggestion!

[foxcpp]

What is the reason for using semicolons, though?
And no, I planned to go over & integrate your PR this weekend.

[angelnu]

What is the reason for using semicolons, though?

Mainly that, in theory, someone might want to use my,weird,email@example.org as email while nobody according to the RFC can use my;weird;email@example.org It is really a constructed case and therefore was looking for your thoughts on it.

And no, I planned to go over & integrate your PR this weekend.

Great - please notice that, as they both touch a common file the second PR you merge will need to be rebased. I did that in my fork as I have been running with both patches in my email server.

[foxcpp]

Mainly that, in theory, someone might want to use my,weird,email@example.org as email while nobody according to the RFC can use my;weird;email@example.org It is really a constructed case and therefore was looking for your thoughts on it.

Both addresses you provided are valid but would need to have local-part quoted. That is "my,weird,email"@example.org and "my;weird;email"@example.org. I would prefer to stick with commas and if handling of such addresses is necessary users should consider table implementations that do not have any restrictions (e.g. pulling aliases from a SQL table).

[foxcpp]

Implemented in bc42464.