Debug and create test for datagov-smtp broker

[nickumia-reisys]

datagov-brokerpak-smtp @ main

How to reproduce

1. Create a datagov-smtp service
2. Bind service to catalog main app
3. Configure CKAN with smtp credentials
4. Perform action that would send email (e.g. harvesting)
5. Wait for email to be generated

Expected behavior

Email sent to desired recipient

Actual behavior

 2022-05-03T21:09:30.89-0400 [APP/TASK/harvester/0] ERR 2022-05-04 01:09:30,896 ERROR [ckan.lib.mailer] SMTPDataError(554, b"Access denied: User `arn:aws:iam::821341638715:user/cf/ses-b7a1cd34ddf3b0cf-csb--e3a67950-0aa6-4cb0-b272-76cfcf93cb8a' is not authorized to perform `ses:SendRawEmail' on resource `arn:aws:ses:us-west-2:821341638715:identity/aaron.borden@gsa.gov'")
   2022-05-03T21:09:30.89-0400 [APP/TASK/harvester/0] ERR Traceback (most recent call last):
   2022-05-03T21:09:30.89-0400 [APP/TASK/harvester/0] ERR File "/home/vcap/deps/1/python/lib/python3.7/site-packages/ckan/lib/mailer.py", line 107, in _mail_recipient
   2022-05-03T21:09:30.89-0400 [APP/TASK/harvester/0] ERR smtp_connection.sendmail(mail_from, [recipient_email], msg.as_string())
   2022-05-03T21:09:30.89-0400 [APP/TASK/harvester/0] ERR File "/home/vcap/deps/1/python/lib/python3.7/smtplib.py", line 893, in sendmail
   2022-05-03T21:09:30.89-0400 [APP/TASK/harvester/0] ERR raise SMTPDataError(code, resp)
   2022-05-03T21:09:30.89-0400 [APP/TASK/harvester/0] ERR smtplib.SMTPDataError: (554, b"Access denied: User `arn:aws:iam::821341638715:user/cf/ses-b7a1cd34ddf3b0cf-csb--e3a67950-0aa6-4cb0-b272-76cfcf93cb8a' is not authorized to perform `ses:SendRawEmail' on resource `arn:aws:ses:us-west-2:821341638715:identity/aaron.borden@gsa.gov'")

Sketch

• Investigate where Aaron's email is set
• Untie smtp brokerpak from his account (since his account is probably deactivated)
• Create a manual test in smtp brokerpak to see if it can send email
• Promote AWS SES Account from Sandbox to Production
• Push and release updates

[nickumia-reisys]

Verified in the AWS Console, the SES instance was verified,

[nickumia-reisys]

Ohhhh, I think I found the problem.. I think we should be doing this,

{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Effect":"Allow",
      "Action":[
        "ses:SendEmail",
        "ses:SendRawEmail"
      ],
      "Resource":"*"
    }
  ]
}

Instead of doing this,

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action":[
        "ses:SendEmail",
        "ses:SendRawEmail"
      ],
      "Resource": "${var.domain_arn}"
    }
  ]
}

The second one restricts sending emails only to itself as opposed to, to anywhere.

[nickumia-reisys]

Okay.. well... there's a few steps that need to happen here,

• Promote our SES Account to production from sandbox to be able to send real emails to external accounts
• This PR needs to be reviewed and merged to be able to send emails to anyone, instead of just to itself.
  • I'm writing a test to manually send an email.

My script will continue to fail with Email Address not verified until our SES Account is promoted to production,

This blocker is described here and I think requires a government person to be involved.

[nickumia-reisys]

The original error was trying to send an email to aaron's email, not use aaron's email to do something.

[nickumia-reisys]

Proof that sending emails works 😄

[mogul]

Ohhhh, I think I found the problem.. I think we should be doing this,

{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Effect":"Allow",
      "Action":[
        "ses:SendEmail",
        "ses:SendRawEmail"
      ],
      "Resource":"*"
    }
  ]
}

Instead of doing this,

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action":[
        "ses:SendEmail",
        "ses:SendRawEmail"
      ],
      "Resource": "${var.domain_arn}"
    }
  ]
}

The second one restricts sending emails only to itself as opposed to, to anywhere.

I don't think that's what is happening here... Restricting to just a particular target address looks different.

[mogul]

See also how the word "Resource" is applied here... It refers to the SES identity being used for sending, not the recipient.

Here's a list of brokered SES identities:

If you make the resource * instead of ${var.domain_arn} then you are saying that credentials brokered for one service instance can also be used to send from the identity associated with another instance. Definitely not what we want!

[nickumia-reisys]

Summary of work:

• Allow sendmail to anyone GSA-TTS/datagov-brokerpak-smtp#15

• Revert Resource * GSA-TTS/datagov-brokerpak-smtp#16

• A manual test was created in the datagov-brokerpak-smtp repo where an SMTP instance can be provisioned locally and then a user can use an email they have access to in order to check if an email can be properly sent from the provisioned SMTP service to their provided email.

• The updates were packaged into a PR for catalog.data.gov. However, it is being held off until we actually transition to cloud.gov and need to send emails. If it was merged prematurely, it could cause confusion for users as they would receive multiple sets of harvesting emails from different platforms.

• Add SMTP Service catalog.data.gov#457