[Feature] Integration with AWS secret Manager

[sooraj2589]

Description
Team is planning to integrate AWS secret manager with Saltstack for the secret management. Can we use aws secret manager as external pillar in saltstack?

Could any one please help with any documents or links?

[sooraj2589]

@garethgreenaway
can you please guide on this?

[garethgreenaway]

@sooraj2589 It doesn't appear that AWS secret manager is an option currently for external pillar.

[sooraj2589]

@garethgreenaway is there anyway we can use aws secret manager to integrate with saltsatck?

[sooraj2589]

@garethgreenaway

Do anyone has a suggestion?

[waynew]

You could write an external pillar and manage it yourself, but it doesn't look like we currently have anything built in

[jamesharr]

I'm considering prototyping this. How do you envision this working? Pull down all secrets? Pull down specific prefixes into specific areas of the pillar?

Would you be able to write a sample config for the ext_pillar, describe the data in AWS Secrets Manager, and describe what the resulting pillar would look like?

That might help push the discussion along a bit.

[natemellendorf]

We require this integration in order to use Salt.

Feel free to pass along any suggestions, else, I can provide a working example here (as a work around using something like ./_pillar/aws_secrets_manager.py) in a later comment.

currently having some troubles getting my salt master to see the new pillar, but will jump over to the Slack community this week to see what I’m doing wrong.

Once it’s working, we’ll ship a PR to salt. This has been a feature request for way too long ;)

Thanks in advance!

Note: Wouldn’t be shocked if this comment resurrects some earlier collaborators, who may have already built this. If so, please open a PR and let’s get SM integrated

[natemellendorf]

Ok, below is what I'm using right now for this.

Some assumptions are that your salt master is running on an EC2 instance and using an instance profile that permits access to your secrets/kms keys in secrets manager. It's pretty basic, but feel free to customize it to your needs.

Install boto3

# salt 3006.5
sudo salt-pip install boto3

/_pillar/aws_secrets_manager.py

import logging
import boto3

log = logging.getLogger(__name__)

def __virtual__():
    try:
        boto3.client("secretsmanager", region_name="us-east-1")
        return True
    except Exception:
        log.error("Failed to initialize AWS Secrets Manager client")
        return False

def ext_pillar(minion_id, pillar, *args, **kwargs):
    secrets = {}
    try:
        session = boto3.session.Session()
        for secret in args:
            client = session.client("secretsmanager", region_name=secret["region"])
            response = client.get_secret_value(SecretId=secret["arn"])
            secrets[secret["name"]] = response["SecretString"]
    except Exception as e:
        log.error(f"Failed to retrieve secret {secret['name']}: {str(e)}")

    return {"aws_secrets": secrets}

ext_pillar config on the master.
(I use GitFS, so I don't setup the ext_pillar root in the config)

ext_pillar:
  - aws_secrets_manager:
    - { name: example, arn: 'arn:aws:secretsmanager:<hidden>', region: 'us-east-1' }

I have to sync pillars after the above is in place, but that's due to how my salt-master is configured:

sudo salt-run saltutil.sync_pillar saltenv=BRANCH_NAME

Example of this working:

$ sudo salt '*' pillar.items saltenv=init
ip-10-10-10-100.ec2.internal:
    ----------
    aws_secrets:
        ----------
        example:
            {"Username":"foo","Password":"bar"}
ip-10-10-10-200.ec2.internal:
    ----------
    aws_secrets:
        ----------
        example:
            {"Username":"foo","Password":"bar"}

I'll work to get this into a PR, and the community can take it forward from there.
I suspect some users may want to specify aws profiles too, so maybe that can be added in.