Improve docs for vault iam auth

examples/vault-consul-ami/auth/sign-request.py

@@ -2,7 +2,9 @@
 # -What-------------------------------------------------------------------------
 # This script creates a request to the AWS Security Token Service API
 # with the action "GetCallerIdentity" and then signs the request using the
-# AWS credentials.
+# AWS credentials. It was modified from the python 2.x example published by
+# J. Thompson, the author of the Vault IAM auth method, at the vault support
+# mailing list. https://groups.google.com/forum/#!topic/vault-tool/Mfi3O-lW60I
 # -Why--------------------------------------------------------------------------
 # We are using python here instead of bash to take advantage of the boto3 library
 # which facilitates this work by an order of magnitude

examples/vault-ec2-auth/README.md

@@ -3,15 +3,17 @@
 This example shows how to use the metadata from an EC2 instance to authenticate
 to a [vault cluster][vault_cluster].
 
-Vault provides multiple [auth methods][auth_methods] such as Username & Password, GitHub
-etc. Among those methods you will find AWS. The way it works is that it
-understands [AWS][aws_auth] as a trusted third party, and relies on AWS itself for affirming
-if an authentication source such as an EC2 Instance or other resources like a
-Lambda Function are legitimate sources or not. Basically, if AWS trusts the
-origin, then so do we.
+Vault provides multiple ways to authenticate a human or machine to Vault, known as
+[auth methods][auth_methods]. For example, a human can authenticate with a Username
+& Password or with GitHub.
 
-There are currently two ways an AWS resource can authenticatate: `ec2` and `iam`. In
-this example, we will explore the first option.
+Among those methods you will find [AWS][aws_auth]. The way it works is that Vault
+understands AWS as a trusted third party, and relies on AWS itself for affirming
+if an authentication source such as an EC2 Instance or other resources like a Lambda
+Function are legitimate sources or not.
+
+There are currently two ways an AWS resource can authenticatate to Vault: `ec2` and `iam`.
+In this example, we demonstrate the [AWS EC2 Auth Method][ec2_auth].
 
 **Note**: To keep this example as simple to deploy and test as possible and because we are
 focusing on authentication, it deploys the Vault cluster into your default VPC and default subnets,
@@ -49,7 +51,7 @@ of the Vault nodes.
 
 EC2 auth is a process in which Vault relies on information about an EC2 instance
 trying to assume a desired authentication role. For different resources that are
-not EC2 instances, please refer to the `iam` auth method.
+not EC2 instances, please refer to the [`iam` auth method example][iam_example].
 
 The workflow is that the client trying to authenticate itself will send a
 signature in its login request, Vault verifies the signature with AWS, checks
@@ -176,7 +178,9 @@ To see the full script for authenticating check the [client user data script][us
 [create_role]: https://www.vaultproject.io/api/auth/aws/index.html#create-role
 [dnsmasq_module]: https://github.com/hashicorp/terraform-aws-consul/tree/master/modules/install-dnsmasq
 [dnsmasq]: http://www.thekelleys.org.uk/dnsmasq/doc.html
+[ec2_auth]: https://www.vaultproject.io/docs/auth/aws.html#ec2-auth-method
 [examples_helper]: https://github.com/hashicorp/terraform-aws-vault/tree/master/examples/vault-examples-helper/vault-examples-helper.sh
+[iam_example]: https://github.com/hashicorp/terraform-aws-vault/tree/master/examples/vault-iam-auth
 [instance_identity]: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-identity-documents.html
 [nonce]: https://www.vaultproject.io/docs/auth/aws.html#client-nonce
 [policies_doc]: https://www.vaultproject.io/docs/concepts/policies.html

examples/vault-iam-auth/README.md

@@ -3,24 +3,17 @@
 This example shows how to use the AWS IAM role attached to a resource to authenticate
 to a [vault cluster][vault_cluster].
 
-Vault provides multiple [auth methods][auth_methods] such as Username & Password, GitHub
-etc. Among those methods you will find AWS. The way it works is that it
-understands [AWS][aws_auth] as a trusted third party, and relies on AWS itself for affirming
-if an authentication source such as an EC2 Instance or other resources like a
-Lambda Function are legitimate sources or not. Basically, if AWS trusts the
-origin, then so do we.
-
 Vault provides multiple ways to authenticate a human or machine to Vault, known as
 [auth methods][auth_methods]. For example, a human can authenticate with a Username
-& Password or with GitHub. In this example, we demonstrate the [AWS Auth Method][aws_auth].
+& Password or with GitHub.
 
-The way it works is that Vault understands [AWS][aws_auth] as a trusted third party, and
-relies on AWS itself for affirming if an authentication source such as an EC2 Instance or
-other resources like a Lambda Function are legitimate sources or not. Basically, if AWS
-trusts the origin, then so do we.
+Among those methods you will find [AWS][aws_auth]. The way it works is that Vault
+understands AWS as a trusted third party, and relies on AWS itself for affirming
+if an authentication source such as an EC2 Instance or other resources like a Lambda
+Function are legitimate sources or not.
 
-There are currently two ways an AWS resource can authenticatate: `ec2` and `iam`. In
-this example, we will explore the second option.
+There are currently two ways an AWS resource can authenticatate to Vault: `ec2` and `iam`.
+In this example, we demonstrate the [AWS IAM Auth Method][iam_auth].
 
 **Note**: To keep this example as simple to deploy and test as possible and because we are
 focusing on authentication, it deploys the Vault cluster into your default VPC and default subnets,
@@ -57,28 +50,49 @@ of the Vault nodes.
 
 ### Vault Authentication using IAM user or role
 
-To read more about Vault IAM auth, refer to [Vault AWS Auth documentation][aws_auth].
+IAM auth is a process in which Vault leverages AWS STS (Security Token Service) to
+identify the AWS IAM principal (user or role) attached to an AWS resource such as
+an ECS Task or a Lambda Function that originates the login request. You can still
+use the `iam` method for EC2 instances attached to a role, like we do in this example,
+but for a login method specifically for EC2 instances, please refer to the
+[`ec2` auth method example][ec2_example].
+
+The workflow is that the client trying to authenticate will create a request to
+the method `GetCallerIdentity` of the AWS STS API (but not yet send it). This
+method basically answers the question "Who am I?". This request is then signed
+with the AWS credentials of the client. The signed result is then sent with the
+login request to the Vault Server. When the Vault server receives a login request
+with the `iam` method, it can execute the STS request without actually knowing
+the contents of the signed part. It then receives a response from STS identifying
+who signed it, which the Vault Server then can check against the ARN of the IAM
+principal bounded to a previously created Vault Role and decide if it should be
+allowed to authenticate or not.
+
+It is important to notice that, when the Vault Server receives this encrypted STS
+API request attached to a login request, to be able to execute it and perform the
+login, the cluster needs to have AWS Policies that will allow the cluster to execute
+the following actions: `iam:GetRole` or `iam:GetUser`, and `sts:GetCallerIdentity`.
+
 
 ### Configuring a Vault server
 
 Before we try to authenticate, we must be sure that the Vault Server is configured
 properly and prepared to receive requests. First, we must make sure the Vault server
 has been initialized (using `vault operator init`) and unsealed (using `vault operator unseal`).
 Next, we must enable Vault to support the AWS auth method (using `vault auth enable aws`).
-Finally, we must define the correct Vault Policies and Roles to declare which EC2
-Instances will have access to which resources in Vault.
+Finally, we must define the correct Vault Policies and Roles to declare which IAM
+Principals will have access to which resources in Vault.
 
 [Policies][policies_doc] are rules that grant or forbid access and actions to certain paths in
 Vault. With one or more policies on hand, you can then finally create the authentication role.
 
 When you create a Role in Vault, you define the Policies that are attached to that
-Role, how principals who assume that Role will re-authenticate, and for how long
-tokens issued for that role will be valid.
+Role and for how long tokens issued for that Role will be valid.
 
 In our example we create a simple Vault Policy that allows writing and reading from
 secrets in the path `secret` namespaced with the prefix `example_`, and then create
-a Vault Role that allows authentication from all instances with a specific `ami id`.
-You can read more about Role creation and check which other instance metadata you can
+a Vault Role that allows authentication from AWS resources attached to a certain IAM Role.
+You can read more about Role creation and check which other configurations you can
 use on auth [here][create_role].
 
 
@@ -100,20 +114,86 @@ vault write \
 See the whole example script at [user-data-vault.sh][user_data_vault].
 
 
-### Authenticating from an instance
+### Authenticating from a client
+
+#### With an HTTP request
+
+The [vault-consul-ami][vault_consul_ami] includes a [python script][py_sign] called
+`sign-request.py`. We use python here instead of bash to take advantage of the
+`boto3` AWS SDK library. This script is a modified version of the Python 2.x example
+posted by J. Thompson, the author of Vault's IAM auth method, at the Vault mailing
+list. It uses `boto3` to create a request to the AWS Security Token Service API
+with the action "GetCallerIdentity" and then signs the request using the AWS credentials.
+The same pattern should work with the AWS SDK in any other supported language such
+as Go, Java or Ruby, for example. For more details on the IAM auth method, there's
+a talk by J. Thompson called [Deep Dive into Vault's AWS Auth Backend][talk].
+
+```bash
+signed_request=$(python /opt/vault/scripts/sign-request.py vault.service.consul)
+```
+
+Once we have the encrypted request created by the python script, we can pass it
+in the body of the login request we will send to the Vault Server.
+
+```
+iam_request_url=$(echo $signed_request | jq -r .iam_request_url)
+iam_request_body=$(echo $signed_request | jq -r .iam_request_body)
+iam_request_headers=$(echo $signed_request | jq -r .iam_request_headers)
+
+
+data=$(cat <<EOF
+{
+  "role":"$VAULT_ROLE_NAME",
+  "iam_http_request_method": "POST",
+  "iam_request_url": "$iam_request_url",
+  "iam_request_body": "$iam_request_body",
+  "iam_request_headers": "$iam_request_headers"
+}
+EOF
+)
+
+curl --request POST --data "$data" https://vault.service.consul:8200/v1/auth/aws/login"
+```
+
+After sending the login request to Vault, Vault will execute the STS request to
+verify the client's identity with AWS and return a JSON object with your login
+information containing the `client_token`. The client token is an ephemeral token
+that you will send with your future operations requests to Vault. It can expire,
+be rotated, or become invalid for some other reason and you might be required to
+authenticate again.
 
-To see the full script for authenticating check the [client user data script][user_data_auth_client].
+To see the full example script for authenticating, check the [client user data script][user_data_auth_client].
 
 
+#### With Vault cli tool
+
+If vault cli is installed we can perform the login operation with it. The `VAULT_ADDR`
+environment variable has to be set and you need to have AWS credentials in some form.
+The vault cli will look for credentials configured in the standard locations such as
+environment variables, ~/.aws/credentials, IAM instance profile, or ECS task role, in
+that order. The way the it works is the same as with an HTTP request. The vault cli
+tool uses the golang AWS SDK to the create the STS API request and sign it with the
+credentials for you. It's important to note that the `role` value being passed is
+the Vault Role name, not the AWS IAM Role name.
+
+```bash
+export VAULT_ADDR=https://vault.service.consul:8200
+vault login -method=aws header_value=vault.service.consul role=vault-role-name
+```
+
 [ami]: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html
 [auth_methods]: https://www.vaultproject.io/docs/auth/index.html
 [aws_auth]:https://www.vaultproject.io/docs/auth/aws.html
 [consul_policy]: https://github.com/hashicorp/terraform-aws-consul/blob/master/modules/consul-iam-policies/main.tf
 [create_role]: https://www.vaultproject.io/api/auth/aws/index.html#create-role
 [dnsmasq_module]: https://github.com/hashicorp/terraform-aws-consul/tree/master/modules/install-dnsmasq
 [dnsmasq]: http://www.thekelleys.org.uk/dnsmasq/doc.html
+[ec2_example]: https://github.com/hashicorp/terraform-aws-vault/tree/master/examples/vault-ec2-auth
 [examples_helper]: https://github.com/hashicorp/terraform-aws-vault/tree/master/examples/vault-examples-helper/vault-examples-helper.sh
+[iam_auth]: https://www.vaultproject.io/docs/auth/aws.html#iam-auth-method
 [policies_doc]: https://www.vaultproject.io/docs/concepts/policies.html
+[py_sign]: https://github.com/hashicorp/terraform-aws-vault/tree/master/examples/vault-consul-ami/auth/sign-request.py
+[talk]: https://www.hashicorp.com/resources/deep-dive-vault-aws-auth-backend
 [user_data_auth_client]: https://github.com/hashicorp/terraform-aws-vault/tree/master/examples/vault-iam-auth/user-data-auth-client.sh
 [user_data_vault]: https://github.com/hashicorp/terraform-aws-vault/tree/master/examples/vault-iam-auth/user-data-vault.sh
 [vault_cluster]: https://github.com/hashicorp/terraform-aws-vault/tree/master/modules/vault-cluster

examples/vault-iam-auth/user-data-auth-client.sh

@@ -79,20 +79,11 @@ login_output=$(retry \
 
 
 # If vault cli is installed we can also perform these operations with vault cli
-# The necessary VAULT_TOKEN and VAULT_ADDR environment variables have to be set
-# This assumes you have AWS credentials configured in the standard locations AWS SDKs
-# search for credentials (environment variables (), ~/.aws/credentials, IAM instance profile,
-# or ECS task role, in that order).
-# export VAULT_TOKEN=$token
+# The VAULT_ADDR environment variable has to be set
+# This assumes you have AWS credentials configured in the standard locations
+# (environment variables, ~/.aws/credentials, IAM instance profile, or ECS task role, in that order).
 # export VAULT_ADDR=https://vault.service.consul:8200
-# vault login -method=aws header_value=vault.service.consul role=aws-role-name
-
-# Example of getting temporary credentials with iam role from instance metadata
-# The AWS session token is necessary here because these credentials are temporary
-# creds=$(curl http://169.254.169.254/latest/meta-data/iam/security-credentials/<AWS-IAM-ROLE-NAME>)
-# export AWS_ACCESS_KEY_ID=$(echo $creds | jq -r .AccessKeyId)
-# export AWS_SECRET_ACCESS_KEY=$(echo $creds | jq -r .SecretAccessKey)
-# export AWS_SESSION_TOKEN=$(echo $creds | jq -r .Token)
+# vault login -method=aws header_value=vault.service.consul role=vault-role-name
 
 
 # We can then use the client token from the login output once login was successful