Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assume Role still not working in provider #10507

Closed
nathancarmona opened this issue Oct 15, 2019 · 6 comments · Fixed by #14077
Closed

Assume Role still not working in provider #10507

nathancarmona opened this issue Oct 15, 2019 · 6 comments · Fixed by #14077
Assignees
Labels
bug Addresses a defect in current functionality. service/sts Issues and PRs that pertain to the sts service.
Milestone

Comments

@nathancarmona
Copy link

nathancarmona commented Oct 15, 2019

Trying to use a role as a part of the provider config is still not working properly. If the target role (the role to be assumed) has been assumed locally, then the provider configuration works as intended, but using a provider to assume a role in another provider still does not worked. This has already been documented in an issue but was closed, and the 2.32.0 release was supposed to fix it. @aeschright has asked that any lingering problems be reference in a new issue(s).

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

0.12.0

Affected Resource(s)

Terraform AWS Provider

Terraform Configuration Files

provider "aws" {
  alias   = "test"
  profile = "main-profile"
  region  = "us-west-2"
}

provider "aws" {
  alias  = "test_role"
  region = "us-west-2"
  version = "~> 2.32.0"
  assume_role {
    role_arn     = "arn:aws:iam::#########:role/assumable-role"
  }
}
data "aws_caller_identity" "current" {
  provider = aws.test_role
}

output "caller_arn" {
  value = "${data.aws_caller_identity.current.arn}"
}

Debug Output

https://gist.github.com/nathancarmona/2f87aa114db57828c908966a653051e3

Panic Output

N/A

Expected Behavior

output is arn:aws:iam::#########:role/assumable-role

Actual Behavior

Error: No valid credential sources found for AWS Provider.
        Please see https://terraform.io/docs/providers/aws/index.html for more information on
        providing credentials for the AWS Provider

  on main.tf line 7, in provider "aws":
   7: provider "aws" {

Steps to Reproduce

  1. terraform plan

Important Factoids

exporting the top level profile to the current shell session remediates the issue: export AWS_PROFILE=main-profile

(should note that shell env is otherwise free of AWS config:

$ printenv | grep AWS
AWS_PROFILE=main-profile

References

Version 2.32 did NOT fix the issue: #10491

Continuum of #8052

@ghost ghost added the service/sts Issues and PRs that pertain to the sts service. label Oct 15, 2019
@github-actions github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label Oct 15, 2019
@mhenniges
Copy link

@nathancarmona I have a config somewhat similar to yours, but was seeing different issues. I could not 'terraform init' against an s3 backend using an assumable role. After a lot of wrestling today, it is now working for me.

Based on your description, I'd suggest adding "profile = "main-profile" to your test_role provider definition. This should make it use main-profile to initially authenticate, and then the from there it will assume to assumable-role. This should have the same effect on that provider as setting the AWS_PROFILE env var, and because that solves it for you, I suspect this will work as well.

If that doesn't work, maybe my findings fighting with terraform init will be helpful; Long story short, using TF version 0.12.13, and aws provider version 2.36.0, things work fine, with a fresh start. I had made those changes in my file relatively early on, and was still failing to successfully init. What finally resolved it was deleting my .terraform directory, and making sure that there was nothing in the terraform cache in ~/.terraform.d/.

What I suspect was happening was that I had an older version of the AWS provider present (2.6 or 2.7 maybe), and that the provider version constraints are not checked and enforced early enough in the init process to handle upgrading the aws provider prior to attaching to the remote backend. So it would try with 2.7, and fail.

N.B. I've not checked the code to be sure.

Hope this helps!

@iniinikoski
Copy link

iniinikoski commented Nov 27, 2019

Hi,

I also had similar issues. What helped here is that I

  • dropped the "initial" alias off completely (meaning, two providers but only one "alias" defined; not sure if this really did anything but)
  • swapped from AWS profiles to access key - for debugging. I actually got the real error out when doing this - using profiles were "masking" the real problem

Hope this helps...

Terraform core version 0.12.16 & AWS provider is 2.40.0 - always latest and greatest ;)

Update: a second confirmation - the issue really persist when using the AWS_PROFILE - but not when using the API keys directly. Should be very repeatable also...

@aeschright aeschright added bug Addresses a defect in current functionality. and removed needs-triage Waiting for first response or review from a maintainer. labels Dec 13, 2019
@smalchow
Copy link

Dears, i had the same problem and investigate back an forth, i think i know where the issue is

First:

Terraform v0.12.18

  • provider.aws v2.42.0
  • provider.local v1.4.0
  • provider.null v2.1.2
  • provider.random v2.2.1
  • provider.template v2.1.2

and i use

  • AWS provider
  • S3 backend
  • assume role within .aws/config

I can verify, that already the "terraform init" fails with

Initializing modules...

Initializing the backend...



Error: No valid credential sources found for AWS Provider.
	Please see https://terraform.io/docs/providers/aws/index.html for more information on
	providing credentials for the AWS Provider

if the profile name that defines the role to assume contains a "-", the source_role can have a "-" that is no problem.

You can verify this by switching between profile names with and without "-"

@bflad bflad self-assigned this Jul 7, 2020
@bflad bflad added this to the v3.0.0 milestone Jul 7, 2020
bflad added a commit that referenced this issue Jul 7, 2020
Reference: #5018
Reference: #6913
Reference: #7333
Reference: #9236
Reference: #9869
Reference: #9898
Reference: #9962
Reference: #9986
Reference: #10507
Reference: #11429
Reference: #12236
Reference: #12727
Reference: #12815
Reference: #13057

Changes:

```
NOTES

* provider: Credential ordering has changed from static, environment, shared credentials, EC2 metadata, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) to static, environment, shared credentials, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata)
* provider: The `AWS_METADATA_TIMEOUT` environment variable no longer has any effect as we now depend on the default AWS Go SDK EC2 Metadata client timeout of one second with two retries

ENHANCEMENTS

* provider: Always enable shared configuration file support (no longer require `AWS_SDK_LOAD_CONFIG` environment variable)
* provider: Add `assume_role` configuration block `duration_seconds`, `policy_arns`, `tags`, and `transitive_tag_keys` arguments

BUG FIXES

* provider: Ensure configured STS endpoint is used during `AssumeRole` API calls
* provider: Prefer AWS shared configuration over EC2 metadata credentials by default
* provider: Prefer CodeBuild, ECS, EKS credentials over EC2 metadata credentials by default
```

Output from acceptance testing:

```
--- PASS: TestAccAWSProvider_Region_AwsCommercial (3.89s)
--- PASS: TestAccAWSProvider_Region_AwsGovCloudUs (3.90s)
--- PASS: TestAccAWSProvider_Region_AwsChina (3.99s)
--- PASS: TestAccAWSProvider_IgnoreTags_Keys_None (4.22s)
--- PASS: TestAccAWSProvider_IgnoreTags_KeyPrefixes_None (4.29s)
--- PASS: TestAccAWSProvider_IgnoreTags_KeyPrefixes_One (4.37s)
--- PASS: TestAccAWSProvider_IgnoreTags_KeyPrefixes_Multiple (4.38s)
--- PASS: TestAccAWSProvider_IgnoreTags_Keys_One (4.39s)
--- PASS: TestAccAWSProvider_IgnoreTags_EmptyConfigurationBlock (4.40s)
--- PASS: TestAccAWSProvider_IgnoreTags_Keys_Multiple (4.40s)
--- PASS: TestAccAWSProvider_Endpoints_Deprecated (4.42s)
--- PASS: TestAccAWSProvider_Endpoints (4.53s)
--- PASS: TestAccAWSProvider_AssumeRole_Empty (8.32s)
```
bflad added a commit that referenced this issue Jul 13, 2020
…14077)

* Update module hashicorp/aws-sdk-go-base to v0.5.0

* provider: Authentication updates for Terraform AWS Provider v3.0.0

Reference: #5018
Reference: #6913
Reference: #7333
Reference: #9236
Reference: #9869
Reference: #9898
Reference: #9962
Reference: #9986
Reference: #10507
Reference: #11429
Reference: #12236
Reference: #12727
Reference: #12815
Reference: #13057

Changes:

```
NOTES

* provider: Credential ordering has changed from static, environment, shared credentials, EC2 metadata, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) to static, environment, shared credentials, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata)
* provider: The `AWS_METADATA_TIMEOUT` environment variable no longer has any effect as we now depend on the default AWS Go SDK EC2 Metadata client timeout of one second with two retries

ENHANCEMENTS

* provider: Always enable shared configuration file support (no longer require `AWS_SDK_LOAD_CONFIG` environment variable)
* provider: Add `assume_role` configuration block `duration_seconds`, `policy_arns`, `tags`, and `transitive_tag_keys` arguments

BUG FIXES

* provider: Ensure configured STS endpoint is used during `AssumeRole` API calls
* provider: Prefer AWS shared configuration over EC2 metadata credentials by default
* provider: Prefer CodeBuild, ECS, EKS credentials over EC2 metadata credentials by default
```

Output from acceptance testing:

```
--- PASS: TestAccAWSProvider_Region_AwsCommercial (3.89s)
--- PASS: TestAccAWSProvider_Region_AwsGovCloudUs (3.90s)
--- PASS: TestAccAWSProvider_Region_AwsChina (3.99s)
--- PASS: TestAccAWSProvider_IgnoreTags_Keys_None (4.22s)
--- PASS: TestAccAWSProvider_IgnoreTags_KeyPrefixes_None (4.29s)
--- PASS: TestAccAWSProvider_IgnoreTags_KeyPrefixes_One (4.37s)
--- PASS: TestAccAWSProvider_IgnoreTags_KeyPrefixes_Multiple (4.38s)
--- PASS: TestAccAWSProvider_IgnoreTags_Keys_One (4.39s)
--- PASS: TestAccAWSProvider_IgnoreTags_EmptyConfigurationBlock (4.40s)
--- PASS: TestAccAWSProvider_IgnoreTags_Keys_Multiple (4.40s)
--- PASS: TestAccAWSProvider_Endpoints_Deprecated (4.42s)
--- PASS: TestAccAWSProvider_Endpoints (4.53s)
--- PASS: TestAccAWSProvider_AssumeRole_Empty (8.32s)
```

* docs/provider: Add authentication changes section to version 3 upgrade guide and remove pre-3.0 notes

Co-authored-by: Renovate Bot <bot@renovateapp.com>
@bflad
Copy link
Contributor

bflad commented Jul 13, 2020

Hi folks 👋 Version 3.0 of the Terraform AWS Provider will include a few authentication changes that should help in this case. Similar enhancements and fixes were applied to the Terraform S3 Backend (part of Terraform CLI) in version 0.13.0-beta2.

The Terraform AWS Provider major version update will release in the next two weeks or so. Please follow the v3.0.0 milestone for tracking the progress of that release. If you are still having trouble after updating when its released, please file a new issue. Thanks!

@ghost
Copy link

ghost commented Jul 31, 2020

This has been released in version 3.0.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ghost
Copy link

ghost commented Aug 12, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Aug 12, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. service/sts Issues and PRs that pertain to the sts service.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants