Cannot use Role ARN to auth for S3 bucket target

[CollectiveUnicorn]

When attempting to setup the S3 target auth using only AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE the configuration seems to fail with the error [ERROR] S3.AccessKeyID has not been declared due to accessKeyID and secretAccessKey not being allowed to be blank.

From my investigations, it seems the AWS config appears to be setup properly, but the first provider in the ChainProvider is the StaticProvider which only throws an error if the credential values (accessKeyID and secretAccessKey) are blank. Which cannot happen due to the SecurityHub error that is thrown when these values are empty strings. Because StaticProvider doesn't actually check the credentials, dummy values don't lead to a configuration error so the configuration completes and the application attempts to auth using them and fails.

Policy Reporter Version: 2.12.0

Has anyone encountered this issue or have a workaround? Alternatively, am I possibly misunderstanding the configuration steps?

[fjogeleit]

Thanks for reporting.
Will check it

[fjogeleit]

I created a fixed version 2.15.3.

Can you overwrite the tag and check if its working? If so I will release a new chart version.

[CollectiveUnicorn]

@fjogeleit The issue with not being able to provide a blank accessKeyID and secretAccessKey has been resolved. Still running into some issues with the AWS config (not using the AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE to create a valid provider) but might need a few more days to determine if that's on our end.

[fjogeleit]

Okay, thanks for your feedback. Let me know if you encounter any other issue I can help with.

[CollectiveUnicorn]

Thanks for your help so far. If you have a way to confirm that the WebIdentityRoleProvider works properly when AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE is specified in env vars, that would help with troubleshooting.

I've verified that the env vars are present and can be used to read and write from the s3 bucket using the AWS cli. But policy-reporter returns the following error error S3: S3 Upload error {"error": "NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors"}.

Inside the chart I have the following values set:

• target.s3.region
• target.s3.endpoint
• target.s3.bucket

With the AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE being set in the env variables outside the chart values.

[fjogeleit]

Okay, I will see if I can test it somehow. I have no aws cluster so I need to set the env vars manually somehow.

[fjogeleit]

i was not able to test a success case because I was not able to set up valid credentials without an ECS cluster but I changed the provider logic, so you should get a more detailed error message why your values are not working. Hope this gives more insights. I published it as version 2.15.4

[CollectiveUnicorn]

After much prodding, I think this code is the source of the issue

if config.Endpoint == "" {
		return nil
	}

when using go api for AWS with IRSA, the API does something to determine the correct credentials and endpoint when "Endpoint" is left blank. Attempting to specify a value for the endpoint causes some other credential related things not to be setup properly leading to failures.

I wrote some code to test this:

func basic_s3_uploader() {
	s3Config := &aws.Config{
		Region: aws.String("us-east-1"),
		// Endpoint: aws.String("s3.amazonaws.com"),
		LogLevel: aws.LogLevel(aws.LogDebugWithHTTPBody),
	}
	s3Session, sessionErr := session.NewSession(s3Config)

	// svc := s3.New(s3Session)
	// svc.Handlers.Send.PushFront(func(r *request.Request) {
	// 	// Print the endpoint URL
	// 	fmt.Println("Endpoint:", r.HTTPRequest.URL.Host)
	// })
	// otherInput := &s3.ListBucketsInput{}
	// _, err := svc.ListBuckets(otherInput)

	// if err != nil {
	// 	print("We got a nil")
	// }

	if sessionErr != nil {
		print("This session has failed with: " + sessionErr.Error())
	}

	uploader := s3manager.NewUploader(s3Session)

	filename := "test-file-to-upload.txt"
	file, err := os.Open(filename)
	if err != nil {
		fmt.Println("Failed to open file", filename, err)
		os.Exit(1)
	}
	defer file.Close()

	input := &s3manager.UploadInput{
		Bucket: aws.String("my-special-bucket"), // bucket's name
		Key:    aws.String(filename),                                       // files destination location
		Body:   file,                                                       // content of the file
	}

	output, err := uploader.UploadWithContext(context.Background(), input)
	print(context.Background())
	print(uploader)

	if err != nil {
		print("Error: " + err.Error())
	}

	print("Output: " + output.Location)
}

I also isolated out the code in aws.go and had no issue uploading to the s3 bucket. Unfortunately I am unable to get a locally compiled version of the entire application running in my k8s cluster to 100% confirm that this will fix it. But the tests seem to heavily imply that's the case.

So if you're able to get an image up without that check on "Endpoint" which restricts it from being blank then i should be able to confirm whether that's the case.

[fjogeleit]

Hey, thanks for your investigation. Sure I will build an image with your changes

[bushong1]

@fjogeleit any updates on this?

[fjogeleit]

Sorry for the delay, busy week. Will have time tomorrow for this.

[fjogeleit]

I just did it directly and created a tag 2.16.2. So you can try it after the build is done.