elbv2: cannot use load balancer access logs when bucket is encrypted with KMS key

[josefaidt]

Describe the bug

I am running into almost exactly what is described in this previous issue and similar to this StackOverflow post where I have:

albFargateService.loadBalancer.logAccessLogs(bucket, 'alb-access')

And am receiving

8:58:34 AM | UPDATE_FAILED        | AWS::ElasticLoadBalancingV2::LoadBalancer     | BotAlbFargateServiceLB34
7310A1
Access Denied for bucket: <my-fake-bucket>. Please check S3bucket permission (Service: AmazonElasticL
oadBalancing; Status Code: 400; Error Code: InvalidConfigurationRequest; Request ID: e95cece4-c4af-4b8f-a6f1
-d1eb0d2e2778; Proxy: null)

Am I missing an additional piece of configuration for the bucket to enable access logs?

Expected Behavior

Access logs are enabled and emitted to bucket

Current Behavior

Running cdk deploy with the snippet shown in the section above we are presented with what appears to be the correct permissions

IAM Statement Changes
┌───┬──────────────────────┬────────┬──────────────────────┬──────────────────────┬────────────────────────┐
│   │ Resource             │ Effect │ Action               │ Principal            │ Condition              │
├───┼──────────────────────┼────────┼──────────────────────┼──────────────────────┼────────────────────────┤
│ + │ ${Bucket.Arn}        │ Allow  │ s3:GetBucketAcl      │ Service:delivery.log │                        │
│   │                      │        │                      │ s.amazonaws.com      │                        │
├───┼──────────────────────┼────────┼──────────────────────┼──────────────────────┼────────────────────────┤
│ + │ ${Bucket.Arn}/elb-ac │ Allow  │ s3:Abort*            │ AWS:arn:${AWS::Parti │                        │
│   │ cess/AWSLogs/1234567 │        │ s3:PutObject         │ tion}:iam::123456789 │                        │
│   │ 53096/*              │        │ s3:PutObjectLegalHol │ 021:root             │                        │
│   │                      │        │ d                    │                      │                        │
│   │                      │        │ s3:PutObjectRetentio │                      │                        │
│   │                      │        │ n                    │                      │                        │
│   │                      │        │ s3:PutObjectTagging  │                      │                        │
│   │                      │        │ s3:PutObjectVersionT │                      │                        │
│   │                      │        │ agging               │                      │                        │
│ + │ ${Bucket.Arn}/elb-ac │ Allow  │ s3:PutObject         │ Service:delivery.log │ "StringEquals": {      │
│   │ cess/AWSLogs/1234567 │        │                      │ s.amazonaws.com      │   "s3:x-amz-acl": "buc │
│   │ 53096/*              │        │                      │                      │ ket-owner-full-control │
│   │                      │        │                      │                      │ "                      │
│   │                      │        │                      │                      │ }                      │
├───┼──────────────────────┼────────┼──────────────────────┼──────────────────────┼────────────────────────┤
│ + │ ${Bucket/Key.Arn}    │ Allow  │ kms:Decrypt          │ AWS:arn:${AWS::Parti │                        │
│   │                      │        │ kms:Encrypt          │ tion}:iam::123456789 │                        │
│   │                      │        │ kms:GenerateDataKey* │ 021:root             │                        │
│   │                      │        │ kms:ReEncrypt*       │                      │                        │
└───┴──────────────────────┴────────┴──────────────────────┴──────────────────────┴────────────────────────┘

And we are receiving the following as CDK starts to deploy the changes

8:58:34 AM | UPDATE_FAILED        | AWS::ElasticLoadBalancingV2::LoadBalancer     | BotAlbFargateServiceLB34
7310A1
Access Denied for bucket: <my-fake-bucket>. Please check S3bucket permission (Service: AmazonElasticL
oadBalancing; Status Code: 400; Error Code: InvalidConfigurationRequest; Request ID: e95cece4-c4af-4b8f-a6f1
-d1eb0d2e2778; Proxy: null)

Reproduction Steps

https://github.com/josefaidt/cdk-accesslogs-repro

Our repository and code can be found here https://github.com/aws-amplify/discord-bot/blob/main/cdk/src/components/hey-amplify-app.ts#L250

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.39.1 (build f188fac)

Framework Version

2.39.1

Node.js Version

v18.7.0

OS

macos

Language

Typescript

Language Version

4.8.2

Other information

No response

[peterwoodworth]

I wasn't able to reproduce this with the snippet you've provided, have you made sure to specify the account and region of your stack's environment?

[josefaidt]

Hey @peterwoodworth sorry about that, I've taken some time to strip away the components of our app irrelevant to the reproduction and have created this repo https://github.com/josefaidt/cdk-accesslogs-repro

[peterwoodworth]

I was able to reproduce this with your repo, thanks!

Based on your setup, I was suspicious of the bucket configurations you had, so i removed them and deployed a bucket with all props set to default. This worked - so this lead me to find that the cause of this issue is with your bucket settings:

If you're using an encrypted bucket, make sure to use an Amazon S3-managed encryption key (SSE-S3). Other encryption methods, such as AWS KMS keys, are not supported for Network Load Balancer access logs.

I believe having your bucket be encrypted with a KMS key prevents you from using access logs here unfortunately. We could add a check for this in the logAccessLogs method to throw an error if bucket.encryptionKey is defined

aws-cdk/packages/@aws-cdk/aws-elasticloadbalancingv2/lib/shared/base-load-balancer.ts

Line 248 in 6ad48a3

public logAccessLogs(bucket: s3.IBucket, prefix?: string) {

[devnotfound]

Hey folks, I accidentally ran into this issue while trying to trace down something else. I have raised a PR for this here
Reviews welcome

[josefaidt]

Hey @peterwoodworth great callout, and the fix was very straightforward. By modifying the following bucket props I was able to enable access logs for my load balancer 🙂

- bucketKeyEnabled: true,
- encryption: s3.BucketEncryption.KMS,
+ encryption: s3.BucketEncryption.S3_MANAGED,

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.