Monitor AWS Glue API Access after AWS update on 16/12

[Khatraf]

User Story

As a DevOps engineer
I want to monitor AWS Glue API calls after the December 16 update
So that I can ensure there are no unexpected AccessDeniedException errors and verify that our IAM policies are functioning as intended.

Value / Purpose

Follow on from ticket #8549 - AWS has announced a change to the Glue BatchGet* APIs, which will result in AccessDeniedException errors if there is a Deny condition on the corresponding Get* operations in IAM policies. While our current policies are expected to work after this update, it’s important to monitor Glue API calls in CloudTrail post-update to ensure there are no unexpected AccessDeniedException errors.

Context / Background

Email from AWS: Hello,

We identified an issue with AWS Glue BatchGet APIs that requires your action. Currently, Glue BatchGet* APIs run successfully despite a Deny condition on one or more of the underlying Get operations. On December 16, 2024, we will deploy a fix for this to ensure BatchGet* APIs will fail with an AccessDeniedException if there is a Deny condition on one of the corresponding Get* operations. Your account has policies which include these contradicting statements. Please refer to the 'Affected resources' tab of your AWS Health Dashboard to see your impacted IAM resources.

You must update your policies to deny or allow AWS Glue Batch* APIs and their corresponding Get* API operations by this date. If you do not take action, the Batch API will not retrieve the resources of the Batch API call being made. Please refer to our "Actions, resources, and condition keys for AWS Glue" user guide for additional information [1].

The following is a list of the affected Glue BatchGet* APIs operations:

BatchGetDevEndpoints
BatchGetJobs
BatchGetBlueprints
BatchGetTriggers
BatchGetWorkflows

The following is a list of the affected Get* API operations:

GetDevEndpoints
GetJobs
GetBlueprints
GetTriggers
GetWorkflows

Useful Contacts

No response

Additional Information

No response

Definition of Done

• review CloudTrail logs for Glue API calls (BatchGet* and Get* operations) for data_engineering_policy & modernisation-platform-oidc-cicd-policy
• check for any AccessDeniedException errors in the logs.
• confirm that Glue BatchGet* APIs are functioning as expected.
• if any issues arise, investigate and update IAM policies as needed.

[Khatraf]

I have checked the logs, and tested on the CLI and didn't get AccessDenied errors so moving this ticket to closed as the permissions work as they should.