Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

performance: move log group filters to invoker #77

Merged
merged 7 commits into from
Aug 6, 2024
Merged

performance: move log group filters to invoker #77

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
4a6bdf6
performance: move log group filters to invoker
dasfmi Aug 6, 2024
f2fdaae
remove LOG_GROUP_ vars from unsubscriber lambda
dasfmi Aug 6, 2024
0cf0e47
Prevented lambda from creating log group
dasfmi Aug 6, 2024
63aa46e
remove AxiomCloudWatchForwarderLambdaARN from unsubscriber
dasfmi Aug 6, 2024
9ac2f68
prevent listener for re-creating log group
dasfmi Aug 6, 2024
3e0ad38
create Forwarder policy to prevent re-creating log groups
dasfmi Aug 6, 2024
eeacf30
dx: set the user-agent header in forwarder
dasfmi Aug 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions cloudformation-stacks/subscriber.template.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,18 +18,22 @@ Parameters:
Type: String
Description: The ARN of the Axiom CloudWatch Forwarder Lambda function used to ship logs to Axiom.
AllowedPattern: ".+" # required
CloudWatchLogGroupsNames:
CloudWatchLogGroupNames:
Type: String
Description: A comma separated list of CloudWatch log groups to subscribe to.
Default: "" # all
CloudWatchLogGroupsPrefix:
CloudWatchLogGroupPrefix:
Type: String
Description: The Prefix of CloudWatch log groups to subscribe to.
Default: "" # all
CloudWatchLogGroupsPattern:
CloudWatchLogGroupPattern:
Type: String
Description: A regular expression pattern of CloudWatch log groups to subscribe to.
Default: "" # all
ForceUpdate:
Type: String
Description: Enter a random string to force an update of the stack.
Default: ""
Resources:
SubscriberPolicy:
Type: AWS::IAM::Policy
Expand Down Expand Up @@ -102,13 +106,14 @@ Resources:
Environment:
Variables:
AXIOM_CLOUDWATCH_FORWARDER_LAMBDA_ARN: !Ref "AxiomCloudWatchForwarderLambdaARN"
LOG_GROUP_NAMES: !Ref "CloudWatchLogGroupsNames"
LOG_GROUP_PREFIX: !Ref "CloudWatchLogGroupsPrefix"
LOG_GROUP_PATTERN: !Ref "CloudWatchLogGroupsPattern"
SubscriberInvoker:
Type: AWS::CloudFormation::CustomResource
DependsOn: SubscriberLambda
Version: "1.0"
Properties:
ServiceToken: !GetAtt SubscriberLambda.Arn
StackName: !Ref AWS::StackName
Force: !Ref "ForceUpdate"
CloudWatchLogGroupNames: !Ref "CloudWatchLogGroupNames"
CloudWatchLogGroupPrefix: !Ref "CloudWatchLogGroupPrefix"
CloudWatchLogGroupPattern: !Ref "CloudWatchLogGroupPattern"
17 changes: 11 additions & 6 deletions cloudformation-stacks/unsubscriber.template.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,18 +18,22 @@ Parameters:
Type: String
Description: The ARN of the Axiom CloudWatch Forwarder Lambda function used to forward logs to Axiom.
AllowedPattern: ".+" # required
CloudWatchLogGroupsNames:
CloudWatchLogGroupNames:
Type: String
Description: A comma-separated list of CloudWatch log groups to unsubscribe from.
Default: "" # all
CloudWatchLogGroupsPrefix:
CloudWatchLogGroupPrefix:
Type: String
Description: The prefix of CloudWatch log groups to unsubscribe from.
Default: "" # all
CloudWatchLogGroupsPattern:
CloudWatchLogGroupPattern:
Type: String
Description: A regular expression pattern for CloudWatch log groups to unsubscribe from.
Default: "" # all
ForceUpdate:
Type: String
Description: Enter a random string to force an update of the stack.
Default: ""
Resources:
UnsubscriberPolicy:
Type: AWS::IAM::Policy
Expand Down Expand Up @@ -99,13 +103,14 @@ Resources:
Environment:
Variables:
AXIOM_CLOUDWATCH_FORWARDER_LAMBDA_ARN: !Ref "AxiomCloudWatchForwarderLambdaARN"
LOG_GROUP_NAMES: !Ref "CloudWatchLogGroupsNames"
LOG_GROUP_PREFIX: !Ref "CloudWatchLogGroupsPrefix"
LOG_GROUP_PATTERN: !Ref "CloudWatchLogGroupsPattern"
UnsubscriberInvoker:
Type: AWS::CloudFormation::CustomResource
DependsOn: UnsubscriberLambda
Version: "1.0"
Properties:
ServiceToken: !GetAtt UnsubscriberLambda.Arn
StackName: !Ref AWS::StackName
Force: !Ref "ForceUpdate"
CloudWatchLogGroupNames: !Ref "CloudWatchLogGroupNames"
CloudWatchLogGroupPrefix: !Ref "CloudWatchLogGroupPrefix"
CloudWatchLogGroupPattern: !Ref "CloudWatchLogGroupPattern"
8 changes: 5 additions & 3 deletions subscriber.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,7 @@
axiom_cloudwatch_forwarder_lambda_arn = os.getenv(
"AXIOM_CLOUDWATCH_FORWARDER_LAMBDA_ARN"
)
log_group_names = os.getenv("LOG_GROUP_NAMES", None)
log_group_prefix = os.getenv("LOG_GROUP_PREFIX", None)
log_group_pattern = os.getenv("LOG_GROUP_PATTERN", None)

log_groups_return_limit = 50


Expand Down Expand Up @@ -97,6 +95,10 @@ def lambda_handler(event: dict, context=None):
)
return

log_group_names = event["ResourceProperties"]["CloudWatchLogGroupNames"]
log_group_prefix = event["ResourceProperties"]["CloudWatchLogGroupPrefix"]
log_group_pattern = event["ResourceProperties"]["CloudWatchLogGroupPattern"]

if (
axiom_cloudwatch_forwarder_lambda_arn is None
or axiom_cloudwatch_forwarder_lambda_arn == ""
Expand Down
7 changes: 4 additions & 3 deletions unsubscriber.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,6 @@
axiom_cloudwatch_forwarder_lambda_arn = os.getenv(
"AXIOM_CLOUDWATCH_FORWARDER_LAMBDA_ARN"
)
log_group_names = os.getenv("LOG_GROUP_NAMES", None)
log_group_prefix = os.getenv("LOG_GROUP_PREFIX", None)
log_group_pattern = os.getenv("LOG_GROUP_PATTERN", None)
log_groups_return_limit = 50


Expand Down Expand Up @@ -92,6 +89,10 @@ def lambda_handler(event: dict, context=None):
cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData)
return

log_group_names = event["ResourceProperties"]["CloudWatchLogGroupNames"]
log_group_prefix = event["ResourceProperties"]["CloudWatchLogGroupPrefix"]
log_group_pattern = event["ResourceProperties"]["CloudWatchLogGroupPattern"]

forwarder_lambda_group_name = (
"/aws/lambda/" + axiom_cloudwatch_forwarder_lambda_arn.split(":")[-1]
)
Expand Down
Loading