Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Avoid flushing metrics when no metrics to flush #3023

Closed
1 of 2 tasks
bml1g12 opened this issue Sep 4, 2024 · 7 comments · Fixed by #3044
Closed
1 of 2 tasks

Feature request: Avoid flushing metrics when no metrics to flush #3023

bml1g12 opened this issue Sep 4, 2024 · 7 comments · Fixed by #3044
Assignees
Labels
completed This item is complete and has been merged/shipped feature-request This item refers to a feature request for an existing or new utility metrics This item relates to the Metrics Utility

Comments

@bml1g12
Copy link

bml1g12 commented Sep 4, 2024

Use case

We are using

    "@middy/core": "^4.0.2",
    "@aws-lambda-powertools/metrics": "^1.0.1",

Metrics with this setup:

const log = new Logger({
  // POWERTOOLS_LOG_LEVEL environment variable sets log level
  serviceName: config.serviceName,
  persistentLogAttributes: {
    release: config.releaseVersion,
    environment: config.stage,
  },
})

const metrics = new Metrics({
  serviceName: config.serviceName,
  namespace: "xxx",
  defaultDimensions: {
    aws_account_id: config.awsAccountId,
    aws_region: config.region,
  },
})

const tracer = new Tracer({ serviceName: config.serviceName })
// You can use tracer.provider attribute to access all methods provided by the AWS X-Ray SDK.
// This is useful when you need a feature available in X-Ray that is not available in the
// Tracer utility, for example, SQL queries tracing, or a custom logger.
tracer.provider.setLogger(log)

const withMiddy = (handler: Handler) =>
  middy(handler)
    .use(captureLambdaHandler(tracer))
    .use(logMetrics(metrics, { captureColdStartMetric: false }))

And if we do not log any metrics cloudwatch logs report what I think is a redundant log that makes logs a little harder to read through and of course expends a tiny amount of money to emit:


2024-09-04T12:33:44.468Z
{
    "_aws": {
        "Timestamp": 1725453224468,
        "CloudWatchMetrics": [
            {
                "Namespace": "xxx",
                "Dimensions": [
                    [
                        "service",
                        "aws_account_id",
                        "aws_region"
                    ]
                ],
                "Metrics": []
            }
        ]
    },
    "service": "xxx",
    "aws_account_id": "xxx",
    "aws_region": "ap-northeast-1"
}

Solution/User Experience

I believe the AWS Powertools Library for Python does not produce such empty logs, I think the ideal behaviour would be to omit these logs entirely if there's no metrics to log?

Alternative solutions

We could of course omit including the library entirely for each lambda, but given the middleware is often used to apply generic boilerplate to all lambdas in a repo, I don't think that would be appropriate if we wanted to only report metrics for some lambdas.

Acknowledgment

Future readers

Please react with 👍 and your use case to help us understand customer demand.

@bml1g12 bml1g12 added feature-request This item refers to a feature request for an existing or new utility triage This item has not been triaged by a maintainer, please wait labels Sep 4, 2024
Copy link

boring-cyborg bot commented Sep 4, 2024

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #typescript channel on our Powertools for AWS Lambda Discord: Invite link

@aws-powertools aws-powertools deleted a comment Sep 4, 2024
@dreamorosi
Copy link
Contributor

Hi @bml1g12, thank you for opening this feature request.

From a first look, it seems that this could be related or at least have a good amount of overlap with #2036, would you mind taking a look and see if this is the case?

If yes, I think we could close this and use the other. That issue hasn't been updated in a few months but I now see that in addition to this issue, there's a decent amount of interest by looking at the 👍, so it could be a good time to repriotize it.

As a side note, v1 of Powertools for AWS Lambda (TypeScript) has reached end of life this week, so I would advise to consider upgrading to a more recent version in the v2.x line.

@github-staff github-staff deleted a comment from bml1g12 Sep 4, 2024
@dreamorosi dreamorosi added metrics This item relates to the Metrics Utility discussing The issue needs to be discussed, elaborated, or refined need-response This item requires a response from a customer and will considered stale after 2 weeks and removed triage This item has not been triaged by a maintainer, please wait labels Sep 4, 2024
@bml1g12
Copy link
Author

bml1g12 commented Sep 5, 2024

Hi @bml1g12, thank you for opening this feature request.

From a first look, it seems that this could be related or at least have a good amount of overlap with #2036, would you mind taking a look and see if this is the case?

If yes, I think we could close this and use the other. That issue hasn't been updated in a few months but I now see that in addition to this issue, there's a decent amount of interest by looking at the 👍, so it could be a good time to repriotize it.

As a side note, v1 of Powertools for AWS Lambda (TypeScript) has reached end of life this week, so I would advise to consider upgrading to a more recent version in the v2.x line.

Thanks @dreamorosi , I think that issue basically covers this request so will close it, although I notice in your response there you focus on the topic of removing the warning which is a good point and something I'd like to be able to hide; however I should note my main concern is the fact that we log an empty metric on top of that; so even if we disable the warning, we'd want to also disable the empty metric log.

Taking Powertools Python as an example, they log this warning using the warning standard module. This gives customers a way to suppress warnings without having to change their code.

In Powertools Python they do not log an empty metric at all, just the warning

@bml1g12 bml1g12 closed this as completed Sep 5, 2024
Copy link
Contributor

github-actions bot commented Sep 5, 2024

⚠️ COMMENT VISIBILITY WARNING ⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

@dreamorosi dreamorosi added confirmed The scope is clear, ready for implementation and removed discussing The issue needs to be discussed, elaborated, or refined need-response This item requires a response from a customer and will considered stale after 2 weeks labels Sep 5, 2024
@dreamorosi dreamorosi reopened this Sep 5, 2024
@dreamorosi
Copy link
Contributor

Hi @bml1g12, thanks for the additional detail.

I didn't understand it at first but in that case I agree with you and we shouldn't log the empty metric.

I've reopened the issue so I can take a look at it irrespective of the other issue, which might take longer.

Copy link
Contributor

⚠️ COMMENT VISIBILITY WARNING ⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

@github-actions github-actions bot added pending-release This item has been merged and will be released soon and removed confirmed The scope is clear, ready for implementation labels Sep 12, 2024
Copy link
Contributor

This is now released under v2.8.0 version!

@github-actions github-actions bot added completed This item is complete and has been merged/shipped and removed pending-release This item has been merged and will be released soon labels Sep 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
completed This item is complete and has been merged/shipped feature-request This item refers to a feature request for an existing or new utility metrics This item relates to the Metrics Utility
Projects
Development

Successfully merging a pull request may close this issue.

2 participants