Feature request: Include Unwritten Log Entries On Error

[jasonwadsworth]

Description of the feature request

It would be great if the logger could collect some unwritten logs (maybe a configurable amount that defaults to 10 or something) and if there is an error logged it would also write any logs that were not already written.

Problem statement

I've always hated that the debug logs are never available when I need them. They are typically turned off in a production system, but when there is an error I'd like to have information from those logs. I don't want them most of the time, so really only when an error occurs. Sampling isn't a good option because the error may be intermittent and the logs will not be present for the relevant time.

Summary of the feature

Basically, at the point when the decision is made whether to log it would either log (current functionality) or store the log. If there was an error log it would log the error as well as everything stored (and flush the store).

Code examples

Benefits for you and the wider AWS community

Faster time to resolution on errors

Describe alternatives you've considered

I've looked at doing something like this myself but haven't tried it yet. As I mentioned, sampling doesn't accomplish what I'm looking for.

Additional context

Related issues, RFCs

[flochaz]

Thx for your suggestion ! Interesting idea that can definitely help reducing logging cost while having a verbose output on error. we will look into it .

[ijemmy]

I like this idea very much. It addresses my main use case for sampling rate. But it's more cost-efficient and we don't miss anything.

I suppose usage will be like this?

import { Logger } from '@aws-lambda-powertools/logger';
const logger = new Logger({
   logAllLevelsOnError: true,
});

In this case, the client needs to make sure that they catch any error and print out the log to get it flushed out.

It can be used with Middy's error-logger, to ensure that uncaught error are always handled.

[jasonwadsworth]

In this case, the client needs to make sure that they catch any error and print out the log to get it flushed out.

Yes, it is definitely dependent on the code catching and logging an error.

Came across this repo, which is doing what I'm talking about here. Doesn't seem too bad in this example, but I haven't looked at the logging code in this repo to see how complex it might be here.

[ijemmy]

There is a blog about this. Useful for implementation reference: https://dev.to/aws-builders/saving-on-aws-lambda-amazon-cloudwatch-logs-costs-51od

Notice the clear() function or the logs can leak to the next Lambda call.

[buggy]

I really like this idea.

Could I suggest using levelOnError instead of logAllLevelsOnError? This would provide additional flexibility by allowing the developer to set the log level on error instead of assuming they want everything.

Use case:

1. Set level to WARN and levelOnError to INFO.
2. Use WARN for warnings that should always be logged
3. Use INFO for additional information (like the event) that will allow debugging when an error occurs
4. Use DEBUG for low level debugging (i.e. logging each iteration of a loop)

Why?
Frequently I only need a little extra information, like the event, to be logged on error. By allowing the developer to set the log level on error I can use INFO for the stuff I want to see when something goes wrong but still have DEBUG littered through my code for when I need to do development / debugging. This is particularly helpful if you only keep the last N items to be written on error because I want some things I log (INFO) but not others (DEBUG).

[dreamorosi]

Powertools for Python introduced a somewhat related feature that allows to emit logs when an exception is raised: see docs here.

[saragerion]

I really like this feature request and it's a very useful use case. I am in favour of adding this in our backlog.

[dreamorosi]

While the feature was implemented in Powertools for Python, at the moment it seems like it's unstable - aws-powertools/powertools-lambda-python/issues/1798

[jasonwadsworth]

Powertools for Python introduced a somewhat related feature that allows to emit logs when an exception is raised: see docs here.

I'm not sure this is the same thing that I'm asking for. It seems that the feature in the Python implementation is to log an exception that wasn't handled. That is very different than logging additional messages (from different log levels) when an error is logged. The issue they are having is related to capturing the uncaught exception. That would not be an issue for what I'm requesting.

[dreamorosi]

My bad, I confused / conflated the two use cases but after re-reading I agree that they are different. Adding back the original labels.

[heitorlessa]

Moving to backlog, as we've got some customer demand. One thing that would be suuuper helpful to prioritize is to have company names, as GitHub doesn't always have that info -- feel free to email us aws-powertools-maintainers@amazon.com if you don't feel comfortable sharing in public.

PS: adding Python issue too so they're linked.