Collapse repeating log messages

[dan1338]

I think I've seen something similar in android studio logcat, where if an identical message gets logged more than n times in a row it gets collapsed instead of littering the log with redundant lines.

Here is a my simple proof of concept

quill_new.mp4

@odygrd What do you think about this functionality?

[odygrd]

Thanks for taking the time for a proof of concept. This functionality is nice, but implementing the counter isn't straightforward. Since the library writes to log files without revising them, implementing a counter requires waiting until encountering the first message that differs from the previous one.

For simply filtering messages without a counter, you can achieve this with a filter. An example that filters duplicate messages can be found here. Please note that you'll need the latest master for this functionality. Feel free to modify the Filter implementation to suit your specific requirements.

Let me know if the filtering example suits your needs

[dan1338]

Not sure if I got that correctly but

... library writes to log files without revising them,

pertains to not being able to modify previously logged entries in a file?

If so then I suppose I agree and I wouldn't advocate adding any file rewriting/seeking mechanism only for that purpose.

My specific objective was to prevent the console window from scrolling all the time, hence why the POC is implemented in ConsoleSink::write_log.

[dan1338]

Also, I've found that the simple solution used in the POC, that is to move one line up before writing the new message (CSI 1A \r) breaks when the message is wider than the terminal and thus takes up more than one line.

For the time being I'll be using my fork as that works good enough for me but it's definitely too rough around the edges to consider merging.

If you'd like to go ahead with something like this, it would be best to have a separate implementation for ConsoleSink that will have platform specific code which fetches the console dimensions and calculates the number of lines occupied by the repeated message as that seems to be the only way to know how many lines to clear.

[odygrd]

Yes all the current sinks just write to the file or console and never overwrite looking backwards. That happens inside write_log tho so it’s possible to seek back and overwrite but makes it more complicated.

I would recommend instead of maintaining a fork to just define your own Sink. You can simply derive from ConsoleSink and overwrite write_log there is an example here https://github.com/odygrd/quill/blob/master/examples/user_defined_sink.cpp

[marlowa]

I am also interested in this functionality. In the logger I am currently using this is something I found I had to add. I am used to syslog which does this automatically. If on my project we switch to quill it would be nice if this was provided.