[Logs UI] Proposal: show log entries hidden by filter

[weltenwort]

Summary

This is a proposal to add the UI capability to temporarily show log entries that have been hidden by the filter.

Rationale

It is a common workflow during investigations to narrow down the set of log entries by a particular criterion (such as log level or presence of a term in the message) and, once a log entry of interest has been found, to inspect other log entries in close temporal proximity. While the first part of that workflow is easily supported by the query bar in the Logs UI, the second part is more difficult.

This proposal aims to improve that second part of the workflow by displaying the presence of filtered-out log entries in a very condensed form with the option of incrementally expanding these filtered-out log entries for the purpose of investigating the context.

Showing the presence of filtered-out log entries

To mark the presence of filtered-out log entries, a small sign could be placed at the left edge of the stream. In order to not take up too much space, that sign could be overlayed on the edge between two adjacent log entries without influencing the vertical flow. An interaction like clicking on the marker could trigger expansion into a segment of fully rendered log entries (see below).

Showing the filtered-out log entries

When interacted with, the markers could expand into a segment that shows (some of) the intermediate log entries that were hidden by the filter. That segment could be marked in some way (e.g. a different background) to signal the fact that the entries therein don't match the filter criteria.

Since there could be an arbitrarily large number of log entries in that segment, they need to be loaded incrementally. At which edge of the segment interval the log entries are loaded (old, new, both) could depend on the scrolling position or on the specific user interaction.

The segment of filtered-out log entries could be collapsible again to continue the workflow on the filtered set of log entries.

Querying for filtered-out log entries

The queries required to determine the number of filtered-out entries and to fetch the entries themselves have a performance cost, because the tiebreaker field _doc (or in the future _seq_no) is not available for filtering in the query. A safe timestamp range filter in combination with a (timestamp, tiebreaker) search_after will probably be required. To mitigate the impact on the website performance, these could be performed incrementally after the shown log entries have been loaded.

Mockups

[elasticmachine]

Pinging @elastic/infra-logs-ui

[weltenwort]

cc @makwarth @roncohen @tbragin

[snide]

I like this. How does the minimap adjust? Does it? Would it give a highlight signifier for the filtered portions?

[weltenwort]

@snide Good question! In general we might want to show the ratio of unfiltered vs filtered log entries in the minimap. I've updated the mockups to include something like that.

Would there be any value in marking the temporarily expanded entries there? I can't come up with a use-case 🤔

[snide]

Would there be any value in marking the temporarily expanded entries there? I can't come up with a use-case

No. I thought the same. I think what you really want to show is just the "hits" against the filter in the minimap. Whatever you do in the full log view shouldn't matter as much.

[roncohen]

@weltenwort thanks for this proposal. As a user, I'd search for something like "severy > 3" within a specific host (host.name: "myhost"). The "filtered entries" as proposed here would cover everything, not just context for host.name: "myhost". If I have hundred of hosts, the hidden context would be very large in many cases and often useless. Other tools have a concept of log stream, meaning a stream of logs from a given host/pod/container. These three are already first level concepts in the logs explorer, so we should make use of them here. I suggest that if either of these are in the filter bar, apply those filter also for the hidden context. The text should then say something like "show more entries for this host/pod/container" depending on the filters already applied.

[roncohen]

I'm also curious about how this could be implemented in a performant way. The hidden documents could be in the thousands. Short of pulling all of the documents, how would you find out which two rows to put the hidden context between?

[roncohen]

cc also @jasonrhodes for input

[weltenwort]

The hidden documents could be in the thousands. Short of pulling all of the documents, how would you find out which two rows to put the hidden context between?

I would use the same querying technique I'm using already. The log viewer uses an "anchor" document and fetches a number of documents before and after using a combination of sort, search_after and limit.

edit: and a range filter for early query rewriting, of course

[weltenwort]

I suggest that if either of these [host/pod/container filters] are in the filter bar, apply those filter also for the hidden context. The text should then say something like "show more entries for this host/pod/container" depending on the filters already applied.

Yes, that could work. We would probably only take into account the topmost conjunction of the (arbitrarily nested) boolean query, right?

Another option would be to go back to some of the original decisions made while preparing for the first release: There was a consideration of providing specific filter controls like host, container and pod. The product decision was ultimately to just use the query bar because it's the most flexible option, but we could revise that decision to include some filter controls tailored to the use-case.

[roncohen]

Another option would be to go back to some of the original decisions made while preparing for the first release: There was a consideration of providing specific filter controls like host, container and pod. The product decision was ultimately to just use the query bar because it's the most flexible option, but we could revise that decision to include some filter controls tailored to the use-case.

I think it's definitely worth considering elevating those specific filter options. APM is looking do do something on the left side: #34922

[weltenwort]

Yes, that looks nice. We could do something like that in the logs ui too unless we think it takes up too much horizontal space.

Alternatively we could use something akin to the structured filters pattern that the design team developed for discover:

[roncohen]

i share your concern about taking up horizontal real estate. Maybe it could roll out? would be great to get some input from @hbharding or @formgeist

[formgeist]

We explored a slew of ideas related to the KQL bar and filters in #30943. We've decided (for now) to go with a sidebar that can display all available filters and more to come along with selected values. As @roncohen mentioned, it could be that we add a collapsed state so it can work as a popover and perhaps only show the selected values from each filter in a summary. It would still demand horizontal space, especially if many filters are selected all at once, but I suppose that's better than not knowing what's selected.

I know that other teams are having thoughts about the filters and KQL bar and its limitations in displaying selected values, so perhaps @hbharding and I can look at some modifications to the APM approach and propose a solution that might match something that will also work as a pattern in Kibana in the longer term.

[weltenwort]

Thanks for providing that decision record ❤️ I agree that the vertical filter menu to the side has some significant usability and scalability advantages, which might make it worth taking up more horizontal space. It's just an aspect to keep in mind.

[jasonrhodes]

Wow, I totally missed this conversation 20 days ago, sorry! I think this is a really valuable feature and one we should start to flesh out more for implementation. There's been some talk from product about a similar, related feature known as "View in Context" which I also think is worth exploring. The motivation is exactly the same as this one but it will have different mockups and possibly a different set of questions, so I'd like to separate that out into a new issue so this history doesn't get confused in case we need to come back to it. I'll create the related issue and link back to this one so we can make a decision once we understand the options.

[blommaep]

Related to this: next to the suggested folding, imho, there is a value in showing results after filter x,y,z but... additionally highlighting in this result (by color/bold/...) the subset matching filter a,b,c. The first filter x,y,z, to filter out relevant log flows (e.g. that might have ended up in different filesin "old days", the second to search on e.g. text, level or context fields, then to immediately read the logs around.

A continuous forward/backward to "view surrounding" is not as efficient and does not take over filter x,y,z making the work extensive.
A browser search won't work if the logs entries are large and need "expand", i.e. risk to miss things.

Posting here, because to my impression, the mechanism behind is similar: all data is queried from es, but differentiated in the display (?).

[weltenwort]

@blommaep thank you for chiming in. What you describe sounds similar to the highlighting functionality added with #40010. It's a bit simplistic right now, but we're currently taking another look at it in the context of #28492.

[elasticmachine]

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)