Incident list filters versus notification filters: they are not the same

[hmpf]

The default for filtering incidents is: if no filter, show me everything. Incidents are opt-out.

The default for filtering notifications is: if no filter, notify about nothing. Notifications are opt-in, there's currently no way to say "I want everything, gimme the firehose".

What does it really mean when acks are filtered out

An ack is two things: a state of the incident, and a specific type of event.

1. Whether an incident is currently acked can be found by incident.acked, and filtering on "acked".
2. Whether an event is of type acked is done by checking if the event.type is set to Event.Type.ACKED. The filtering is a bit broken but happens in FilterWrapper.event_fits(). If the filter has event_type set to one and only one of the Event.Types type, the event fits. We want to support multiple types.

In the frontend, we can hide or show only acked incidents by toggling acked. It is currently not possible to hide acking-events, and I am not sure whether that makes sense at all for the incident list, since it shows the state of incidents, not events. We cannot really show with a live update of the incidents list the effect of hiding or choosing a specific event.

So what happens to notifications if we filter away acked incidents, but want, say, events of type Event.Type.CLOSE even for incidents that are acked?

The incident-state of "open" is the same as the event-types "INCIDENT_START", "INCIDENT_REOPEN" while the incident-state of "closed" is the same as the event-types "INCIDENT_END" and "CLOSE". An acked incident can coexist with any of these.

[hmpf]

I think the real problem is caused by having a single mechanism make filters both for the incident list and the notification events.

Adding frontend-support for filtering on event-types won't be visible in the incidents list.

If we split out sigh notification filters to a separate page we can have an events-list that shows the effects of notification-filters, including event-type.

We could change the filter for event types to tristates (optional bool).

{
    "event_types": {
        Event.Type.INCIDENT_CHANGE: False,
        Event.Type.ACKNOWLEDGE: False,
    }
}

The above filter would never send notifications for change-events (so far: changing ticket-url) or ack-events regardless of the incident list. The above filter would be ignored by the incidents-list.

What would having a filter {"acked": False, event_types: {Event.Type.INCIDENT_CLOSE: True}} mean for notifications? Following how filters currently work, this would first remove all acked incidents (and all their events) and only show close-events, not any of the others. Should the filter {"event_types": {Event.Type.ACKNOWLEDGEMENT: False}} mean opt-in to all events except acks or would it be necessary to set every event-type we do want to True? Or maybe have a list of "do want" vs. "do not want"?

Regardless, code-changes are needed to support notification of CLOSE-events that are on acked incidents.

[hmpf]

Another question: is the fallback filter for the incident-list, the notifications, or both? I think the idea was that it was for the notifications.

[lunkwill42]

The original problem

The original problem (which #293 was supposed to be a short-term workaround for) was this:

The CNaaS team wants to be notified by SMS or email when some incidents are opened and later when they are resolved or closed. They do not care about other events that occur on the matching incidents.

However, filters (currently) only filter on incident attributes, not on event attributes (I.e. i believe they can be made to filter on event attributes, but this is not available from the UI). Since notifications are based on events, every event that occurs on a matching incident is sent as a notification. This includes things like the event for someone ACKing an incident, or the CHANGE event that occurs when someone adds a ticket URL to the matching incident. These notifications are basically spam for the CNaaS team.

Since changing how notification filters work is a "big task", we opted to do a short-term workaround by adding a fallback filter to the config file, which could filter away certain types of events from all notifications. However, I think #293 was based on some orally communicated design spec, not a written one (there is no associated issue describing the requested feature), and therefore resulted in some misunderstandings.

Incident filter expressions vs event filter expressions

What we need is some way to specify an event type match expression (#466 and #699 are about this). However, as @hmpf states, filter expressions matching on event attributes have no effect when it comes to filtering the incident list in the UI. Their only use is for filtering during notification.

In other words, adding UI widgets to enable the user to filter for event attributes on the incidents list page makes no sense - this would likely only confuse the user, since it would appear to have no effect on the incidents list.

What I previously remember discussing (and it doesn't look like we wrote it down anywhere, sadly) was perhaps adding some UI widget to do event-specific filtering from the notification profile page instead. Something like "subscribe to filter CNaas during work hours, but filter out any events that aren't of type X, Y or Z" Unfortunately, I think #466 and #699 are all about adding this functionality to existing incident filters, and are perhaps short of the mark when this idea is considered.

[hmpf]

Ditto @podliashanyk!

[hmpf]

Is "open" her a combination of "start" and "reopen"? Since there is no "open"-event. Ditto "close", does it stand in for "close" and "end"?

[lunkwill42]

By "OPEN" here, I mean "START". I didn't have the event names handy when I made it 😆
And no, "CLOSE" and "END" are separate types that mean different things

(this is just a mockup, the option in the event dropdown are mean to correspond to all allowable values of Event.type)

[hmpf]

I was really asking if you wanted to combine some for ease of use.

[lunkwill42]

Nah, I want the list to explicitly be just the allowed values for Event.type. Simpler to implement, and puts the onus on the user to familiarize themselves with the types.

[hmpf]

"None" also means "missing from Filter.filter".

API serialization has all keys.

	value	incidents	events
no filter		show all	send none
open	True	show only open	send only open
	False	show only closed	send only closed
	None	fallback	fallback
stateful	True	show only stateful	send only stateful
	False	show only stateless	send only stateless
	None	fallback	fallback
acked	True	show only acked	send only acked
	False	show only unacked	send only unacked
	None	fallback	fallback
maxlevel	1..5	show [1, x]	send [1, x]
tag	[]	fallback	fallback
	[x=y]	show x=y, OR	send x=y, OR
source	[]	fallback	fallback
	[x]	show only x, OR	send only x, OR
event_types	[]	fallback	fallback
	[x]	N/A	send only x, OR

• open:True AND acked:False -> show open AND unacked incidents, send open OR unacked events
• maxlevel:3 -> show maxlevel <= 3, send maxlevel <= 3
• event_type:TICKET OR event_type:REOPEN -> show incidents that either has a ticket OR is reopened, send ticket events OR reopen events
• open:True AND event_type:[] -> show all open and ignore event type, send all open and ignore event type

The question arises: what about:

• acked:True AND event_type:[acked:False]
• open:True AND event_type:[open:False]

[hmpf]

Also, why shouldn't it be possible in the incident list to only list incidents that do not have a ticket, or have been reopened, or has been automatically closed by a source, or has been manually closed by a person.

[johannaengland]

How I have understood event_types is that it is not a dictionary like {Event.Type.INCIDENT_END:True, ...}, but actually just a list like [Event.Type.INCIDENT_END, Event.Type.CLOSE]. Events that are of one of the types in the list are the only events that will lead to notifications being sent. That list cannot be empty in a filter. But the key event_types can be missing for a filter, which would lead to event_type not influencing if a notification is sent or not.

[hmpf]

I'm thinking ahead, for full boolean support. We shouldn't build something for event types that makes booleans harder.

It might be that the real problem is that we check both incident and event in order to send an event. Ignoring stateless: An incident being open means both that its endtime is not set, and that there exists exactly one open event and zero or more reopen events.

It might be that we should have two types of filters and two places to make them after all, one for incidents and one for events. We'll need a live view of what happens when filtering on event types anyway.

[johannaengland]

Also, why shouldn't it be possible in the incident list to only list incidents that do not have a ticket, or have been reopened, or has been automatically closed by a source, or has been manually closed by a person.

The first thing is already possible in the backend: api/v1/incidents/?ticket=true.
The rest would also be possible to implement, but those would be other attributes then than event_types, since that is explicitly made for filtering events for notifications, not incidents.