Map EPICS alarms to JAWS alarms

[slominskir]

Currently we forward EPICS CA messages into the alarm system field-for-field, making priority options (severity) alarm type specific. It might be cleaner to define priority/severity in the registration topic instead. This would mean that EPICS alarms would need to be split per severity - likely doubling our 13,000+ EPICS alarms to 26,000+ since EPICS has a two severity system: MAJOR and MINOR. However, when non-EPICS alarms are defined a priority/severity field would be shared / exist. Our epics2kafka-alarms translation layer would need to be updated. Client GUIs/HMI would likely be cleaner if all alarms have same set of possible priorities regardless of source and the alarming schema would be simpler - especially if we drop the "STAT" field, as I don't think it is used.

[slominskir]

This may need to be divided into two separate issues/questions:

1. Does an EPICS PV represent one alarm or many when integrated into our alarm system?
2. How do we handle heterogeneous active alarming messages?

EPICS PV Integration
An alarm is often discussed as a well-defined entity each with a unique name mapping to a fixed priority and logic type. However, an EPICS PV in the strictest sense represents multiple alarms then as it can have dynamic alarm priorities (severities) and dynamic logic types (STAT field) associated with one PV name. Currently EPICS messages are passed field-for-field and record-for-record as active alarming messages, but we might actually want to translate them from one EPICS PV update to one of many possible alarms.

Heterogenous Alarm Sources
Note: The above question about EPICS integration impacts this question because if we decide that we'll map EPICS PVs to multiple alarms and alarming and acks become simple messages (without attributes) and all attributes are then consolidated in the registered-alarms topic then this question becomes unnecessary.

We'd like to originate alarms from any source (such as scripts outside of EPICS), but we haven't nailed down exactly what the expectations are for the attributes attached to alarms from various sources.

There are actually two sets of attributes at the moment due to EPICS's way of alarming with attributes attached. An alarm currently has:

• Registered Alarm Attributes
• Active Alarm Attributes

Attributes of a registered alarm will likely be a mix of shared items plus source specific (EPICS PV name must be included in registration of EPICS alarm, but not non-EPICS alarm). The tricky part is the active alarm attributes.

Specifically, can the active alarming attributes vary, or must they be translated to match a single definition of what attributes an alarm must have? For example, we could say that all alarms, regardless of source, must match the EPICS form - contain a STAT field and a SEVR field. Or we could say all alarms must match some other single JLAB form that we define (then translate EPICS alarms into it). Or, we can say that each alarm source can define it's own set of attributes (this is what the current prototype does).

We need to be careful not to duplicate attributes in the active alarm messages and registered alarm messages. For example, we would like to put a "priority" field in the registered alarms topic (as non-epics alarms would like to set priority there perhaps), but EPICS defines priority (as severity) with its active alarming messages.

Active Alarm Message Attributes	Description	Pros	Cons
Fixed	We must define the one fixed format and then define translation rules for each source	Single schema for clients	Translation rules are difficult to agree on, fields lost in translation
Variable	We simply provide the messaging engine and forward messages field-for-field.	Nothing is lost in translation.	Clients must handle variable types
None	An alarm has all attributes consolidated into the registered alarm topic	Simple, easy.	Must map an EPICS monitor update to one of many possible alarms for a single PV.

[slominskir]

For completeness we should mention we could force all alarms to just look like EPICS alarms (include STAT and SEVR) - this is a prominent case of the Fixed Active Alarm Message Attributes option above. This isn't great because:

1. The STAT field is EPICS IOC specific, and even in EPICS we don't really use the STAT field right now, but we'd be forcing every alarm source to use it going forward
2. Attaching attributes to the active alarming messages seems to be the wrong place - each unique alarm should be given a unique name. With EPICS a single PV name doesn't exactly tell you what the alarm is (I believe alarms are supposed to have a fixed, registered priority and type).

There likely isn't going to be much appetite for dividing each EPICS PV into something like (21 STAT fields) X (2 SEVR fields) = 42 different logical alarms (at least not manually), especially since it sounds like only a handful of STAT field value are even recognizable. Perhaps we find some middle ground, like divide each EPICS PV into two alarms: one MAJOR and one MINOR? Or we could come up with some wildcard syntax in the registration database that maps a single registration record to multiple logical alarms? This could be similar (but also very different than) the Alarm Grouping feature?

[slominskir]

Mapping an EPICS monitor update to one of two unique alarms (a MAJOR and a MINOR) is problematic in our current setup because we're using Kafka Single Message Transforms (SMT) - epics2kafka-alarms to serialize messages into the kafka-alarm-system format. A Single Message Transform can only transform one-to-one for a given message and also is not supposed to maintain any state. I believe we would need to transform a single EPICS monitor update message to two unique kafka-alarm-system messages and possibly maintain state in some scenarios. There are two scenarios specifically where this would be needed:

1. When a SEVR = NO_ALARM message is received the translation layer would need to write tombstone (no alarm) messages for both the MAJOR and MINOR alarms (OR otherwise would have to track state of which one was alarming).
2. A single PV cannot be both MAJOR and MINOR alarming simultaneously in EPICS, so the translation layer would need to tombstone the sister alarm when an alarming update is received (or clients would need to tolerate both MINOR and MAJOR at the same time, possibly masking MINOR with MAJOR)

The cost to redo this translation in a Kafka Streams app or to split it between the SMT and a new Kafka Streams app needs to be weighed with the benefit - a potentially cleaner schema.

[slominskir]

We're sticking with EPICs alarm mapping to JAWS alarm 1-to-1 for now. Can be re-opened later if we change our mind.