semantics of signing

[dhh1128]

A digital signature by A.id.privkey over (A.id, payload) creates a strong association between A.id and the payload. However, the nature of that association is context-dependent. What we know for certain is that the payload was seen by the controller of A.id when the signature was created -- but what A.id intended the act of signing to mean is not explicit in the data itself.

In the context of TSP, @SmithSamuelM has carefully defined this association to be one of provenance: the payload "comes from" A.id (see the explanatory quote from p 2-3 of the KERI whitepaper, which I'll append below for those who are interested). This is fine, and I have no problem with it. It is the most natural association to assume, and I am glad that we are making the nature of the association explicit.

But other associations besides provenance are conceivable:

• A signature could mean that we endorse content created by someone else. This is the meaning of signatures on a petition, for example. We are not claiming that the petition "comes from" us when we sign it.
• A signature could mean that we are creating evidence we have seen something, and nothing more. This is the meaning of signatures from notaries or witnesses on a will.
• A signature could mean that we are accepting contractual accountability.

The meaning of signatures is typically obvious from the content. Once we know it's a petition, we understand that a signature is an endorsement. Once we know it's a contract, and we read the sentence that says, "In witness thereof, I affix my signature on this ___th day of ___", we know the signature is an acceptance of legalities. But Sam has been very clear that the payload in his TSP proposal is purely an opaque bag of bits. Therefore, the signature we see on that bag of bits cannot have an interpretation that depends on the meaning of the bits. Hence his careful definition that it means "comes from" is crucial (and sufficient, and I support it).

Here's the rub -- and the reason for this discussion: I think people might mix these two cases up, and I think we need to be very clear in the TSP spec that this cannot happen.

• If the opaque payload of a TSP message happens to contain a petition, then the signature on the TSP message MUST NOT be construed as an endorsement of the petition, because it already has a defined meaning of "comes from". The petition comes from the sender, but the signature does not mean it is endorsed by or signed by the sender.
• If the opaque payload of a TSP message happens to contain something that is begging for notary-like witnessing, the signature on the TSP message MUST NOT be construed as that form of witnessing, because it already has a defined meaning of "comes from." The sender is not witnessing in a notary-sense.
• If the opaque payload of a TSP message happens to contain a contract, the signature on the TSP message MUST NOT be construed as a signature that accepts legal responsibility. The contract comes from the sender; it is not signed by the sender in a legal sense.

The way we construe this is important, because we don't want people refusing to sign a TSP message because they have an incorrect idea of what it means. In #22, I gave an example where a CFO has a concern about a financial report, so he sends a message to the CEO about it. The CFO MUST sign the message to the CEO, even though he maybe disagrees with some of the data in the financial report. The signature on the TSP message does not mean the CFO endorses the report; it means he is the proximate sender of it.

In order to get signing semantics that are not "comes from", I suggest that we say that a different signing envelope MUST be used. So to send a signed petition as a TSP message, the sender would first put the petition into an endorsement envelope and sign that. Then the endorsement envelope would become the opaque bits of the TSP message payload, which the sender would then sign again to apply the "comes from" meaning.

Do we all agree?

---

Sam's explanation of what a signature means in his TSP proposal:

Authenticity of verifiable packet payload is derived from mapping between an asymmetric key-pair and identifier. Attached digital signature exclusively (nonrepudiably) associates verifiable payload to the holder of the private key from the key-pair. The overlay’s
security is contingent on the mapping’s security. In general, with such a security overlay, the message payload inside an IP packet includes a unique identifier provided by the identity system that is exclusive to the sender of the packet. The payload is signed with a digital signature generated by the private key of a (public, private) key-pair. This signature is attached to the packet. The identity system provides a directory that links or maps the public key from the (public, private) key pair to the sender identifier. A valid
signature verifies that the packet came from the holder of the private key. The mapping between identifier and public key establishes that the message came from the holder. This approach assumes that only a valid sender of the packet with payload has access to the private key, i.e. the sender is the exclusive holder or controller of the private key. As a result the recipient of the packet may use the identity system to look-up the corresponding public key given the unique sender identifier in the packet payload and then with that public key, cryptographically verify the signature attached to the payload. Only the holder of the private key may generate such a verifiable payload. Without access to the private key, an imposter may not forge a verifiable payload. This security mechanism ostensibly establishes that the payload in the packet came from the correct source. Ostensibly, because the strength of this establishment depends on the strength of security of the mapping between identifier and key-pair. Indeed, the overlay’s security is contingent on the mapping’s security.

[SmithSamuelM]

@dhh1128 I agree that if there is no other context that we must be careful not to infer that a signature on an authenticatable packet means more than “comes from” however that is not the same as saying that a receiver can’t open the payload and make that inference.

It also ignores a important source of context that I left out of my simplified diagram in the proposal but described in #20. That additional context is protocol type, packet type namspacing. A more nuanced definition of “authenticatable message” would had a message metadata header as part of the signed portion of the authenticatable message. This header is a often practical necessity in order to support parsing but it conveys type information about the payload. So it is already a wrapper for the payload that we get for free. So to be clear an authenticatable message that include type metadata can say more than just where it comes from it can express intent with extreme economy of expression. The IP packet header for example includes a protocol field that defines the type of protocol for the payload. The protocol payload type is not inside the payload its in the IP wrapper. IANA defines 144 different IP protocol types. I was surprised that there were so many, Because that are only a handful that I have ever had to pay attention too. Most are routing protocols. A handful of those support multi-cast (which overload the meaning of the destination field). As far as I could tell the only non-routing protocols of any note are UDP and TCP.

There is a very important convention in protocol design that I assumed but did not state in my proposal and the are the encapsulation rules for nested headers and payloads for layered protocls. In terms of encapsulation rules we can say these define “what parts of a packet can a given layer see and/or change”. The convention for encapsulation rules for layered protocols are usually not the same as the rules for object orientated programming encapsulation nor functional programming nor semantic RDF linked data encapsulation nor any other type of encapsulation. IMHO This is primarily because stacked protocols value economy of expression much more than these other applications which in Is because in general bandwidth is the most expensive resource often by several orders of magnitude over compute or storage resources. If we attempt to apply the encapsulation rules of these other programming constructs we will likely end up with overly verbose protocols and net less expressive power has a result. If we carefully outline and follow a consistent set of protocol encapsulation rules we will be able to increase economy of expression without creating ambiguity thereby increasing expressive power.

So what parts of a packet can a layer see/change?

A stack has both a send and receive side. A packet encounters the layers in an inverse order on the send side than it does on the receive side. if the layers always appear in the same vertical order then the side of the stack gives direction to the packet. On the send side the packet moves down the stack. On the receive side the packet moves up the stack.
Remember the purpose of the enapsulation rules is to provide clean separation of concerns between the layers while also providing economy of expression.

Each layer has a layer specific header and then a payload. The payload could then consist of the header of the layer above plus the its payload. This structure is recursively nested. When building protocols using a low level language like C where the packets are expressed as simple C structs its hard not to follow this convention for packets at each layer. When building protocols that leverage more complex data structures like field maps or lists or semantic triples its easy to not follow this convention. In #20 I talk about an “effective header” to make it clear that we are still following the convention event though it may not appear to be so because the boundaries between parts of a packet are not so clearly delineated.

On the send side the usual encapsulation rules may be summarized as follows. A given send layer may change not only the data in its header but also the data in in the header of the layer below. Usually changing data in the header of the layer below is by passing a parameter to the layer below instructing it of a desired value.

For example, as mentioned above, the IP header has a protocol field. On the send side, the IP layer needs to know what type of protocol the layer above is tunneling through the IP layer in order to set the protocol header type value. So either,

1. The IP layer has to look inside its payload and figure that out. or
2. The layer above that knows what type it is needs to be able to set the protocol field in the IP header (in the layer below it) which means it can change the protocol type field in the header of the layer below.

The first option is bad idea because every tunneled protocol layer above a layer would either have to have a copy of the protocol type as a required field so the IP layer can find it inside its payload in order to set its header or the IP layer would have to be smart enough to be able to figure out the protocol type field from other information in its payload. The former reduces economy of expression. The later would make the IP layer thicker and make tunneling especially non-standard tunneling problematic in general.

In conclusion, we can enforce that the payload is opaque on the send side if we provide a mechanism for the layer above to instruct the layer below on how to set fields or data in its header that are specific to the layer above. Morever for economy of expression, no layer should keep a duplicate copy in its header of any identical data that appears in the header of the layer below. The function of the layer above can unambiguously set its semantics to avoid needing such a copy as long as it can instruct the layer below to set that value. It can assume its there.

On the receive side the usual encapsulation rules may be summarized as follows. A given receive layer may see not only the data in its header but also the data in in the header of all the layers below. A given receive layer may not see into its payload. The former is a natural result of the economy of expression rule from the send side that says no layer should have in its header a copy of any identical data from the header of the layer above. The latter ensures that we got the functionality encapsulated. If a layer needed to see into its payload in order to perform its function, then it would need to depend on the fact that every protocol that tunneled through that layer had that required field. This would make tunneling problematic as any changes to the functionality of a given layer would require changes to every tunneled protocol. This breaks encapsulation. Whereas a higher layer seeing into the headers of lower layers for what would otherwise be duplicated data given economy of expressing. And if the otherwise duplicated data much change then the layer above would have had to change anyway. So no explosion of work.

Returning to the IP header protocol field example. On the receive side, the IP layer needs to know what type of protocol the layer above is tunneling through the IP layer in order to know which protocol to use to parse or handle its payload. So either,

1. The IP layer has to look inside its payload and figure that out. or
2. The IP layer needs to have that value in its header as MetaData for the layer above.

Since the IP layer is already done processing its own layer by the time the layer above processes its layer, it can leave that information around or pass it along to the layer above. The intent of the sender as to what layer above protocol should handle the packet was conveyed by the sender in the IP header not the layer above’s header. If data is common to two layers, it means it may serve two roles.

1. as metadata
2. as layer data.

Economy of expression says no need to have redundant embedded copies. Just keep it around.

Another way of saying this is that a layer can not look up the stack but a layer can and may look down the stack.
There is one caveat. Looking down the stack usually stops at the spanning layer.

This looking down is limited to layers at or above the spanning layer. We want the spanning layer to be a veil that layers above cannot see through in general (although routing, logging and debugging applications may remove that veil), indeed routing applications are a necessary exception. Otherwise we will reduce adoptability and weakness of the layers above the spanning layer because they will be dependent on specific parts of the support and thereby reduce adoptability. As a litmus test if a layer above the spanning layer can’t function properly without seeing through the spanning layer’s viel then either the spanning layer is misdesigned or the application is.

So lets elaborate on these encapsulation rules with respect to @dhh1128 Daniels proposal.
Economy of expression is increased by explicitly allowing a given send side upper layer to instruct a lower layer how set data in that lower layer’s header so that it does not have to keep a redundant copy. Which on the receive side means that a given upper layer above must be able to see that data in the header of the lower layer. Using these conventions, the set of data from all the headers of any lower layers with respect to a given layer may be thought of as metadata or that given layer (upper). Metadata may be used to convey intent.

Think of it this way, in many of the use cases for adding an additional wrapper to place additional restrictions that involve metadata from the layer below, the answer is to make a copy of the meta-data and put it in the new encapsulated wrapper for the layer above. This increases verbosity without removing ambiguity.

This distinction of must not look up for isolation by may look down for economy of expression (i.e. metadata directionality) is an important one somewhat unique to layered protocol design. Although if you squint, a prototype programming languages (like JavaScript) appears to have an equivalent rule. One can look down (by ascending the prototype heirarchy) but one cannot look up (by descending the prototype hierarchy.

I will give an abstract example for protocol design.

Say I have an authenticatable message where the source AID is represented by Bob, The payload is “Loves Alice”. How do I infer intent?

Well the layer has metadata, that is the packet type. If the packer type is Sender statements. Then we can infer “Bob loves Alice” as the statement. If the packet type is sender statement meant for the object of the statement. Then we can infer Bob loves Alice meant to be sent to Alice.

We can be more general. Suppose we define a reference syntax. The word “I” in a payload means that the sender said it.

So now we don’t need a packet type of sender statement per se, becasue we can explicity refer to the sender by using a compact reference “I”

So if the payload is “I love Alice” we can then convert this to “Bob Loves Alice” when Bob is the sender and Doug loves Alice when Doug is the sender.

By allowing a given layer (the statement in this case) can see and therefore refer to its metadata we can have economy of expression and enable self referential payload where self-referential means refering to its own metadata.

In KERI a rotation packet has a type. The rules for processing rotation packets are different than for interaction packets. . The rotation packet as an explicit AID field that represents the controller . Is the controller AID in the header or in the payload? The seeing rules mean that when using a flexible data format like a field map, that the answer can be both. It is seeable by both the layer below if its in that layers header and by the layer above because anything in the layer below’s header is seeable as meta data. Thus we get economy of expression because it only has to appear once in the packet. Not twice. The signature on the packet is that of the controller. Which in its header is saying controller said x. But in the payload of the packer is eqivalent to I said X where I refers to the controller from the header.

In terms of intent casting, there are other AIDs in a rotation packet that are not the controller. These AIDS are only in the payload and therefore only in the layer above. The layer below is the spanning layer. The layer above is the rotation packet layer. There is no spanning layer packet its just an effective header that includes the controller AID. In the rotation packet layer, the rules for processing AIDs is different for receivers who are named in the router packet than receivers who are not named in the router packet because the rotation packet type indicates that they are. We don’t need another wrapper, the packet type is the effective wrapper.

If I didn’t care about economy of expression then I could model all the functionality of my layered protocol with semantic wrappers. Or I could use UML statecharts (which is essentially that), or IOFLO or some other protocol modeling language or RDF or Labeled Property graphs since graph language can be repurposed to model protocols. But I give up economy of expression to do it. My expressive power which is meaning per character is lower.

So its entirely problematic to make proposals for a concrete protocol using a different serialization language than the concrete implementation uses unless you intend to generate the concrete serialization from a transformer for that modeling language. UML state charts has such a transformer and the C code it produces is very very ugly. This includes the structs in the .h files which is its serialization format.

[dhh1128]

response truncated?

[SmithSamuelM]

So nothing I said here is meant to counter any the things that @dhh1128 said we want to be able to express. it is meant to point out that An authenticatable packet has a header with metadata (packet type) besides the sender ID so it can unambiguously say more than “comes from” without adding any wrappers.

[SmithSamuelM]

Try refreshing I had to submit more than once.

[dhh1128]

we must be careful not to infer that a signature on an authenticatable packet means more than “comes from” however that is not the same as saying that a receiver can’t open the payload and make that inference.

I think it has to amount to the same thing -- we must preclude the inference -- or we get into trouble.

I get that something at a high level can look down the stack and see stuff and reason about it. So a trust task can look at something that's 4 layers above TSP and reason about TSP metadata. A good example of this is that the payment trust task could look down the stack and confirm that it is occurring in a context where confidentiality has been guaranteed.

However, TSP metadata cannot mean 2 things. If it's going to mean "comes from" it cannot also mean "endorsed by"; otherwise I have no way to mean one without the other. The act of endorsing and the act of emitting are two different things. Perhaps there is an optimization that allows us to officially declare that both acts occurred together, with a single signature -- but then we are creating a layer violation by saying that "endorsed by" semantics are embodied by something at a layer that doesn't know what "endorsed by" means.

[dhh1128]

We could have a petition that says, "Transmitting this petition to another person over TSP with a signature constitutes an endorsement." If such language is present, then a layer above TSP is justified in saying that the signature on the TSP packet constitutes endorsement. But this is no longer inference; it is explicit. And it's not a very good idea, anyway, because it couples the act of signing and the act of sending, both temporally and logically, such that it now becomes impossible to send the petition to someone unless you endorse it yourself.

[SmithSamuelM]

I agree it can’t mean two contradictory things but it can mean two compatible things. If “comes from” is modified in the data then a received can infer unabigously that the sender is a controller and therefore the data in the payload is safely inferred to be comes from a controller not just some AID and the type can change the rules for how a reciver interprets comes from MY controller when the reciver is a witness for that controll to be diffeent from comes from A controller when the reciever is not a witness. Context modifies intent. And type is a way to convey very specific very rich context and therefore modified intent.

[SmithSamuelM]

I feel like you are trying to model this without any types. This is a very human natural language way to do it. Which sounds nice but is largely impractical. We use types largely because natural language processing is hard. Its easy to write becasue we are humans and we can do natural language processing without thinking about it much. But we want this to be a performant protocol stack so we should be leveraging type information and not inferring it from natural language expressions. Wrapping is another way of expressing type because each wrapper I assume has a type but you are essentially not allowing the set of wrappers to themselves convey intent. Each wrapper much only convey an atomic intent not the combination thereof. This is not economical. Its feels easier but is much more verbose.

[dhh1128]

That additional context is protocol type, packet type namespacing.

This is what I was proposing with my "schema" field (last row in table here). Technically if we want sufficient context on this topic, we need two pieces of metadata -- the schema of an individual message within a protocol, and a reference to the protocol to which that message type belongs. But if we say that a given message schema always belongs to a particular protocol, then just referencing the schema of the message type is sufficient. On the other hand, referencing the protocol alone is NOT sufficient. Knowing that a message belongs to an online auction protocol is not enough; knowing that it is a "bid" message type in an online auction as opposed to a "announce-results" message in an online auction IS sufficient.

[SmithSamuelM]

So I agree violently with what you say above. But it seems that although you say that, its not reflected IMHO in your argument that the. meaning of an outer wrapper can’t be implicitly modified by the contents of any inner wrapper. As long as the rules from implied modification are consistent and unambiguous it could be used. You can argue you don’t want to use it because your are worried you wont be able to define consistent implications. But economy of expression is a good reason to try.

[dhh1128]

I suggest that the way forward on this topic is to look at concrete examples, because we are both arguing theory and not understanding one another very well.

[SmithSamuelM]

So we have two examples. KERI which only uses protocol type and message type. And ACDC which uses protocol type, one implied message type and schema.

Keri is more efficient by a lot than ACDC. Both sit on top of the same spanning layer. KERI does lots of implied contextual processing because it want to be extremely terse. ACDC not so much and ACDC uses wrappers more. But ACDC is not an idictment of KERI nor is KERI an indictment of ACDC. Both work just fine. If we only allow one then we go down rabbit hole. I just spend a week arguing with a group of architects that are trapped in the semantic triples is the only way to express a protocol rabbit hole. Lets be more open and more general by not assuming only nested wrappers but allow for implied wrappers and well known techniques for economy of expression in the same protocol stack.

[SmithSamuelM]

CESR, CBOR, JSON, MGPK can all express the same semantics using completely different syntax. There is some ugliness to do that but not a lot. Likewise we can convey compatible inferred meaning using different syntax. And trying to reason that a given statement can’t be modified by nested content that can make reference to the outer content but must have a different vocabulary and different variables is very inefficient. Its not what I am saying either. I did not define authenticable packet to only. mean “comes from sender” and then not be to say “comes from sender who is a controller” that is implied from how it is used and “comes freom sender who is a witness” that comes from a difference use. Whereas as I understand you proposal you are saying that I MUST mean that I can only say “comes from sender” and its wrong to infer (who is a controller). That therefore we MUSt have another statement (comes from sender (AID)). and (AID is a controller) where the latter can imply the former Thats not economical layering

[SmithSamuelM]

As you now in ACDC schema is type. So the type of a packet is defined by its schema. So the signer could be defined by the schema and the structure or type of intent of the signer could be defined by the schema with the actual intent values provided by the ACDC field values.

KERI messages have no schema. The schema is implied by the type. But the same reasoning applies. The signer is defined by the implied schema and the structure or type of intent of the signer could be defined by the implied schema.

With CESR which can either be attached or embedded in either KERI or ACDC. The schema is implied by the CESR derivation code in combination with any wrapping count codes. The count codes are nestable so the meaning of any inner nesting is inferred from the actual set of nestings in combination not separately. An indexed signature nested in a witness signature count code means the indices reference the witness list. An indexted siganture nested in a controller signature count code mean the indices reference the key list of the controller keys in its latest establishment event. Since schema in ACDC only applies to the ACDC itself not its attachments, the schema of a CESR attachment is inferred from its code. Likewise for KERI with attachments.

Composition is additive in meaning and the outer wrappers provide context for the inner wrappers. And the inner wrappers extend or constrain the meaning of the inner wrappers. Which is contradictory to your statement that “comes from” cannot be modified by the contents of the payload. The payload is not opaque to the layer in which it is interpreted. There is no actual spanning layer in that sense its just that any packet that has an AID that the schema (implied or actual) designates the source of the packet can be the AID that the packet comes from but that same AID without being repeated can be extended or constrained or modified by the contents of any layer above (i.e. inside).

When CESR is embedded in ACDC then the implied schema of the CESR is in composition with the explicity schema of the ACDC. The inside schema is extending, modifying constraining the meaning of any fields in the outside layers to which it refers.

When CESR is embedded in KERI then the implied schema of the CESR is in composition with the implicit schema of KERI but it can function just the same as above.

Layering is additive so it is compositive but your example does not use additive layering it uses independent, isolated layering.

RDF triples expressed in the style of JSON-LD force a certain structure. This has biased how DID Docs and VCs are written. Because triples have this very weak expressive structure, composition expression are naturally verbose and uneconomical. RDF graphs typically have a significant percentage of their nodes that are blank nodes often well over 50%. This is because to compose one must reify the composition as another triple. We should not repeat that mistake in the 80’s we called knowledge graphs made from triples (is-a has-a ) graphs because they were a narrow simplification of a semantic or knowledge graph and did not find them very useful compared to unsimplified semantic graphs (which are now called labeled property graphs).

[dhh1128]

This might be a case where agree but are talking past each other, but I can't tell yet. What do you imagine is different when sending a signed message having as the payload a signed (endorsed) petition versus a signed message having as the payload an unsigned (unendorsed) one? I see three possible answers. The first answer is that there is no difference at all; the messages are bit-for-bit identical. This is intolerable to me because it is impossible to tell my two cases apart; the high-level petition logic has hijacked the semantics of the "comes from" construct. The second answer is that the two messages differ only in the bits of a schema and/or protocol field; the bits of the payload are identical. In the endorsement case the message is part of a "here's my endorsement of the petition" protocol, and in the other it is part of a "here's a document I'm sharing" protocol. This solves the ambiguity problem, and i could live with it, but it creates a new problem because now a higher layer has to keep the lower layer's data together with its higher-level construct to prove the intent of the sender. This is the equivalent of requiring an app to not just retain the JSON body of a POST, but the IP packets that carried it. It seems to me that this is a layer violation. I will say more in a follow-on post

…

On Sat, Feb 18, 2023, 2:34 AM Samuel Smith ***@***.***> wrote: As you now in ACDC schema is type. So the type of a packet is defined by its schema. So the signer could be defined by the schema and the structure or type of intent of the signer could be defined by the schema with the actual intent values provided by the ACDC field values. KERI messages have no schema. The schema is implied by the type. But the same reasoning applies. The signer is defined by the implied schema and the structure or type of intent of the signer could be defined by the implied schema. With CESR which can either be attached or embedded in either KERI or ACDC. The schema is implied by the CESR derivation code in combination with any wrapping count codes. The count codes are nestable so the meaning of any inner nesting is inferred from the actual set of nestings in combination not separately. An indexed signature nested in a witness signature count code means the indices reference the witness list. An indexted siganture nested in a controller signature count code mean the indices reference the key list of the controller keys in its latest establishment event. Since schema in ACDC only applies to the ACDC itself not its attachments, the schema of a CESR attachment is inferred from its code. Likewise for KERI with attachments. Composition is additive in meaning and the outer wrappers provide context for the inner wrappers. And the inner wrappers extend or constrain the meaning of the inner wrappers. Which is contradictory to your statement that “comes from” cannot be modified by the contents of the payload. The payload is not opaque to the layer in which it is interpreted. There is no actual spanning layer in that sense its just that any packet that has an AID that the schema (implied or actual) designates the source of the packet can be the AID that the packet comes from but that same AID without being repeated can be extended or constrained or modified by the contents of any layer above (i.e. inside). — Reply to this email directly, view it on GitHub <#24 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAQ3JCDVAPT6RZGR7NUHU23WYARLZANCNFSM6AAAAAAU72KSPM> . You are receiving this because you were mentioned.Message ID: <trustoverip/trust-spanning-protocol/repo-discussions/24/comments/5016026@ github.com>

[SmithSamuelM]

I realized one potential source of confusion. I proposed that a trust spanning layer is in the form of a identifier system security overlay. I did not define what I meant by overlay. The term overlay is essential to understand how to actually implement the functionality of what I proposed. The overlay is essential. Overlays are a type of virtualization. I can take some existing thing and then overlay it and the the result is a new thing with some new property or functionality.

A security overlay as I use the term, imbues a given set of data or information with a security property. In this case the property is secure attribution to a cryptonym. If I overlay data with the security overlay then the data becomes authenticatable in the narrow sense of securely (cryptographically verifiably) attributable to a cryptonym. If the data is not just meaningless data but is a statement that conveys intent then the overlay imbues the intent conveyed by statement with the property of authenticatability. The overlay is additive not subtractive.

An overlay can be stateless or stateful. For peristent AIDs the overlay is stateful.
Overlays can be nested (stacked). A stateful overlay can be recursivley stacked on top of itself. Each application of the nested stateful overlay changes the state that is passed to next application of the overlay.

So the state that KERI identifier security overlay maintains, that is, a KEL (key event log) its itself produced by a recursive application of the overlay function.

The beauty of using an security overlay as the trust spanning layer is that it can be stacked on top of other overlays that are not security overlays thereby imbuing the overlaid overlay with its security properties.

Overlays are also described as wrappers. In programming languages like Python a type of overlay is a decorator.

The information flow with respect to an overlay can be two way or one way. A stateful overlay can use two way information flow. But it may not be a good idea in general.

To simplify tunneling, we generally want information or properties to flow from the overlay into the data or statement or protocol being overlaid and not the other way. We don’t want the invocation of the property imbued by the overlay to be dependent on the tunneled information with one exception, for economy of expression any information that is redundant to both the overlay function and the data being overlaid. So the type of the statement being overlaid can provide the context that declares which elements are AIDs and the intent of a signature by each AID. We could define an overlay. mechanism that inverts this. We can have statements that make compact or implied references to AIDs that are not explicitly in the statement but are attached or injected by the overlay. Both are economical. And a single overlaid statement could employ both types of economy of expression.

Nested overlays of different types gives deeper meaning to the PAC layering prioritization. Where the outermost overlay is authenticity which wraps confidentiality, which wraps privacy.

To elaborate, any AID in any data or statement that is overlaid with a security overlay that provides authenticatability now becomes authenticatable. This does not limit the intent casting that the statement already had. On the send side a statement may be composed that conveys its intent with respect to its AIDs and then it may be overlaid with the spanning layer overlay it so that any AID inside that statement could be the source for which an attached signature can be authenticated. Each attached signature can make authenticatable a different intent. The overlay doesn’t care it just works. The statement cares and any recipient of the statement can rest assured that all the different intents were conveyed in an authenticatable manner. Each intent does not need a different overlay because what role a particular AID provides is determined by its role in the overlaid statement. To reiterate, the authentication overlay itself is independent. (does not care about the role of its AID) Its purely additive. The statement itself defines the role and the intent of any attached signatures and all the overlay does is imbue those separable intents with the property of authenticatability.

fFor example, in a KERI rotation event packet there are multiple AIDs. The controller has an AID. The rotation event packet implicitly defines a schema that defines the role or function of each of those AIDs. This then informs what an attached signature from any of those AIDS means. When the controller attaches its signature(s) then it make the controllers intent to change its key state via that event authenticatable. When the witnesses designated in that event attach their signatures it makes their intent to witness (verb) (a type of endorsement) authenticatable. This provides economy of expression.

In my next presentation (in two weeks) I will present a formalism for a confidential message also as an overlay. Stacked with the authentication identifer security overlay then makes a confidential message also authenticatable, i.e. an Authenticatable confidential message.

The intest casting is provided by the data portion of the message and the type and complexity of the intent casting is a function of the type of message.

The overlay is a virtual layer because the AIDs that the overlay modifies may be already embedded in the data. My diagram of an authenticatable message that showed the identifier outside the data was meant to convey the elements and show what was wrapped and to show the information the overlay needs to know. It is easy to misinterpret that meaning without explaining that its application is as an overlay. I now realize that the discussion of a spanning layer using IP as an analogy got a little off track because I didn’t qualify that when applied, the authentication layer is an overlay. I.e. a virtual layer.

[dhh1128]

This helps me a lot. Thank you for that clarification. I continue to disbelieve the assertion that an authenticity overlay is minimally sufficient, but I now see how my contention about the need for various pieces of metadata to be authenticatable fits nicely into what you had imagined. Further, I see how it gets much closer to being minimally sufficient since it can be applied all over the place.

I find myself a bit puzzled by your example about an authenticatable confidential message. It seems to me like the order you are imagining is the opposite of what it should be. It sounds like you are advocating alternative 1 in this drawing (where authenticity of a confidential message is a thing), and my expectation was that we wanted alternative 2 (where confidentiality of an authentic message is a thing):

Perhaps I am misunderstanding what you mean by an "authenticatable confidential message"?

Here is my reasoning about wanting alternative 2: The sender of a message is sensitive metadata that needs confidentiality guarantees. (The one thing we absolutely cannot omit is the destination address, else we cannot route it. By making the dest address PLUS the sender address provable to arbitrary viewers, we allow surveillance, don't we?) Authenticatable and confidential are kind of opposites, in a way; they make sense together because we have different audiences. The audience that wants authenticatable is the recipient, not intermediaries; and we want confidentiality from eavesdropping intermediaries. Right?

We are also committing the sender to the encrypted form of the data rather than the plaintext form of the data. I know that the encrypted derives the plaintext, but this means that if I encrypt 3 times for 3 different audience members, I have to sign 3 times. I want to sign once -- the plaintext -- and then encrypt 3 times (or better yet, asymmetrically encrypt a newly generated symmetric key 3 times and append that suffix to a big symmetrically encrypted payload like a genome so I am not re-encrypting large amounts of data uselessly). Also, an auditor can no longer establish the authenticity of plaintext; she or he now depends on the cooperation of the recipient before the meaning of what the sender signed is knowable.

Did I misunderstand your vision?

[SmithSamuelM]

I think we are converging. I am glad explaining the overlay concept assumption helped.

There are two issues at play with Signing and Encryption and they have alot with the interplay. the security and the supporting crypto as they do with the abstractions you would like for confidentiality and authenticity. But there is also an important definitional distinction. Apparently we are using two different definitions of confidentiality.

When you say that “the sender of the message is sensitive metadata that needs confidentiality.”. It appears to be using a different definition.

As I have presented elsewhere on the PAC theorem. I use the surveillance legal definitions of confidentiality and privacy. Recently, privacy has come to mean something else besides the surveillance definition.

In surveillance law privacy refers to the parties to a conversation or an exchange of information. Privacy is knowledge about the parties involved in the conversation”. The legal term is “non-content” metadata. Basically “Who Said It”

In surveillance law confidentiality refers to the contents of a conversation or an exchange of information. The legal term is content data. Basically “What was said”

So there are strong legal protections for confidentiality around “what was said”, i.e. content data. The search and seizure laws requires typically a search warrant in order to view content. The legal protections privacy around “who said it”, i.e. non-content meta-data are comparatively weak. The search and seizure laws only require a subpoena from an officer (not a judge) and the who of any conversation in a public place is not protected in general. This distinction does not mean we may want to do better. But one of the reasons for the distinction in law is practical. Protecting the confidentiality of content can be done in many practical ways. Protecting the privacy of non-content meta-data is entirely problematic from a practical point of view.

So we you say you want to protect the confidentiality of sender as sensitive metadata you are combining two definitions. Using the legal definitions, the statement is you want to protect the privacy of the sender metadata.

So we can do that with confidentiality by layering the message inside another message.
The question is what is the layering of message. 1 or 2 ? I can always layer 1 inside of another 1 to get confidential (privacy of sender metadata) When I try to layer 2 I have the chicken and egg problem of how do I know I want to forward, or receive 2. I can’t unless I first decrypt 2. This exposes me to DDOS or probing attacks. An encrypted channel must still authenticate the ends, Its just not doing it on a datagram by datagram basis. So event an encrypted channel is more 1 than 2.

In a different conversation where I listed three ways to encrypt 1,2,3 You said you didn’t like 3 which was to have encryption on the outside. So maybe I mistunderstood.

Anyway. the last overlay applied is the outside. So If I want confidential content information I first apply the confidentiality overlay to that content and then if I want that content to be authenticatable then I apply the authenticity overlay to that. This gives me 1. Now how do I protect privacy in this construction.

1. The AID are cryptonymous. So they are private by default unless I link them. I can correlate multiple conversations to the same cryptonym but the privacy vis a vis a legal entity or natural person is still protected.
2. I can layer inside 1 another 1 of authenticable data wrapper where the cryptonym of the inner layer is not exposed to the outer layer making the privacy confidential.

[SmithSamuelM]

I actually think we are talking past each other. And the nub is what comes first or what comes second.

I don't know how important that really is.

So I 100% agree with the example above as a mechanism to bootstrap.

Except that you left out one important step. How does Alice know Bob's encryption key so that she can encrypt only to Bob?
I assume there is an out of band exchange where Bob reveals to Alice that there is some AID that Bob controls to which Alice can send encrypted data and be assured that only Bob can decrypt it. (to the extent that Alice trusts Bob).

Otherwise how does Bob know amongst all the real AIDs he is using which one to apply to decrypt Alices's message. I suppose you implied that the throwaway AID of Alice is also sent in the clear with the packet so that Bob knows which decryption key to use. So I have a packet from Alice that is of the following form.

[routable wrapper, ip header][ authentication wrapper Alices throwaway AID as source AID][Payload Encrypted content to Bobs Real AID][Attached signatures]

So from my perspective this looks like a wrapped packet where the outer wrapper is an authentication overlay that overlays an encrypted body with attached signatures where at least one of the signatures is from the outer authentication overlay. The encrypted content is yet another layer in the overlay. Bob's Dest AID can be implied by the encryption because Alice only uses the ephemeral AID as to send from so Bob can thereby look up the key state for the corresponding Dest AID in order that Bob knows which of his keys to use to decrypt and by successfully decrypting verify the mapping was valid.

Suppose that in that OOB exchange where Bob gives Alice his real AID, that Alice gives Bob her real AID. Now both can use the throwaway setup to send a wrapped packet. But they could do something else.
Both Bob and Alice could instead send encrypted packets directly to each others real AIDs without the throwaway setup. In fact because the real AIDS were exchanged out of band. Bob and Alice can sign their packets as well with the real AID. The packet itself only has IP addresses in its header (protected or not) which you don't count as in scope. So now Bob and Alice decrypt the body portion of the packet itself and get the real AID of the sender in order to verify the signature. In this case, the outer wrapper is encryption and the inner wrapper is authentication. However, both Bob and Alice have the problem of knowing which of their real AIDs to use to decrypt. They could use a unique real AID per source. Bob gives to Alice the real AID that Alice must encrypt to to send to Bob and Bob has to use some other information to guess that it came from Alice like the source IP. IF its spoofed the worst that happens is it DDoses Bob. But this would just be kicking the can down the road.

So I conclude that the authenticatible setup with an outer wrapper is the most pragmatic approach even if its not the only approach theoretically. Many group encryption schemes relay on an OOB setup to define the "channel" by which the group communicates. Given the assumption implied by the setup, all packets can be fully encrypted with no unencrypted source or destination IDs (other than IP) This is an exception that proves the rule IMHO. The apprach is impractical without some authentication mechanism for setup and at least an implicit way to determine what channel or group is engaged so that one can lookup the keystate for the decryption. The weakness of this approach is for the setup because it is not crypto based so can be attacked.

If IP addresses out of scope then there is no privacy loss because no AID is sent in the clear by either party. They just have to decrypt every packet to know who it came from because the signatures are embedded in the packet.

If they don't want to decrypt every packet then the throwaway setup would allow the encrypted packet to be signed by the ephemeral key that they must also exchange out of band in addition to the real key. Then every packet is signed by the ephemeral key on the outside and then encrypted to the real key and in the encryption is a signature on the plain text using the real key which was provided out of band.

By multiplying OOB mechanisms we lower the automatability of the system and increase transaction cost and friction. so the ephemeral setup using an ephemeral (throwaway) AID is applying the authenticatablilty overlay in order to enable or bootstrap lookup of keystate for decryption of encapsulated encrypted content with less OOB.

I don’t see that there is anything to disagree about except that one can arbitrarily define OOB mechanisms as out of scope to hide that authentication is necessary in order to securely lookup or map decryption key state at the destination. The overlay(s) (authentication and confidentiality) map identifiers to key state. For some identifiers the key state is not dynamic its fixed and so the identifer used must be ephemeral (non-persistent). They are still used to lookup signing key station in order to authenticate a packet to a source identifier in order to securely lookup decryption key state for the destination identifier. But whenever one uses ephemeral identifiers for setup which identifiers must be abandoned eventually because its keys can’t be rotated then the OOB exchange of those identifiers must be repeated and over time this builds up a lot of OOB exchanges needed for ongoing key maintenance. This I call the PGP problem. I used PGP for several years and then gave it up the first time I had to rotate my key(s) because I had to repeat all the accumulated OOB exchanges. Throwaway setups are not without cost. Better would be a setup that is privacy preserving of the real AID but is not throwaway and only has a one-time OOB setup. There are several options for doing this. I will diagram one for my next presentation.

[SmithSamuelM]

@dhh1128

TSP is trying to do too much if it wants to solve both IP packet-level abuse (a problem at the waist) and abusive handling of application-level messages (a problem at the neck). TSP is about the neck, not the waist.

In my mind one of the main motivations for a narrow neck is to fix the unfixable problems at the waist. No amount of work on the waist will fix it. Its unfixable. But if we have a true trust spanning layer neck, then we can ignore the problems at the waist. They can’t hurt us. This is what the property of “end-verifiability” does it fixes the problems of the waist without having to change the waist. I think that these are in scope because if the TSP does not fix the problems of the waist enough (not completely but enough) then we just have another band-aid.

[SmithSamuelM]

@dhh1128

A legal perspective may help with privacy (e.g., your graduated disclosure idea), but I don't think it's super insightful WRT our confidentiality goals.

I am not using the legal definition here because it provides any security beyond the crypto. I am using it because it is a useful definition to clarify how we talk about identifiers and layering. I also believe that more than a little convenient if the definitions align for other reasons.

Identifiers are used to look up key state. That is the important essense of the security overlay. I need a secure mechanism to lookup key state so that I can do other things securely like encrypt and decrypt content because I can’t encrypt properly unless I know the correct destination IDs key state for such encryption. And the destination doesn’t know which of its description keys to use unless it has an identifier to look up key in order to decrypt the packet. And the destination doesn’t know if it cares about a packet that it knows how to decrypt unless it can securely attribute the packet to a source. So Identifiers as metadata are essential to key state lookup, even for ephemeral identifiers. So practically speaking at any layer there are at least one and maybe more identifiers that are unencrypted for the lookup of that layers keys. The outermost layer has one or more unencrypted identifiers to bootstrap itself which enables the nested lookup down all the layers. Of course the outermost layers unencrypted identifiers could be implicit because they are provided out of band. but that as I said above is the exception the proves the rule.

[dhh1128]

I actually think we are talking past each other. And the nub is what comes first or what comes second... Except that you left out one important step. How does Alice know Bob's encryption key so that she can encrypt only to Bob? ... So I have a packet from Alice that is of the following form.

[routable wrapper, ip header][ authentication wrapper Alices throwaway AID as source AID][Payload Encrypted content to Bobs Real AID][Attached signatures]

So from my perspective this looks like a wrapped packet... Suppose that in that OOB exchange where Bob gives Alice his real AID, that Alice gives Bob her real AID. Now both can use the throwaway setup to send a wrapped packet. But they could do something else... So I conclude that the authenticatible setup with an outer wrapper is the most pragmatic approach

Thank you for taking the time to write this. I felt really happy about a sense of convergence as I read it. I don't think our alignment is perfect, but I nodded my head about a lot of it. I did not fully understand your textual depiction of the packet structure because I can't tell from the brackets how the layer containment relationships are imagined to work. But I assume we can sort out those details.

I specifically DO agree that I left out the part about how Alice discovers Bob's identifier and key state. This was not because I was unaware of that dependency, but because there are two important and quite different cases, and I didn't want to take the time to explore them. But I'll say a little about them now.

In one case, Alice is private but Bob is a public entity. Bob doesn't know Alice, but Alice knows (of) Bob; she is attempting to contact him via his public identifier and in association with his public reputation. In such a case, I assume his identifier is discoverable without Bob taking any new action to help Alice. Probably that means he previously published an OOBI somewhere, but it is uninteresting exactly where or how. What's noteworthy about this case is that most organizations and public figures will form vast numbers of connections having the same posture as this public Bob I am postulating. Because of this, being able to connect where public Bob is passive and unaware of the connection attempt in advance is a highly important optimization. A protocol that requires Taylor Swift to proactively engage in setup for a connection to each of her fans will never be accepted. A protocol that requires her to react to proactive behavior by the Alices of the world (e.g., to prove that she controls the AID in question, and then, if needed, after an encrypted channel exists, to prove that the AID on Taylor's end is associated with Taylor's public human identity) is fine.

In the other case, both entities lack useful public reputation, so they must each be proactive about sharing an OOBI. Maybe Alice and Bob meet at a trade show, and Bob scan's Alice's badge. This gives him her OOBI. He then contacts her back, and presents his own OOBI. This establishes a cryptographically secure channel bound to AIDs, but the face-to-face interaction allows them to go beyond that and make the probably-safe assumption that the human identity they intend are bound to those AIDs. If they're not confident enough, they can always use credentials to do that level 2 authentication instead or in addition.

(The two cases I gave are not the full set of possibilities. Alice could be public, too. However, I discount these because I don't think a public entity should ever reach out directly to form a pairwise channel where their side of the channel is bound to a public identifier. A public identifier is anywise, meaning it is intended for use in arbitrary contexts; this is antithetical to setting up an encrypted pairwise channel. If Nike wants to connect with supplier X, it should create an AID for the Nike:X relationship, so it can keep that separate from the Nike:Y relationship. It can leverage the Nike:Anybdy AID for reputation as the connection gets set up, but it doesn't do this by emitting a message that is publicly observable as coming from Nike:Anybody and going to X; it instead emits a message that is publicly observable as coming from ThrowawayAID:X, having a confidential payload that X can tell comes from Nike:Anybody but introduces the Nike:X AID for the purposes of managing that specific relationship.)

The larger point I'm trying to make here is that OOBI exchange as you've conceived of it in the KERI ecosystem feels like an important ingredient -- but not the whole story on how connections have to be built. I'd like to explore that further in TSPTF meetings.

[SmithSamuelM]

@dhh1128 direct mode in KERI exists to enable private-to-private. In private-to-public, an open decryption service endpoint may be subject to interactive chosen crypt text or chosen plain text attacks. So wrapping the incoming with a source authentication makes it easy to defeat such attacks.

[SmithSamuelM]

@dhh1128

intermediaries

I see too classes of intermediaries, those that are authenticatable and those that are not. Clearly, the endpoints of a conversation to be protected from exploitation by the unauthenticatable. But exploitable is not necessariiy the same as confidential. There are also two types of destinations, authenticatable and non-authenticatable. We can route over infrastructure that has a destination address that is its self is not authenticatable, but the routing was set up out of band to the conversation.

There are two main classes of routing algorithms, table based and vector based. Many robust routing algorithms employ both in stages or phases. A tables based is out-of-band to the conversation. A vector based is in-band to the conversation. We can have secure authenticatable confidential conversations with both. So assuming too much about how a destination may be expressed can lead us down the rabbit hole of only supporting one or the other type of routing.

[SmithSamuelM]

Lets say I want to set up an encrypted channel. Why would I do that? Why not just use the ubiquitous HTTPS/TLS/DNS/CA encrypted channel infrastructure that I get for free?

There are many use cases where HTTPs/DNS/CA is problematic. But in the context of the TSP, the reason I would not use HTTPS is because its got security problems that stem from its key rotation mechanism. Or to summarize, the authentication mechanism is insecure. How does that exhibit? It primarily exhibits not in a third party being able to surveil the content of a channel. It exhibits because the destination endpoint to whom I am sending confidential data is not the intended recipient of that data. So I have neither a private nor confidential channel (with respect to the endpoint). If the endpoint can't be trusted to respect the confidentiality of data I sent over the channel, then my data is not confidential. If the endpoint can't be trusted to respect the privacy of the any AIDs or references I make to AIDs that belong to me then my privacy is not protected. But if all I care about is some random foreign agent snooping on my packets then I am protected, unless of course that random foreign agent happens to hijack my connection and is the other endpoint. So if I don't authenticate the other endpoint before I send any data the channel does not protect either may confidentiality or my privacy. Lets say that I do authenticate the other side and it fails. Then I drop the channel. The setup of the channel has now been changed. I have added another authentication step to enable at least my confidentiality to be protected.

So if I want to have a confidential channel, I have to first authenticate the other endpoint. So I know that to whom I am sending my confidential data is the correct party. That means that the other party will have to disclose to me something that allows me to know they are the correct party. Now if I have to do this authentication for every single packet it would be impractical in general. (there are use cases where its justified). So I want to do it rarely and hopefully just once unless some evidence tells me I need to repeat it.

So a confidential channel that had a really really secure setup could be very useful.

But what if only some of my data is confidential wouldn't it be nice if I could select which parts of data were confidential and any recipient would know that it was from me without my reauthenticating every single time.

So I might want to have a channel that is fully encrypted with secure setup. I might want to have a datagram not a channel that allows me to distribute confidential data to the intended recipients. I might want to have a private channel that allows me to distribute data to the intended recipients but that third parties can't tell its me sending the data. So do I do that in one layer, two layers, three layers?

Its application dependent. There is no one size fits all. But the setup of every layer requires authentication of some kind or the parties involved can't make securely attributable statements. A confidential statement disclosed to an unintended party is not confidential. So I need to tightly bind confidential disclosure intention with authenticated parties. At least authenticated to some cryptographic construct that is sufficiently secure relative to the specificity of the confidential intent.

[Oskar-van-Deventer]

@dhh1128 I agree, let us specify the semantics of signing at ToIP Layer 2 as "comes from", and explicitly exclude any other semantics.

Consequently, we should encourage not reusing public keys across ToIP layers. Instead, one could sign a layer-3 key with their layer-2 key to signal whom that layer-3 key "comes from".

--- aside ---
@dhh1128 , @SmithSamuelM may I encourage you to be a bit more compact in your discussion contributions, please? This Daniel-Sam discussion is already 10 000 words. This makes it rather inaccessible to other TSP participants. If a verbose comment is needed, then could you precede it with a TLDR, please?
--- aside ---

Oskar

[talltree]

Having just taken 5 minutes to only SCAN this discussion (which was entirely new since I last was able to catch up with some discussions last Friday), my first reaction was to agree with @Oskar-van-Deventer that a TL:DR summary would be helpful.

At the same time, I also realize that, with this level of protocol design, topics are will sometimes get very deep. And in those cases, there will be threads like this that indeed need to get very long and detailed. So I don't want to discourage that.

I'm hoping we can find a way to then post TL:DR summaries of such topics so not everyone on the TSPTF has to read everything to stay relatively current.