Chat Message Container

[Yannick-Malins]

Enhancement Request

Use Case:

This proposal describes an initial structure to contain the "message" part of our new messaging-related intents

The objective is to be as generic as possible, building something that not only makes sense for IM chat applications, but also for asynchronous channels such as email

We also want to be flexible: for the textual part of the message we use a record to map mime type to content, allowing us to start with the simplest possible type (plain text) whilst we discuss further enhancements (e.g. markdown). This also allows applications to send multiple types (e.g. always sending plain-text as a "fallback")

The second record maps "entities" to unique IDs. An entity could be something as simple as an image attachment (see first example below), or something more advanced like an fdc3 intent that the user wants to embed into the outgoing message (see second example). The ID will allow us to reference specific entities from specific locations within the text when defining more advanced text formats.

Contexts

Message

Message to be sent in the context of StartChat or SendChatMessage

Details

export const enum TextMimeTypeEnum {
  'text/plain',
  //later we could extend support like 'text/markdown', 'finos/arbitrarycustomformat' etc
}

export const enum EntityTypeEnum {
  'fileAttachment',
  'fdc3Intent'
  // Later we could extend support to other types such as 'inline-image', 'news'
}

export interface FileAttachmentEntity {
  type: EntityTypeEnum.fileAttachment;
  data: {
    name: string;
    dataUri: string; // Base64 encoded file - 'data:image/png;base64,{BASE64_DATA}';
  }
}

export interface FDC3IntentEntity {
  type: EntityTypeEnum.fdc3Intent;
  data: {
    title: string;    // Text displayed on the button raising the intent
    intent: Intents;  // Intent type (ViewChart, etc...)
    context: Context; 
    app?: TargetApp;
  }
}

type Entity = FileAttachmentEntity | FDC3IntentEntity;

export interface Message {
  text: Partial<Record<keyof typeof TextMimeTypeEnum, string>>;
  entities?: Record<string, Entity>;
}

Example

Here is an example of a simple text message with attached PNG image. This type of message could very well be sent to an email client, as well as any IM app out there

const messageWithImage: Message = {
  text: {
    'text/plain': 'Hello, can you see the image attached?'
  },
  entities: {
    '0': {
      type: EntityTypeEnum.fileAttachment,
      data: {
        name: 'myImage.png',
        dataUri: 'data:image/png;base64,iVBORw0KGgoAAAANSUh...'
      }
    }
  }
}

A more advanced example, for clients that are able to take an FDC3 intent and inject it into an action button for example. In future the idea would be to reference this type of intent from within the "text", perhaps as a form of hyperlink

const messageWithIntent: Message = {
  text: {
    'text/plain': 'Hey, have a look at this chart!'
  },
  entities: {
    '0': {
      type: EntityTypeEnum.fdc3Intent,
      data: {
        title: 'Click to view Chart',
        intent: 'ViewChart',
        context: {
          type: 'fdc3.chart',
          instruments: [
            {
              type: 'fdc3.instrument',
              id: {
                ticker: 'EURUSD'
              }
            }
          ],
          range: {
            type: 'fdc3.dateRange',
            starttime: '2020-09-01T08:00:00.000Z',
            endtime: '2020-10-31T08:00:00.000Z'
          },
          style: 'candle'
        }
      }
    }
  }
}

Additional Information

#620

#765

[kriswest]

Thanks @Yannick-Symphony this looks good.

A couple of quick first thoughts:

• I think you should add an entity for just a context (in addition to intent+context) as the API can raise just that (fdc3.rainseIntentForContext) and the DA presents the user with a list of options (you can also retrieve these programmatically via fdc3.findIntentForContext).
  • raiseIntent is an action (choose an app to complete if more than one)
  • raiseIntentForContext is a prompt asking what action do you want to take (from a list)
• I wonder if you might want to reference any of the entities in the text (so you can render a pill or similar) - although that would go beyond test/plain I suppose.
• I think you can drop the data entity and move the subfields to the top level of each entity (more similar to the structure of existing context types).
• type field perhaps will get confused with an fdc3 context type. I think we should either make the entities actual FDC3 context types (well worth doing if they might ever be used on their own...) or rename the field something like entityType to differentiate.
  • I think the entities might well get used on their own: e.g. a file, an action (preconfigured intent and content)

[Yannick-Malins]

Hi @kriswest !

1. For this would we create a new entity, or just make the entity something generic like "fdc3", and then if it contains just a context it's a context, and if it's got intent+context it's an intent? or would that be too messy?
2. Yes, going further the idea would be to reference entities from within the message body - a possibility in markdown would be to do something like look at this chart: [price chart](entity://3) pretty nice (not necessarily that exact syntax, TBD further down the road)
3. +4 : So something like this?

const messageWithIntent: Message = {
  text: {
    'text/plain': 'Hey, have a look at this chart!'
  },
  entities: {
    '0': {
      entityType: EntityTypeEnum.fdc3,
      title: 'Click to view Chart',
      intent: 'ViewChart',
      context: {
        type: 'fdc3.chart',
        instruments: [
          {
            type: 'fdc3.instrument',
            id: {
              ticker: 'EURUSD'
            }
          }
        ],
        range: {
          type: 'fdc3.dateRange',
          starttime: '2020-09-01T08:00:00.000Z',
          endtime: '2020-10-31T08:00:00.000Z'
        },
        style: 'candle'
      }
    }
  }
}

[kriswest]

Hi @Yannick-Symphony, I think this is likely to come up again (on notifications), so perhaps we should create a full context type for an 'action', e.g.:

{
    type: "fdc3.action",
    title: "EUR/USD",     //optional
    intent: undefined,    //optional, i.e. can be omitted to imply raiseIntentForContext rather than raiseIntent
    context: {            //required
         type: 'fdc3.instrument',
         id: {
             ticker: 'EURUSD'
         }
     }
}

However, I suppose there is also a use case for broadcasting that context on a user channel. That could be dealt with by a UI (just offer it as an a resolver option when displaying for raiseIntentForContext), but isn't possible programmatically without adding additional fields (to indicate broadcast and channel)... Perhaps add an optional broadcast field which accepts a string channel name?

Turning this into a full context would mean that entitytype can remain as type and your entities field is Context[]

I've only just noticed that entities is currently Record<string, Entity>, with numeric keys in the example... Any reason not to make it an array? Do you anticipate a need for (non-numeric) string keys?

[Yannick-Malins]

I've only just noticed that entities is currently Record<string, Entity>, with numeric keys in the example... Any reason not to make it an array? Do you anticipate a need for (non-numeric) string keys?

Later on, when we standardise a richer format for the text field, having a map with string keys will make referencing specific entities from within the text cleaner. e.g. look at this chart: [euros to dollars](entity://chart_EURUSD) - pretty nice!

Turning this into a full context would mean that entitytype can remain as type and your entities field is Context[]

It might be wise to keep the flexibility to pass more than standardised fdc3 objects, especially if we want to integrate with generic messaging/email systems: we'll quickly need entity types like "fileAttachment", "inlineImage", "vcard" etc. In which case we'd better keep "entitites" and "entitytype"

so does this match your vision?

export interface FDC3ActionEntity {
  type: EntityTypeEnum.fdc3Action;
  title?: string;    // Text displayed on the button raising the intent
  intent?: Intents;  // if an intent, contains the intent type
  context: Context; 
  app?: TargetApp;
  broadcastChannel?: string; // if broadcast, contains channel
}

example of the bare minimum - a context without title, app or intent:

{
    type: "fdc3.action",
    context: {
         type: 'fdc3.instrument',
         id: {
             ticker: 'EURUSD'
         }
     }
}

[robmoffat]

Based on discussion in the intents and context meeting, it is slightly unclear from the use-case exactly what is being addressed here.

Really, what we are trying to achieve is something like the "share" menu on your phone, where you are presented with a number of destinations (messages, email, WhatsApp) that can accept the shared data.

It might be worth tidying up the Use Case: part of the description along these lines to make it clearer.