Finish populating the canonical SDK feature list

[QuintinWillison]

This is work to complete the feature tree established in ably/ably-common#142.

Certain parts of the tree are almost certainly complete - e.g.:

• client options:
  • shared: Options:
  • REST: Options:
  • Realtime: Options
• Push Notifications: Administration:

But there's plenty where there's either hanging nodes that need children adding, or there are parts of the tree entirely missing.

Sources of truth and/or clues as to what still needs to be added include:

• features/ephemeral-notes.md: Where, during work on Canonical SDK feature list ably-common#142, I was used YAML comment syntax to 'mark off' the nodes from the existing IDL that I had mirrored over to the canonical feature tree. It would make sense to continue marking off in this manner, as an atomic part of subsequent PRs that expand the feature list.
• The Features spec: A simple search for the '#' character tends to reveal where spec points describe methods/functions or properties/fields.
• Existing SDK Repositories: Of particular insight are ably-js, ably-java and ably-ruby. There are some APIs that are present in SDKs but not in the features spec.. Though, in discussion with paddybyers and @jaley, there appears to be general consensus that these may not need to be included in the canonical features list. I'm not sure that is necessarily the case for all of these, so we should still consider them for inclusion, especially if they have potential use across multiple SDKs.

┆Issue is synchronized with this Jira Uncategorised by Unito

[QuintinWillison]

In response to feedback from @owenpearson on ably/ably-common#142, I've decided to remove the ephemeral notes document from that PR. I'm capturing the outstanding items, as far as I'm aware of them, here...

From the current IDL, having removed pure data types and enums (e.g. TokenParams, ErrorInfo) as their use is implicit due to their adoption by the methods and properties that have been kept here:

• class Rest:

  device() => io LocalDevice
  request(
    String method,
    String path,
    Dict<String, String> params?,
    JsonObject | JsonArray body?,
    Dict<String, String> headers
  ) => io HttpPaginatedResponse // RSC19
  stats(
    start: Time api-default epoch(), // RSC6b1
    end: Time api-default now(), // RSC6b1
    direction: .Backwards | .Forwards api-default .Backwards, // RSC6b2
    limit: int api-default 100, // RSC6b3
    unit: .Minute | .Hour | .Day | .Month api-default .Minute // RSC6b4
  ) => io PaginatedResult<Stats> // RSC6a
  time() => io Time // RSC16
  

• class Realtime:

  auth: Auth // RTC4
  device() => io LocalDevice
  clientId: String? // proxy for RSA7
  connection: Connection // RTC2
  request(
    String method,
    String path,
    Dict<String, String> params?,
    JsonObject | JsonArray body?,
    Dict<String, String> headers?
  ) => io HttpPaginatedResponse // RTC9
  stats: // Same as Rest.stats, RTC5a
  close() // proxy for RTN12
  connect() // proxy for RTN11
  time() => io Time // RTC6a
  

• class TokenParams: // RSAA8e

  capability: String api-default '{"*":["*"]}' // RSA9f, TK2b
  clientId: String? // TK2c
  nonce: String? // RSA9c, Tk2d
  timestamp: Time? // RSA9d, Tk2d
  ttl: Duration api-default 60min // RSA9e, TK2a
  

• class TokenDetails:

  +fromJson(String | JsonObject) -> TokenDetails// TD7
  capability: String // TD5
  clientId: String? // TD6
  expires: Time // TD3
  issued: Time // TD4
  token: String // TD2
  

• class TokenRequest:

  +fromJson(String | JsonObject) -> TokenRequest // TE6
  capability: String // TE3
  clientId: String? // TE2
  keyName: String // TE2
  mac: String // TE2
  nonce: String // TE2
  timestamp: Time? // TE5
  ttl: Duration? api-default 60min // TE4
  

• class RestChannel:

  publish(Message, params?: Dict<String, Stringifiable>) => io // RSL1
  publish([Message], params?: Dict<String, Stringifiable>) => io // RSL1
  publish(name: String?, data: Data?) => io // RSL1
  setOptions(options: ChannelOptions) => io // RSL7 - note asynchronous return value for
    // compatibility with RealtimeChannel#setOptions; not required for REST-only libraries
  // Only on platforms that support receiving notifications:
  push: PushChannel // RSH4
  

• class RealtimeChannel:

  embeds EventEmitter<ChannelEvent, ChannelStateChange?> // RTL2a, RTL2d, RTL2e
  errorReason: ErrorInfo? // RTL4e
  state: ChannelState // RTL2b
  presence: RealtimePresence // RTL9
  properties: ChannelProperties // CP1, RTL15
  push: PushChannel
  modes: readonly [ChannelMode] // RTL4m
  params: readonly Dict<String, String> // RTL4k1
  attach() => io // RTL4d
  detach() => io // RTL5e
  history(
    start: Time, // RTL10a
    end: Time api-default now(), // RTL10a
    direction: .Backwards | .Forwards api-default .Backwards, // RTL10a
    limit: int api-default 100, // RTL10a
    untilAttach: Bool default false // RTL10b
  ) => io PaginatedResult<Message> // RSL2a
  publish(Message) => io // RTL6i
  publish([Message]) => io // RTL6i
  publish(name: String?, data: Data?) => io // RTL6i
  subscribe((Message) ->) => io // RTL7a
  subscribe(String, (Message) ->) => io // RTL7b
  unsubscribe() // RTL8a, RTE5
  unsubscribe((Message) ->) // RTL8a
  unsubscribe(String, (Message) ->) // RTL8a
  setOptions(options: ChannelOptions) => io // RTL16
  

• class PushChannel:

  subscribeDevice() => io // RSH7a
  subscribeClient() => io // RSH7b
  unsubscribeDevice() => io // RSH7c
  unsubscribeClient() => io // RSH7d
  listSubscriptions() => io PaginatedResult<PushChannelSubscription> // RSH7e
  

• class ChannelStateChange:

  current: ChannelState // RTL2a, RTL2b
  event: ChannelEvent // TH5
  previous: ChannelState // RTL2a, RTL2b
  reason: ErrorInfo? // RTL2e, TH3
  resumed: Boolean // RTL2f, TH4
  

• class ChannelOptions:

  +withCipherKey(key: Binary | String)? -> ChannelOptions // TB3
  cipher: (CipherParams | Params)? // RSL5a, TB2b
  params?: Dict<String, String> // TB2c
  modes?: [ChannelMode] // TB2d
  

• class CipherParams:

  algorithm: String default "AES" // TZ2a
  key: Binary // TZ2d
  keyLength: Int // TZ2b
  mode: String default "CBC" // TZ2c
  

• class Crypto:

  +getDefaultParams(Params) -> CipherParams // RSE1
  +generateRandomKey(keyLength: Int?) => io Binary // RSE2
  

• class RealtimePresence:

  syncComplete: Bool // RTP13
  get(
    waitForSync: Bool default true, // RTP11c1
    clientId: String?, // RTP11c2
    connectionId: String?, // RTP11c3
  ) => io [PresenceMessage] // RTP11
  history(
    start: Time, // RTP12a
    end: Time, // RTP12a
    direction: .Backwards | .Forwards api-default .Backwards, // RTP12a
    limit: int api-default 100, // RTP12a
  ) => io PaginatedResult<PresenceMessage> // RTP12c
  // presence state modifiers
  enter(Data?, extras?: JsonObject) => io // RTP8
  update(Data?, extras?: JsonObject) => io // RTP9
  leave(Data?, extras?: JsonObject) => io // RTP10
  enterClient(clientId: String, Data?, extras?: JsonObject) => io // RTP4, RTP14, RTP15
  updateClient(clientId: String, Data?, extras?: JsonObject) => io // RTP15
  leaveClient(clientId: String, Data?, extras?: JsonObject) => io // RTP15
  

• class ConnectionDetails:

  clientId: String? // RSA12a, CD2a
  connectionKey: String // RTN15e, CD2b
  connectionStateTtl: Duration // CD2f, RTN14e, DF1a
  maxFrameSize: Int // CD2d
  maxInboundRate: Int // CD2e
  maxMessageSize: Int // CD2c
  serverId: String // CD2g
  maxIdleInterval: Duration // CD2h
  

• class Message:

  constructor(name: String?, data: Data?) // TM2
  constructor(name: String?, data: Data?, clientId: String?) // TM2
  +fromEncoded(JsonObject, ChannelOptions?) -> Message // TM3
  +fromEncodedArray(JsonArray, ChannelOptions?) -> [Message] // TM3
  clientId: String? // RSL1g1, TM2b
  connectionId: String? // TM2c
  data: Data? // TM2d
  encoding: String? // TM2e
  extras: JsonObject? // TM2i
  id: String // TM2a
  name: String? // TM2g
  timestamp: Time // TM2f
  

• class PresenceMessage

  +fromEncoded(JsonObject, ChannelOptions?) -> PresenceMessage // TP4
  +fromEncodedArray(JsonArray, ChannelOptions?) -> [PresenceMessage] // TP4
  action: PresenceAction // TP3b
  clientId: String // TP3c
  connectionId: String // TP3d
  data: Data? // TP3e
  encoding: String? // TP3f
  extras: JsonObject? // TP3i
  id: String // TP3a
  timestamp: Time // TP3g
  memberKey() -> String // TP3h
  

• class AuthDetails: // RTC8

  accessToken: String // AD2
  

• class Connection:

  embeds EventEmitter<ConnectionEvent, ConnectionStateChange> // RTN4a, RTN4e, RTN4g
  errorReason: ErrorInfo? // RTN14a
  id: String? // RTN8
  key: String? // RTN9
  recoveryKey: String? // RTN16b, RTN16c
  serial: Int? // RTN10
  state: ConnectionState // RTN4d
  close() // RTN12
  connect() // RTC1b, RTN3, RTN11
  ping() => io // RTN13
  

• class PushChannelSubscription:

  +forDevice(channel: String, deviceId: String) => PushChannelSubscription
  +forClientId(channel: String, clientId: String) => PushChannelSubscription
  deviceId: String? // PCS2, PCS5, PCS6
  clientId: String? // PCS3, PCS6
  channel: String // PCS4
  

• class VCDiffDecoder

  decode([byte] delta, [byte] base) -> [byte]
  

I've also not included EventEmitter in the above list as it's not really a feature of the SDK. It represents a highly opinionated view as to how a stream of events might be consumed by applications, which doesn't fit with idiomatic foundational APIs available in some languages (e.g. Go channels and Kotlin shared flows).

Things not necessarily fully surfaced by the IDL:

• Environment Fallbacks
• Idempotent Publishing
• Stream Resume
• HTTP/2 support

Also:

• need to ensure to include something representing all current rows in our Feature Support Matrix

[QuintinWillison]

I reached out for feedback from others across our engineering team at 9415be2, where the rendered output was in this state (being the deployed build from #52).

Lewis' Observations

@lmars contributed these observations (in this Slack message (internal)), all of which I will address before closing this issue and declaring the canonical SDK feature list complete as our version 1.2.0 epoch...

Features that perhaps aren't covered

• Pagination (start/end/direction/limit query parameters and accepted values, Link rel headers)
• Errors (how they are represented with message, code, statusCode, href, etc.)
• Fallbacks (when primary host unavailable, try fallbacks in random order, point at random regions so affects latency)
• Publish message with name
• Publish message with headers in extras
• Publish message with references in extras
• Publish JSON serializable object
• Publish multiple messages
• Subscribe to messages by name
• Subscribe to messages by reference
• Token Revocation
• Batch publish
• Batch presence
• Channel status
• Incremental backoff
• User claims

Other points

• I think it would be good to use a consistent tense, perhaps imperative present tense like git recommends for commit messages (https://github.com/git/git/blob/master/Documentation/SubmittingPatches#L181-L187), so "Provide a literal token" rather than "Providing a literal token", or "Return continuous message history" rather than "Used to return continuous message history"
• "Fallback Hosts" is currently under "Testing", but customers who use custom CNAMEs are also given an explicit list of fallbacks, so this option isn't just for testing. Similar for restHost and realtimeHost, they're used for custom CNAMEs too, so not just for development

Marc's Observations

@marcduiker contributed these observations (in this Slack message (internal))...

At the moment it's a bit overwhelming maybe, but I can imagine the nodes could be collapsed to initially show only the first level(s). Would a search be possible as well, and filter on keywords?

To which I responded with:

This current rendering is pretty much intentionally built to be raw and unfiltered. Even once the manifest (compliance declaration) files have moved to the downstream SDK repositories then I think this matrix, in this location, is likely to remain close to its current form (though it can grow those usability enhancements you suggest 😉).

My expectation is that SDK Team will end up working with EdX/Docs/Website to work out how we can expose this matrix detail in a more friendly, customer-facing, distilled form. I think that might be by exposing a subset of the nodes - perhaps transformed into a different, perhaps flat, structure. 🤔

I've created #55 to add collapsible table rows.

[QuintinWillison]

As far as I know I've addressed all observations made, either by merging a pull request or by creating a new issue (defer to later). As such, I'm declaring this issue complete.