diff --git a/README.md b/README.md index 28bd66f..2f80fd7 100644 --- a/README.md +++ b/README.md @@ -2,25 +2,13 @@ The primary goal of a well-designed API for a multi-chain digital wallet is to provide the data required to construct transactions and to allow deriving the current state of the wallet while the set of subscribed blockchains is continuously extended. This includes aggregating transactions and on-chain events to present users their transaction history, current balance, as well as chain-specific features such as stake delegation or staking rewards. -The proposed API design offers different endpoints to retrieve the same data in order to support a wide range of edge clients, +The proposed API design offers different endpoints to retrieve the same data in order to support a wide range of edge clients, in particular clients with an intermittent connection or that are bandwidth-constrained. The API was desgined with an overarching abstraction in mind to focus on the fundamental value that wallets provide: -> #### The ability to transact +> The ability to transact -### What does a wallet need to construct transactions? +## Problem Description [CPS](./docs/CPS/CPS-XXXX/) -In general, to be able to construct transactions the following data is required: - -- any transaction related to a client's wallet (incoming/ receiving & outgoing/ spending) -- staking rewards *if applicable* -- network, era or epoch specific data, like: - - protocol parameters - tx fee calculation (Cardano) - - current fee rate, satoshis per byte, sats/byte (Bitcoin) - - gas limit, max fee per gas, max priority fee per gas, nonce (Ethereum) -- the current tip/ block height for validity intervals of transactions - -## API Design - -We divide the wallet-optimized API into a [push-based, event-driven API](./docs/01-Stream-api.md) and a request/ response API. \ No newline at end of file +## Improvement Proposal [CIP](./docs/CIP/CIP-XXXX/) diff --git a/docs/01-Stream-api.md b/docs/01-Stream-api.md deleted file mode 100644 index 11ad856..0000000 --- a/docs/01-Stream-api.md +++ /dev/null @@ -1,137 +0,0 @@ -# Push-based API Overview - -The push-based API for digital wallets is designed to provide clients with a continuous stream of **relevant** on-chain events, enabling them to derive their current wallet state efficiently. This API supports various blockchain models, including UTXO-based chains and account-based chains. Additionally, clients receive secondary data relevant to the blockchain network they are subscribed to, ensuring comprehensive and up-to-date information. - -The following serves as introduction to the wallet-optimized, event-driven API using **websockets**. -This document outlines the structure and common elements of messages that clients may process as well as the overall protocol. - -## Protocol - -We use a bidirectional protocol and encode all messages as JSON objects. Messages adhere to a specific structure, ensuring consistency and facilitating efficient communication within our event-driven protocol. As we strive to support multiple blockchains, a server-sent message will always reference the originating chain it relates to. - -### Message Format - -Instead of relying on typed messages that may vary highly depending on the underlying blockchain, our design aims to enrich and aggregate data in compound messages. - -#### Aggregate Compound Message Format - -Any server-sent message consists of multiple top level keys with client-relevant data. Below, one can see how a chain reference is part of -a generic server-sent message: - -```json -{ - /* top level message fields like transaction, genesis, rewards, point etc. */ - "chain": { - "blockchain": "cardano", - "network": "mainnet" - /* other blockchain specific fields */ - } -} -``` - -Furthermore, any server-sent message is accompanied by a specific timestamp indicating the exact point in time within the respective chain. - -### Tracking Time on Blockchains - -Blocks on blockchains are often compared to the heartbeat of the chain, representing their own measurement of time. However, the actual time it takes to produce a block on blockchains is non-deterministic. Many independent actors within the distributed system compete to build the next block via Proof-of-Work (PoW), (delegated) Proof of Stake (dPoS/PoS) or another consensus mechanism. Consequently, blocks are produced on a probabilistic basis but are eventually evenly distributed over time with the help of algorithms like Bitcoin's difficulty adjustment or Cardano's protocol parameters that define how frequently blocks can occur for a predefined time period, termed _epoch_. - -Our API needs a reliable measurement of time to construct a stream of ordered, client-relevant, on-chain events that connected clients can easily consume. Time is crucial for clients for various reasons, including: - -- monitoring the progress of synchronization up to the chain's tip -- defining transaction validity intervals - -> [!NOTE] -> Transaction validity intervals allow clients to submit transactions with a specified lifetime, which serves several important purposes: -> -> - protects against replay attacks by ensuring transactions can't be resubmitted after expiration -> - helps manage network congestion by allowing expired transactions to be discarded -> - supports operations that need to occur within specific timeframes - -Some blockchains, like Cardano, define a notion of time based on slots, which are fixed intervals of time (typically seconds). Slot numbers either represent elapsed seconds since the network's genesis block (absolute) or beginning of epoch (epoch). In contrast, other blockchains like Bitcoin don't use a fixed time interval and instead rely on an intrinsic clock derived from block production for transaction validity intervals. - -### Our Protocol's Way of Tracking Time - -We borrow the term `point` from the Cardano blockchain to create an abstract notion of time that is sent along side any sever-sent message. By default, whenever a new transaction is deemed relevant for a connected client, the server enriches the message by adding the current "`point` in on-chain time" of the respective chain. That may look different depending on the blockchain a message relates to: - -#### Bitcoin Point - -```json -{ - "point": { - "height": 849561, - "hash": "00000000000000000000e51f4863683fdcf1ab41eb1eb0d0ab53ee1e69df11bb" - } - /* ... */ -} -``` - -#### Cardano Point - -```json -{ - "point": { - "slot": 127838345, - "hash": "9f06ab6ecce25041b3a55db6c1abe225a65d46f7ff9237a8287a78925b86d10e" - } - /* ... */ -} -``` - -#### Ethereum Point - -```json -{ - "point": { - "height": 20175853, - "hash": "0xc5ae7e8e0107fe45f7f31ddb2c0456ec92547ce288eb606ddda4aec738e3c8ec" - } - /* ... */ -} -``` - -> [!NOTE] -> -> New top level message keys can be added at any time. -> Clients are expected to iterate over messages and skip those they do not support. - -More information for time and event sequencing can be found in the section [Event Sequencing and Synchronization](./messages/index.md#event-sequencing-and-synchronization). - -### Error Handling - -Any cases of failure trigger an error message that is appended to a list of `errors`: - -```json -{ - "errors": [ - { - "type": "error type", - "message": "error message" - } - ] - /* ... */ -} -``` - -## Message Types - -We distinguish between _client-sent_ and _server-sent_ messages. Client messages are typed messages. Each message defines a `type`, whereby server-sent messages follow the aggregated, compound message structure explained above. An index of client message types and server-sent message parts is detailed [here](./messages/index.md). - -## Keep Alive/ Heartbeat Messages - -To conserve server resources, we shift connection maintenance responsibility to clients. This allows the server to terminate any inactive connections not regularly refreshed by clients. - -More information can be found [here](./messages/client/heartbeat.md). - -## Authentication - -API authentication in its first version is implicitly handled by the client specifying the topics of interest in their initial [`subscribe`](./messages/client/subscribe.md) message. This message must include a **signature** that verifies the ownership of the provided credentials, which are used to filter block transactions. - -More information can be found [here](./messages/client/subscribe.md). - -## Other Topics - -[1. Message Type Index](./messages/index.md) - -[2. Synchronization](./02-Synchronization.md) - -[3. Encoding Standard](./03-Encoding.md) diff --git a/docs/02-Synchronization-Sequence.svg b/docs/02-Synchronization-Sequence.svg deleted file mode 100644 index f054d94..0000000 --- a/docs/02-Synchronization-Sequence.svg +++ /dev/null @@ -1 +0,0 @@ -WalletWalletServerServerDBDBProjectorProjectorCardano NodeCardano NodeTask QueueTask QueueProjection WorkerProjection WorkerWallet Connectsconnect(credentials, points: PointOrOrigin[])findIntersection(points)response: intersection point or genesisalt[if point is genesis]<event> {network genesis parameters}Catch-up Phase: Get intersection eventsprotocol parametersprotocolParametersSince(point)response: protocolParameters: [protocol-parameters]era summarieseraSummariesSince(point)response: eraSummaries: [era-summaries]epoch summariesstake summariesstakeSummariesSince(point)response: stakeSummaries: [stake-summary]supply summariessupplySummariesSince(point)response: suppleSummaries: [supply-summary]rewardsrewardsSince(point, credentials)response: rewardsSincePoint: [rewards]transactionstransactionsSince(point, credentials)response: transactionsSincePoint: [transaction + block header]alt[if transaction contains native assets]assetInfo(assetId[])response: assetInfo: [asset meta data]Enrich transaction messageswith native asset meta data.Merge, sort responses by point.<event>{[transactions + block header],[rewards],[epoch summary],[era summary],[protocol parameters]}Wallet Synchronization<event> Chain Syncwrite & notify(extendedBlock)<event> notify(extendedBlock)loop[For each connected client]alt[if block tx is relevant for client:]Enrich tx message(tip, pp etc.)<event> { [transaction + block header] }[if minimum tip threshold interval passed:]<event> { tip }Asynchronous projection<event> Chain Syncinsert or delete taskgetTask()return taskinsert & notify<event> {metadata: {pool:{id, metadata}}<event> {metadata: {pool:{id, metadata}} \ No newline at end of file diff --git a/docs/02-Synchronization.md b/docs/02-Synchronization.md deleted file mode 100644 index 580b450..0000000 --- a/docs/02-Synchronization.md +++ /dev/null @@ -1,3 +0,0 @@ -# Synchronization - -![Sequence Diagram Synchronization](./02-Synchronization-Sequence.svg) \ No newline at end of file diff --git a/docs/03-Encoding.md b/docs/03-Encoding.md deleted file mode 100644 index b219a56..0000000 --- a/docs/03-Encoding.md +++ /dev/null @@ -1,20 +0,0 @@ -# Encoding Standards - -Our proposed communication protocol **adopts blockchain-native encoding standards** to ensure compatibility and efficiency. This approach leverages existing conventions familiar to clients while minimizing conversion overhead. The protocol incorporates the following encoding standards: - -## Transaction Hash Encoding - -Transaction hashes are typically encoded using **hexadecimal representation**. This standard provides a compact and human-readable format for unique transaction identifiers. - -## Address Encoding - -Address encoding varies depending on the blockchain network: - - Bitcoin: Uses Base58Check encoding for legacy addresses and `Bech32` for SegWit addresses. - - Ethereum: Employs hexadecimal encoding with a `0x` prefix. - - Cardano (and others): use custom address formats (e.g., `Bech32` variants). - - -## Native Asset Policy Encoding - -For blockchain networks supporting native assets (e.g., Cardano), policy identifiers are typically encoded in hexadecimal format. - diff --git a/docs/CIP/CIP-XXXX/README.md b/docs/CIP/CIP-XXXX/README.md new file mode 100644 index 0000000..6355d9f --- /dev/null +++ b/docs/CIP/CIP-XXXX/README.md @@ -0,0 +1,259 @@ +--- +CIP: 1 +Title: Event-driven, wallet-optimized API +Status: Proposed +Category: Wallets +Authors: + - William Wolff + - Hızır Sefa İrken + - Rhys Bartels-Waller + - Martynas Kazlauskas + - Daniele Ricci +Implementors: N/A +Solution-To: CPS-???? +Discussions: +Created: 2024-07-10 +License: CC-BY-4.0 +--- + +## Abstract + +We propose a multi-chain, event-driven, push-based API optimized for wallet applications, leveraging the principle that a wallet's state should be a pure function of the blockchain state. This approach eliminates polling strategies and aggregating views, enabling real-time synchronization with the blockchain as new blocks are appended. Our solution addresses key challenges in current wallet implementations, including rollback handling, address discovery and session management for serving stateful clients. + +## Motivation: why is this CIP necessary? + +The CIP provides a partial solution to the problems described in [CPS-????](../CPS/CPS-XXXX/README.md). In particular, we strive to define a standardized approach serving data to edge clients in an unopinionated, close to blockchain native format using a push-based protocol. + +## Specification + +### Introduction + +#### What does a wallet need to construct transactions? + +- past transactions (receiving & spending) to derive balance, UTxO set, delegation and governance state +- reward account balance (staking, voting) +- network, era or epoch specific data, like: + - protocol parameters (tx fees, plutus cost models, ...) + - genesis (security param, slot length) +- the current tip/ block height for validity intervals of transactions as well as sync progress + +#### How do we serve this data? + +Through a push-based API, wallets are provided with a continuous stream of **relevant** on-chain events, enabling them to derive their current wallet state efficiently. This API supports various blockchain models, including UTXO-based chains and account-based chains. Additionally, clients receive secondary data relevant to the blockchain network they are subscribed to, ensuring comprehensive and up-to-date information. + +#### Protocol + +We use a bidirectional protocol and encode all messages as JSON objects, while transactions contained in messages are preserved in the chain's native encoding standard (cbor). Messages adhere to a specific structure, ensuring consistency and facilitating efficient communication within our event-driven protocol. As we strive to support multiple blockchains, a server-sent message will always reference the originating chain it relates to. + +### Message Types & Structure + +We distinguish between client-sent and server-sent messages. + +#### Client + +Client-sent messages are **typed** messages, such as: + +```json5 +{ + "type": "" + // ... +} +``` + +This allows the server to quickly identify the client's intent. More details on what typed messages exist will be covered in [Client Typed Messages](#client-typed-messages) section. + +#### Server + +Instead of typed messages, server-sent messages vary in structure depending on the underlying blockchain. Our design aims to be chain-agnostic and serve data from different blockchains in future. Therefore, we rely on **untyped** messages that are enriched and aggregated into compound messages. We call each top level key a server message partial. + +This leads to a structure that has many top level keys of which, not every key may be present for every message, for example: + +```json +{ + "transactions": [ + /* ... */ + ], + "resolvedInputs": [ + /* ... */ + ], + "rewards": { + /* ... */ + }, + "tip": { + /* ... */ + } + // ... +} +``` + +> [!IMPORTANT] +> +> New message partials can be added at any time, which allows for flexible extension of the API. +> Clients are expected to check for message key presence and ignore those they do not support. + +#### Errors + +Any cases of server-side failure trigger an error message that is append as separate `error` message partial: + +```json +{ + "error": { + "type": "error type", + "message": "error message" + } + /* ... */ +} +``` + +### Authentication + +A newly connected client authenticates as part of the [`subscribe`](./messages/client/subscribe.md) message that is sent to the server to synchronize a specific wallet account up to the chain's current tip. This message must include a **signature** that verifies the ownership of a payment/ stake child private key. + +The private key serves as **authentication** purpose in our design. Here's how it works: + +#### Account Authentication + +Clients connect and authenticate via subscribe messages by providing: + +- a digital signature +- a public key whose Blake2b-256 hash corresponds to one of their provided credentials: payment and stake key hash(es) or script hash (es) in case of shared wallets + +This ensures that only authorized clients can access their account data. + +The `signature` field is generated by signing with the private payment/ stake key a SHA256 HMAC hashed string. The prehash string is constructed by concatenating the subscribed blockchain + timestamp. + +The server verifies the signature, timestamp and whether the Blake2b-256 hash of public key matches one of the credentials before serving any data to the client. If any part of the `subscribe` message is invalid, the server sends an error and closes the connection. + +#### Transaction Querying + +The credentials also enable efficient querying of client-relevant data: + +1. **Indexing Strategy** + + We index transactions by both payment and stake credential. This dual-indexing approach allows for comprehensive transaction tracking. + +2. **BIP32 Wallet Query Process** + + The server queries our index using the corresponding key hashes per client to serve only relevant transactions to subscribed clients. + +### Ordering of Events + +Our API needs a reliable measurement of time to construct a stream of ordered, client-relevant, on-chain events that connected clients can easily consume. Time is crucial for clients for various reasons, including: + +- monitoring the progress of synchronization up to the chain's tip +- defining transaction validity intervals + +> +> - supports operations that need to occur within specific timeframes +> - helps manage network congestion by allowing expired transactions to be discarded +> - protects against replay attacks by ensuring transactions can't be resubmitted after expiration + +We borrow the term `point` from the Cardano blockchain to introduce a server-sent message partial. For instance, if a new transaction is deemed relevant for a connected client, the server enriches the message by adding the current "`point` in on-chain time" of the respective chain. For Cardano, this typically looks as follows: + +#### Cardano Point + +```json +{ + "point": { + "slot": 127838345, + "hash": "9f06ab6ecce25041b3a55db6c1abe225a65d46f7ff9237a8287a78925b86d10e" + } + /* ... */ +} +``` + +> [!NOTE] +> Clients may safely assume that any server-sent message, aside from error messages contain `point` partial. + +### Synchronization + +We refer to synchronization as the process that a wallet undertakes to catch up to the current chain's tip in order to derive its latest state and proceed to transact. The aforedescribed `point` represents the main anchor for wallets, to be able to: + +- resume the sync process since their last known point +- keep track of their current position in the blockchain +- track and handle chain reorganizations (rollbacks) + +The following sequence diagram shows, how a client connects providing an array of `point`s and extended public key. The client provides more than those two bits which is described in the [`subscribe`](./messages/client/subscribe.md) message. +Subsequently, the server tries to find an intersection given the list of `point`s. + +For an identified intersection, the server computes and publishes an ordered stream of events. +This event stream provides a chronological record of on-chain events and affectively state changes for the wallet to derive its latest state. + +> [!NOTE] +> The server sources client-relevant data through credential-dependent, point-specific database queries, though the implementation details are beyond this CIP's scope. + +![Sequence Diagram Synchronization](./images/02-Synchronization-Sequence.svg) + +#### Resumability + +One of the traditional operational challenges of serving stateful clients is session management during times of scaling. In our protocol, the server can disconnect clients at any time while clients can resume synchronization from their last known `point`s after reconnecting. To minimize reconnection attempts, clients should provide a list of `point`s based on their most recent state, including some that may have been affected by rollbacks. This approach reduces the likelihood of the server failing to find a valid intersection, which would otherwise result in dropped connections and additional round trips. + +> [!NOTE] +> More details are described in the [`subscribe`](./messages/client/subscribe.md) message. + +#### Rollback Handling + +In a [CAP](https://en.wikipedia.org/wiki/CAP_theorem) system like Cardano, which balances global consistency with availability, there is a challenge in dealing with rollback events that impact transaction finality. Regardless of whether one uses their own full node or a server provider for transaction submission - if such events are not properly handled with the help of monitoring the rolling window of [k blocks](https://plutus-apps.readthedocs.io/en/latest/plutus/explanations/rollback.html) and respective resubmission, there is no guarantee that the transaction is finalized, eg. becomes part of the immutable pefix of the Cardano blockchain. + +Since our wallet-optimized protocol defines that server-sent messages include the `point` message partial, the client can check if its last `point` is greater than any subsequent message `point` from the server, without the need to introduce a `rollback` specific message partial. This assumes that the client keeps a local copy of the volatile segment of the blockchain defined by the security parameter. + +#### Connection Management + +Clients are expected to reconnect on error, when providing invalid `point`s for which a server cannot find an intersection. +Clients may send multiple [`subscribe`](./messages/client/subscribe.md) messages for different accounts, but subsequently take responsibility to coordinate server-sent events by time to their possibly differently synchronized accounts. When receiving a message that references a `point` that the first account has already seen, the client only applies it for its second account for example. + +### Versioning + +We propose to version message partials individually. In our protocol, the client sends its list of supported versions as part of the initial [`subscribe`](./messages/client/subscribe.md) message. Upon receiving this information, the server selects the highest mutually supported version and associates it with the client. +This method requires the server to implement version-based message construction, allowing it to tailor its responses to each client's supported version. + +### Encoding + +Our proposed communication protocol **adopts blockchain-native encoding standards** to ensure compatibility and efficiency. This approach leverages existing conventions familiar to clients while minimizing conversion overhead on the server-side. (see [CPS - Fragile API](../../CPS/CPS-XXXX/README.md#fragile-api)) + +#### Address Encoding + +Address encoding varies depending on the blockchain network. In Cardano the protocol servers bech32 encoded addresses. + +### Extensions + +Clients can turn on certain `extensions` by setting a respective flags in their subscription [`config](./messages/client/subscribe.md#subscribe-example) which tells the server to for example: + +- to enrich transaction output asset values by their respective off-chain metadata such as decimal offsets +- to resolve transaction inputs (transaction output references) as transaction outputs + +## Rationale: how does this CIP achieve its goals? + +A primary consideration for the protocol design was to minimize message round trips between client and server as well as the amount of work to be done by the server for each client. Thus, the API's authentication does not require an initial message exchange but rather defines a standard verifable method for the server to either drop a client early and proceed to publish its relevant data. + +### Client Typed Messages + +- [`heartbeat`](./messages/client/heartbeat.md): Connection management/ keep alive + +> [!NOTE] +> Since the API supports extensions, transport-specific messages can be implemented such as this. + +- [`subscribe`](./messages/client/subscribe.md): Define topics of interest & authentication + +### Server Message Partials + +- [`welcome`](./messages/server/welcome.md): Service details, sent once immediately upon establishing a connection +- [`genesis`](./messages/server/genesis.md): If applicable for the subscribed blockchain, it serves static data used as part of the network bootstrap +- [`transaction`](./messages/server/transaction.md): A new transaction +- [`tip`](./messages/server/tip.md): A new block was appended +- [`protocol parameters`](./messages/server/protocol-parameters.md): Epoch based message for updated protocol parameters +- [`era summary`](./messages/server/era-summary.md): Era based message for updated slot length + +## Path to Active + +### Acceptance Criteria + +[!TODO] + +### Implementation Plan + +[!TODO] + +## Copyright + +This CIP is licensed under [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/legalcode). diff --git a/docs/CIP/CIP-XXXX/images/02-Synchronization-Sequence.svg b/docs/CIP/CIP-XXXX/images/02-Synchronization-Sequence.svg new file mode 100644 index 0000000..9dde2f3 --- /dev/null +++ b/docs/CIP/CIP-XXXX/images/02-Synchronization-Sequence.svg @@ -0,0 +1 @@ +WalletWalletServerServerDBDBProjectorProjectorCardano NodeCardano NodeTask QueueTask QueueProjection WorkerProjection WorkerWallet Connects<event> welcome {service details}subscribe(credentials, points: PointOrOrigin[])findIntersection(points)response: intersection point or genesisalt[if no intersection found]<event> { invalid point error }subscribe(points: PointOrOrigin[])alt[if point is genesis]<event> {network genesis parameters}Replay Phase: Get intersection eventsprotocol parametersprotocolParametersSince(point)response: protocolParameters: [protocol-parameters]era summarieseraSummariesSince(point)response: eraSummaries: [era-summaries]epoch summariesstake summariesstakeSummariesSince(point)response: stakeSummaries: [stake-summary]supply summariessupplySummariesSince(point)response: suppleSummaries: [supply-summary]rewardsrewardsSince(point, credentials)response: rewardsSincePoint: [rewards]transactionstransactionsSince(point, credentials)response: transactionsSincePoint: [transaction + block header]alt[if transaction contains native assets]assetInfo(assetId[])response: assetInfo: [asset meta data]Enrich transaction messageswith native asset meta data.Merge, sort responses by point.<event>{[transactions + block header],[rewards],[epoch summary],[era summary],[protocol parameters]}Follow Tip<event> Chain Syncwrite & notify(extendedBlock)<event> notify(extendedBlock)loop[For each connected client]alt[if block tx is relevant for client:]Enrich tx message(tip, pp etc.)<event> { [transaction + block header] }[if minimum tip threshold interval passed:]<event> { tip } \ No newline at end of file diff --git a/docs/messages/client/heartbeat.md b/docs/CIP/CIP-XXXX/messages/client/heartbeat.md similarity index 100% rename from docs/messages/client/heartbeat.md rename to docs/CIP/CIP-XXXX/messages/client/heartbeat.md diff --git a/docs/CIP/CIP-XXXX/messages/client/subscribe.md b/docs/CIP/CIP-XXXX/messages/client/subscribe.md new file mode 100644 index 0000000..3edccf5 --- /dev/null +++ b/docs/CIP/CIP-XXXX/messages/client/subscribe.md @@ -0,0 +1,285 @@ +# Client Subscribe Message + +Sent by client to server right after establishing a connection. New clients must authenticate themselves when subscribing to any blockchain events via a signature. + +## Schema + +```json +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "type": "object", + "properties": { + "type": { + "type": "string", + "enum": ["subscribe"] + }, + "topic": { + "type": "array", + "items": { + "type": "object", + "properties": { + "blockchain": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "network": { + "type": "string" + } + }, + "required": ["name", "network"] + }, + "signature": { + "type": "string", + "pattern": "^[A-Za-z0-9+/=]*$" + }, + /* */ + "cardano": { + "type": "object", + "properties": { + "credentials": { + "type": "object", + "properties": { + "payment": { + "type": "array", + "items": { + "type": "string", + "pattern": "^[A-Za-z0-9+/=]*$" + } + }, + "stake": { + "type": "array", + "items": { + "type": "string", + "pattern": "^[A-Za-z0-9+/=]*$" + } + } + }, + "required": ["payment", "stake"] + }, + "points": { + "type": "array", + "properties": { + "slot": { + "type": "integer", + "minimum": 0 + }, + "hash": { + "type": "string", + "pattern": "^[A-Za-z0-9+/=]*$" + } + }, + "required": ["slot", "hash"] + }, + "extensions": { + "type": "array", + "items": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "config": {}, + "version": { + "type": "string" + } + }, + "required": ["name", "config", "version"] + } + } + }, + "required": ["credentials", "extensions", "points"] + } + }, + "required": ["blockchain", "signature"] + } + }, + "timestamp": { + "type": "string", + "format": "date-time" + } + }, + "required": ["type", "topic", "timestamp"], + "additionalProperties": false +} +``` + +The `signature` field is generated by signing a SHA256 HMAC hashed string. The prehash string is constructed by concatenating the subscribed `blockchain` + `timestamp`. + +> [!NOTE] +> For subscribing to multiple blockchains, the prehash string must be generated for each as `.`, for example: +> `cardano.mainnet+2024-06-27T12:34:56Z`. + +Apply a SHA256 HMAC using either the payment signing key or staking signing key, and then base64-encode it as final payload within the initial message. Pass the output of this SHA256 HMAC to the signature field. + +The signature verifies ownership of the private key, whose corresponding public key is used to filter relevant block transactions. + +## Subscribe Example + +```json +{ + "type": "subscribe", + "topic": { + "blockchain": { "name": "cardano", "network": "mainnet" }, + "signature": "OGNiOWIyNGVjOTMxZmY3N2MzYjQxOTY3OWE0YTcwMzczZmVkZmIxNDZmMDE0ODk0Nzg4YjUxMmIzMjE4MDdiYw==", // base64, SHA256 HMAC with your signing key + "cardano": { + "credentials": { + "payment": [ "0d166978f407505f157c9f56fdb3358e60d1589295b2f7a1e66c1574" ], + "stake": [ "82c00414a674fd7e7657aa5634e2086910c2f210e87f22ce880a0063" ] + }, + "points": [ + { + "slot": 66268628, + "hash": "47b8ec3a58a4a69cb5e3397c36cb3966913882fa8179cae10a5d3f9319c4ae66" + }, + { + "slot": 87868775, + "hash": "074985b22edc01b9579a2e571dc125e044aecf812ee45d50e6fb6fef979fd0d0" + } + ], + "extensions": [ + { + "name": "resolveTxInput", + "config": true, + "version": "1.0" + }, + { + "name": "assetMetadata", + "config": false, + "version": "1.0" + } + ] + } + }, + "timestamp": "2024-06-27T12:34:56Z" +} +``` + +## Subscription Extensions + +Extensions are a way for clients to define if the server shall enable/disable some features like enriching data that it publishes. When serving data on-chain it may reference off-chain data or past on-chain data +which is more complex to track. To simplify the client-side consumption of such events, the server can aggregate or pull data from other datasources to enrich the +events it serves to clients at their discretion. To switch extensions on/off, the client provides an array of choices. +Setting extension's config value to `false` will switch it off if the extension is on by default. + +### Resolving transaction inputs + +For UTxO blockchains like Bitcoin or Cardano, transaction inputs have the following structure: + +```json +{ + "outputIndex": { + "type": "number" + }, + "txHash": { + "type": "string" + } +} +``` + +When setting the `resolveTxInput` extension config to `true`, the server will include the transaction input address that was spent. + +### Asset Metadata + +[!TODO] + +## Subscription Object + +The subscription object is blockchain specific, because there are different networks, credentials or other fields required. +Chain specific fields are grouped under a specific field named after the given chain. +In the case of Cardano blockchain, all the relevant fields are placed under a field named `cardano`. +Below we list the currently supported subscriptions: + +### Cardano + +This is the format for a Cardano specific object in the `topic` object. A client can subscribe to multiple topics one +after another at any given time. A client can subscribe to multiple accounts by providing multiple credentials in the +`credentials` array. +The client can provide multiple starting `points` in the mandatory `points` array. +The first valid point provided in the array will be used as the starting point. If all the points are invalid, the +subscription will fail. +In case of an empty array, starting point will be the genesis. + +```json +{ + "blockchain": { + "name": "cardano", + "network": { + "type": "string", + "enum": ["mainnet", "preprod", "preview"] + } + }, + "signature": { + "type": "string", + "pattern": "^[A-Za-z0-9+/=]*$" + }, + "cardano": { + "type": "object", + "properties": { + "credentials": { + "type": "object", + "properties": { + "payment": { + "type": "array", + "items": { + "type": "string", + "pattern": "^[A-Za-z0-9+/=]*$" + } + }, + "stake": { + "type": "array", + "items": { + "type": "string", + "pattern": "^[A-Za-z0-9+/=]*$" + } + } + }, + "required": ["payment", "stake"] + }, + "points": { + "type": "array", + "properties": { + "slot": { + "type": "integer", + "minimum": 0 + }, + "hash": { + "type": "string", + "pattern": "^[A-Za-z0-9+/=]*$" + } + }, + "required": ["slot", "hash"] + }, + "extensions": { + "type": "array", + "items": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "config": {}, + "version": { + "type": "string" + } + }, + "required": ["name", "config", "version"] + } + } + }, + "required": ["credentials", "points", "extensions"] + }, + "required": ["blockchain", "signature", "cardano"] +} +``` + +### Other blockchains + +Any other blockchain schema should follow the same structure as the above Cardano example by applying these: + +- Group blockchain specific properties under one property named after the blockchain (see `cardano` field) +- Keep chain specific authentication fields under `credentials` property +- Keep chain specific configuration and extensions under `extensions` property +- Keep chain specific starting point fields in `points` array. If not supporting multiple points, use a `point` object. +- Define a JSON Schema in the documentation diff --git a/docs/messages/client/unsubscribe.md b/docs/CIP/CIP-XXXX/messages/client/unsubscribe.md similarity index 94% rename from docs/messages/client/unsubscribe.md rename to docs/CIP/CIP-XXXX/messages/client/unsubscribe.md index 539ecee..04b8952 100644 --- a/docs/messages/client/unsubscribe.md +++ b/docs/CIP/CIP-XXXX/messages/client/unsubscribe.md @@ -31,6 +31,6 @@ Upon receiving an unsubscribe message, the server will: { "type": "acknowledge", "status": "success", - "timestamp": "2024-06-27T14:30:00Z", + "timestamp": "2024-06-27T14:30:00Z" } -``` +``` \ No newline at end of file diff --git a/docs/messages/server/cardano/era-summary.md b/docs/CIP/CIP-XXXX/messages/server/era-summary.md similarity index 100% rename from docs/messages/server/cardano/era-summary.md rename to docs/CIP/CIP-XXXX/messages/server/era-summary.md diff --git a/docs/messages/server/genesis.md b/docs/CIP/CIP-XXXX/messages/server/genesis.md similarity index 100% rename from docs/messages/server/genesis.md rename to docs/CIP/CIP-XXXX/messages/server/genesis.md diff --git a/docs/messages/server/cardano/protocol-parameters.md b/docs/CIP/CIP-XXXX/messages/server/protocol-parameters.md similarity index 100% rename from docs/messages/server/cardano/protocol-parameters.md rename to docs/CIP/CIP-XXXX/messages/server/protocol-parameters.md diff --git a/docs/messages/server/cardano/rewards.md b/docs/CIP/CIP-XXXX/messages/server/rewards.md similarity index 100% rename from docs/messages/server/cardano/rewards.md rename to docs/CIP/CIP-XXXX/messages/server/rewards.md diff --git a/docs/messages/server/tip.md b/docs/CIP/CIP-XXXX/messages/server/tip.md similarity index 100% rename from docs/messages/server/tip.md rename to docs/CIP/CIP-XXXX/messages/server/tip.md diff --git a/docs/messages/server/transaction.md b/docs/CIP/CIP-XXXX/messages/server/transaction.md similarity index 100% rename from docs/messages/server/transaction.md rename to docs/CIP/CIP-XXXX/messages/server/transaction.md diff --git a/docs/CIP/CIP-XXXX/messages/server/welcome.md b/docs/CIP/CIP-XXXX/messages/server/welcome.md new file mode 100644 index 0000000..7689466 --- /dev/null +++ b/docs/CIP/CIP-XXXX/messages/server/welcome.md @@ -0,0 +1,125 @@ +# Welcome Message Partial + +This message is sent to all clients as the first message immediately upon establishing a connection. +It contains essential details about the server and API, helping clients understand the environment they are interacting +with. + +The message can contain: + +- A list of supported blockchains, networks and extensions +- Versioning information + +## Schema + +```json +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "title": "Schema for Welcome message partial", + "type": "object", + "properties": { + "welcome": { + "type": "object", + "properties": { + "blockchains": { + "type": "array", + "items": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "network": { + "type": "string" + }, + "extensions": { + "type": "array", + "items": { + "type": "object", + "properties": { + "name": { + "type": "string" + }, + "config": {}, // using 'false' will switch off the extension + "default": {}, + "switchable": { + "type": "boolean" + }, + "versions": { + "type": "array", + "items": { + "type": "string" + } + } + }, + "required": [ + "name", + "config", + "default", + "switchable", + "versions" + ] + } + }, + "version": { + "type": "string" + } + }, + "required": [ + "name", + "network", + "extensions", + "version" + ] + } + } + }, + "required": [ + "blockchains" + ] + } + }, + "required": [ + "welcome" + ] +} +``` + +### Message Example + +This message will contain details for all the served blockchains and networks in the API. + +```json +{ + "welcome": { + "blockchains": [ + { + "name": "cardano", + "network": "mainnet", + "extensions": [ + { + "name": "assetMetadata", + "config": false, + "switchable": true, + "versions": [ + "1.0", + "1.1" + ] + }, + { + "name": "CIP-XXXX", + "config": { + "timeout": 10 + }, + "switchable": false, + "versions": [ + "1.0" + ] + } + ], + "uri": "wss://www.example.com/socketserver", + "version": "0.1" + } + ] + } +} +``` diff --git a/docs/02-Synchronization.puml b/docs/CIP/CIP-XXXX/src/02-Synchronization-Sequence.puml similarity index 82% rename from docs/02-Synchronization.puml rename to docs/CIP/CIP-XXXX/src/02-Synchronization-Sequence.puml index eed5d7c..32c2fda 100644 --- a/docs/02-Synchronization.puml +++ b/docs/CIP/CIP-XXXX/src/02-Synchronization-Sequence.puml @@ -9,15 +9,21 @@ queue "Task Queue" as Queue participant "Projection Worker" as Worker group Wallet Connects - Wallet -> Server: connect(credentials, points: PointOrOrigin[]) + Server -> Wallet: welcome {service details} + Wallet -> Server: subscribe(credentials, points: PointOrOrigin[]) Server -> DB: findIntersection(points) DB -> Server: response: intersection point or genesis + alt if no intersection found + Server -> Wallet: { invalid point error } + Wallet --> Server: subscribe(points: PointOrOrigin[]) + end + alt if point is genesis Server -> Wallet: {network genesis parameters} end - group Catch-up Phase: Get intersection events + group Replay Phase: Get intersection events group protocol parameters Server -> DB: protocolParametersSince(point) DB --> Server: response: protocolParameters: [protocol-parameters] @@ -60,7 +66,7 @@ group Wallet Connects end -group Wallet Synchronization +group Follow Tip Node -> Projector: Chain Sync Projector -> DB: write & notify(extendedBlock) DB -> Server: notify(extendedBlock) @@ -75,14 +81,4 @@ group Wallet Synchronization end end -group Asynchronous projection - Node -> Projector: Chain Sync - Projector -> Queue: insert or delete task - Worker -> Queue: getTask() - Queue --> Worker: return task - Worker -> DB: insert & notify - DB -> Server: {metadata: {pool:{id, metadata}} - Server -> Wallet: {metadata: {pool:{id, metadata}} -end - @enduml diff --git a/docs/CIP/CIP-XXXX/src/cardano-service.yml b/docs/CIP/CIP-XXXX/src/cardano-service.yml new file mode 100644 index 0000000..effd153 --- /dev/null +++ b/docs/CIP/CIP-XXXX/src/cardano-service.yml @@ -0,0 +1,658 @@ +asyncapi: 3.0.0 +info: + title: Wallet API + version: 1.0.0 + description: >- + The primary goal of a well-designed API for a multi-chain digital wallet is + to provide the data required to construct transactions and to allow deriving + the current state of the wallet while the set of subscribed blockchains is + continuously extended. This includes aggregating transactions and on-chain + events to present users their transaction history, current balance, as well + as chain-specific features such as stake delegation or staking rewards. + + + The proposed API design offers different endpoints to retrieve the same data + in order to support a wide range of edge clients, in particular clients with + an intermittent connection or that are bandwidth-constrained. + license: + name: Apache 2.0 + url: 'https://www.apache.org/licenses/LICENSE-2.0' +defaultContentType: application/json +servers: + local-server: + host: '127.0.0.1:{port}' + protocol: wss + description: Default instance, when running a local server. + variables: + port: + default: '8080' + tags: + - name: 'env:local' + description: This environment is for local + - name: 'visibility:private' + description: This resource is locally visible +channels: + cardano: + address: /v1/wallet + messages: + welcome: + $ref: '#/components/messages/welcome' + genesis: + $ref: '#/components/messages/genesis' + transaction: + $ref: '#/components/messages/transaction' + protocolParameters: + $ref: '#/components/messages/protocolParameters' + eraSummary: + $ref: '#/components/messages/eraSummary' + subscribe: + $ref: '#/components/messages/subscribe' +operations: + doSubscribe: + summary: A client subscription + description: >- + Sent by client to server right after establishing a connection. New + clients must authenticate themselves when subscribing to any blockchain + events via a signature. + action: send + messages: + - $ref: '#/channels/cardano/messages/subscribe' + channel: + $ref: '#/channels/cardano' + onConnect: + summary: On client connection + description: Event received when a client connects to the server + action: receive + messages: + - $ref: '#/channels/cardano/messages/welcome' + channel: + $ref: '#/channels/cardano' + onGenesis: + summary: On client syncing the genesis + description: Event received if the client is syncing from the origin + action: receive + messages: + - $ref: '#/channels/cardano/messages/genesis' + channel: + $ref: '#/channels/cardano' + onTransaction: + summary: On a relevant transaction event + description: Event received if there is a relevant transaction for the client + action: receive + messages: + - $ref: '#/channels/cardano/messages/transaction' + channel: + $ref: '#/channels/cardano' + onEpochBoundary: + summary: On epoch boundry + description: Event receieved on every epoch boundary transition + action: receive + messages: + - $ref: '#/channels/cardano/messages/protocolParameters' + channel: + $ref: '#/channels/cardano' + onEraTransition: + summary: On era transtitions + description: Event receieved on era transitions for updated slot length + action: receive + messages: + - $ref: '#/channels/cardano/messages/eraSummary' + channel: + $ref: '#/channels/cardano' +components: + messages: + welcome: + name: Welcome + title: Welcome message + summary: >- + This message is sent to all clients as the first message immediately + upon establishing a connection. It contains essential details about + the server and API, helping clients understand the environment they are + interacting with. + contentType: application/json + payload: + $ref: '#/components/schemas/welcomePayload' + examples: + - + name: Welcome + summary: Welcome message example + payload: + blockchains: + - name: cardano + network: mainnet + extensions: + - name: assetMetadata + config: false + switchable: true + versions: + - '1.0' + - '1.1' + - name: CIP-XXXX + config: + timeout: 10 + switchable: false + versions: + - '1.0' + uri: wss://www.example.com/socketserver + version: '0.1' + + genesis: + name: Genesis + title: Genesis message + summary: >- + This top level message key is added by the server as part of any + synchronization process that starts from the genesis point, also + sometimes referred to origin. + contentType: application/json + payload: + $ref: '#/components/schemas/genesisPayload' + transaction: + name: Transaction + title: Transaction message + summary: >- + The server broadcasts new, relevant transaction data to connected + clients using chain-specific encoding protocols. This approach closely + mimics the native blockchain synchronization process, ensuring + compatibility and efficiency. By adhering to each blockchain's native + encoding, the system maintains consistency with existing communication + protocols. + contentType: application/json + payload: + $ref: '#/components/schemas/transactionPayload' + protocolParameters: + name: protocolParameters + title: Protocol Parameters Server Message Partial + summary: >- + This top level message key is added by the server as part of any + synchronization process for every epoch boundary transition. + contentType: application/json + payload: + $ref: '#/components/schemas/protocolParameterPayload' + examples: + - + name: protocolParameters + summary: Protocol Parameters message example + payload: + epoch: 225 + minFeeA: 44 + minFeeB: 155381 + maxBlockSize: 65536 + maxTxSize: 16384 + maxBlockHeaderSize: 1100 + keyDeposit: '2000000' + poolDeposit: '500000000' + eMax: 18 + nOpt: 150 + a0: 0.3 + rho: 0.003 + tau: 0.2 + decentralisationParam: 0.5 + extraEntropy: + protocolMajorVer: 2 + protocolMinorVer: 0 + minUtxo: '1000000' + minPoolCost: '340000000' + nonce: 1a3be38bcbb7911969283716ad7aa550250226b76a61fc51cc9a9a35d9276d81 + costModels: + plutusV1: + addIntegerCpuArgumentsIntercept: 197209 + addIntegerCpuArgumentsSlope: 0 + plutusV2: + addIntegerCpuArgumentsIntercept: 197209 + addIntegerCpuArgumentsSlope: 0 + priceMem: 0.0577 + priceStep: 7.21e-05 + maxTxExMem: '10000000' + maxTxExSteps: '10000000000' + maxBlockExMem: '50000000' + maxBlockExSteps: '40000000000' + maxValSize: '5000' + collateralPercent: 150 + maxCollateralInputs: 3 + coinsPerUtxoSize: '34482' + tip: + name: Tip + title: Server Tip Message Partial + summary: >- + This message partial is added to all server-sent messages and represents + the latest current tip of the chain. + payload: + $ref: '#/components/schemas/tipPayload' + examples: + - + name: Server Tip message + summary: Cardano Example + payload: + tip: + point: + slot: 127838345 + hash: 9f06ab6ecce25041b3a55db6c1abe225a65d46f7ff9237a8287a78925b86d10e + - + name: Server Tip message + summary: Bitcnoin Example + payload: + tip: + point: + height: 849561 + hash: 00000000000000000000e51f4863683fdcf1ab41eb1eb0d0ab53ee1e69df11bb + + eraSummary: + name: Era Summary + title: Era Summary Server Message Partial + summary: >- + Part of any synchronization process for every era transition. It has + similarities with hardfork events on other blockchains but has been + represented by its own message partial. The primary reason a wallet + needs to be aware of era transitions in Cardano is due to potential + network changes. Specifically, since the beginning of Cardano's Shelley + era, the Ouroboros consensus protocol was introduced, which defined a + slot length—a specific duration of time during which a block can be + produced by a leader (stake pool). Initially, this slot length was set + to one second but may change in the future. Therefore, depending on the + slot length, the conversion of a specific point in on-chain time may + vary. In order to show and submit transaction times correctly, wallets + need to know each era's slot length. + payload: + $ref: '#/components/schemas/eraSummaryPayload' + examples: + - + name: Era Summary Message + summary: Era Summary Message example + payload: + parameters: + epochLength: 432000 + slotLength: 1 + safeZone: 129600 + start: + epoch: 74 + slot: 1598400 + time: '2020-09-11T08:36:51.000Z' + end: + epoch: 102 + slot: 13694400 + time: '2020-12-30T05:04:51.000Z' + subscribe: + name: Subscribe + title: Subscribe message + summary: >- + A message containing client's interests and preferences. New clients + must authenticate themselves when subscribing to any blockchain events + via a signature. + contentType: application/json + payload: + $ref: '#/components/schemas/subscribePayload' + examples: + - + name: Subscribe + summary: Subscribe message + payload: + type: subscribe + topic: + blockchain: + name: cardano + network: mainnet + signature: OGNiOWIyNGVjOTMxZmY3N2MzYjQxOTY3OWE0YTcwMzczZmVkZmIxNDZmMDE0ODk0Nzg4YjUxMmIzMjE4MDdiYw== + cardano: + credentials: + payment: + - 0d166978f407505f157c9f56fdb3358e60d1589295b2f7a1e66c1574 + stake: + - 82c00414a674fd7e7657aa5634e2086910c2f210e87f22ce880a0063 + points: + - slot: 66268628 + hash: 47b8ec3a58a4a69cb5e3397c36cb3966913882fa8179cae10a5d3f9319c4ae66 + - slot: 87868775 + hash: '074985b22edc01b9579a2e571dc125e044aecf812ee45d50e6fb6fef979fd0d0' + extensions: + - name: resolveTxInput + config: true + version: '1.0' + - name: assetMetadata + config: false + version: '1.0' + timestamp: '2024-06-27T12:34:56Z' + + schemas: + welcomePayload: + type: object + properties: + blockchains: + type: array + items: + type: object + properties: + name: + type: string + network: + type: string + extensions: + type: array + items: + type: object + properties: + name: + type: string + config: {} + default: {} + switchable: + type: boolean + versions: + type: array + items: + type: string + required: + - name + - config + - default + - switchable + - versions + version: + type: string + required: + - name + - network + - extensions + - version + required: + - blockchains + genesisPayload: + type: object + properties: + activeSlotsCoefficient: + type: number + format: float + updateQuorum: + type: integer + maxLovelaceSupply: + type: string + networkMagic: + type: integer + epochLength: + type: integer + systemStart: + type: integer + slotsPerKesPeriod: + type: integer + slotLength: + type: integer + maxKesEvolutions: + type: integer + securityParam: + type: integer + required: + - activeSlotsCoefficient + - updateQuorum + - maxLovelaceSupply + - networkMagic + - epochLength + - systemStart + - slotsPerKesPeriod + - slotLength + - maxKesEvolutions + - securityParam + transactionPayload: + type: object + properties: + outputIndex: + type: number + txHash: + type: string + transactions: + type: array + format: string + resolvedInputs: + type: array + format: string + point: + type: object + $ref: '#/components/schemas/pointPayload' + chain: + type: string + tip: + type: object + $ref: '#/components/schemas/pointPayload' + pointPayload: + type: object + properties: + slot: + type: integer + hash: + type: string + height: + type: string + required: + - hash + tipPayload: + type: object + properties: + point: + $ref: '#/components/schemas/pointPayload' + required: + - point + protocolParameterPayload: + type: object + properties: + epoch: + type: integer + minFeeA: + type: integer + minFeeB: + type: integer + maxBlockSize: + type: integer + maxTxSize: + type: integer + maxBlockHeaderSize: + type: integer + keyDeposit: + type: string + poolDeposit: + type: string + eMax: + type: integer + nOpt: + type: integer + a0: + type: number + rho: + type: number + tau: + type: number + decentralisationParam: + type: number + extraEntropy: + type: + - 'null' + - string + protocolMajorVer: + type: integer + protocolMinorVer: + type: integer + minUtxo: + type: string + minPoolCost: + type: string + nonce: + type: string + costModels: + type: object + properties: + plutusV1: + type: object + plutusV2: + type: object + priceMem: + type: number + priceStep: + type: number + maxTxExMem: + type: string + maxTxExSteps: + type: string + maxBlockExMem: + type: string + maxBlockExSteps: + type: string + maxValSize: + type: string + collateralPercent: + type: integer + maxCollateralInputs: + type: integer + coinsPerUtxoSize: + type: string + required: + - epoch + - minFeeA + - minFeeB + - maxBlockSize + - maxTxSize + - maxBlockHeaderSize + - keyDeposit + - poolDeposit + - eMax + - nOpt + - a0 + - rho + - tau + - decentralisationParam + - extraEntropy + - protocolMajorVer + - protocolMinorVer + - minUtxo + - minPoolCost + - nonce + - costModels + - priceMem + - priceStep + - maxTxExMem + - maxTxExSteps + - maxBlockExMem + - maxBlockExSteps + - maxValSize + - collateralPercent + - maxCollateralInputs + - coinsPerUtxoSize + eraSummaryPayload: + type: object + properties: + parameters: + type: object + properties: + epochLength: + type: integer + slotLength: + type: integer + description: Milliseconds + required: + - epochLength + - slotLength + start: + type: object + properties: + slot: + type: integer + time: + type: string + format: date-time + required: + - slot + - time + required: + - parameters + - start + subscribePayload: + type: object + properties: + type: + type: string + enum: + - subscribe + topic: + type: array + items: + type: object + properties: + blockchain: + type: object + properties: + name: + type: string + network: + type: string + required: + - name + - network + publicKey: + type: string + pattern: '^[A-Za-z0-9+/=]*$' + signature: + type: string + pattern: '^[A-Za-z0-9+/=]*$' + cardano: + type: object + properties: + credentials: + type: object + properties: + payment: + type: array + items: + type: string + pattern: '^[A-Za-z0-9+/=]*$' + stake: + type: array + items: + type: string + pattern: '^[A-Za-z0-9+/=]*$' + required: + - payment + - stake + points: + type: array + items: + type: object + $ref: '#/components/schemas/pointPayload' + properties: + slot: + type: integer + minimum: 0 + hash: + type: string + pattern: '^[A-Za-z0-9+/=]*$' + required: + - slot + - hash + extensions: + type: array + items: + type: object + properties: + name: + type: string + config: + type: object + version: + type: string + required: + - name + - config + - version + required: + - credentials + - extensions + - points + required: + - blockchain + - signature + timestamp: + type: string + format: date-time + required: + - type + - topic + - timestamp + sentAt: + type: string + format: date-time + description: Date and time when the message was sent. diff --git a/docs/CPS/CPS-XXXX/README.md b/docs/CPS/CPS-XXXX/README.md index a1c8b33..45b296e 100644 --- a/docs/CPS/CPS-XXXX/README.md +++ b/docs/CPS/CPS-XXXX/README.md @@ -8,8 +8,7 @@ Authors: - William Wolff - Martynas Kazlauskas - Daniele Ricci - - Sefa Irken - + - Hızır Sefa İrken Proposed Solutions: [] Discussions: diff --git a/docs/messages/client/subscribe.md b/docs/messages/client/subscribe.md deleted file mode 100644 index 19ac518..0000000 --- a/docs/messages/client/subscribe.md +++ /dev/null @@ -1,228 +0,0 @@ -# Client Subscribe Message - -Sent by client to server right after establishing a connection. New clients must authenticate themselves when subscribing to any blockchain events via a signature. - -## Schema - -```json -{ - "$schema": "http://json-schema.org/draft-07/schema#", - "type": "object", - "properties": { - "type": { - "type": "string", - "enum": [ - "subscribe" - ] - }, - "topics": { - "type": "array", - "items": { - "type": "object", - "properties": { - "blockchain": { - "type": "object", - "properties": { - "name": { - "type": "string" - }, - "network": { - "type": "string" - } - }, - "required": [ - "name", - "network" - ] - }, - "publicKey": { - "type": "string", - "pattern": "^[A-Za-z0-9+/=]*$" - }, - "signature": { - "type": "string", - "pattern": "^[A-Za-z0-9+/=]*$" - }, - /* */ - "cardano": { - "type": "object", - "properties": { - "credentials": { - "type": "object", - "properties": { - "payment": { - "type": "array", - "items": { - "type": "string", - "pattern": "^[A-Za-z0-9+/=]*$" - } - }, - "stake": { - "type": "array", - "items": { - "type": "string", - "pattern": "^[A-Za-z0-9+/=]*$" - } - } - }, - "required": [ - "payment", - "stake" - ] - }, - "config": { - "type": "object", - "properties": { - "resolveTxInput": { - "type": "boolean", - "default": false - }, - "assetMetadata": { - "type": "boolean", - "default": false - } - } - } - }, - "required": [ - "credentials", "config" - ] - } - }, - "required": [ - "blockchain", - "publicKey", - "signature" - ] - } - }, - "timestamp": { - "type": "string", - "format": "date-time" - } - }, - "required": [ - "type", - "topics", - "timestamp" - ], - "additionalProperties": false -} -``` - -The `signature` field is generated by signing a SHA256 HMAC hashed string. The prehash string is constructed by concatenating the subscribed `blockchains` + `timestamp`. - -> [!NOTE] -> For subscribing to multiple blockchains, the prehash string must be generated for each as `.`, for example: -> `cardano.mainnet+2024-06-27T12:34:56Z`. - -Apply a SHA256 HMAC using either the payment signing key or staking signing key, and then base64-encode it as final payload within the initial message. Pass the output of this SHA256 HMAC to the signature field. - -The signature verifies ownership of the private key, whose corresponding public key is used to filter relevant block transactions. - -## Subscribe Example - -```json -{ - "type": "subscribe", - "topics": [ - { - "blockchain": { "name": "cardano", "network": "mainnet" }, - "publicKey": "publicKey_example", - "signature": "OGNiOWIyNGVjOTMxZmY3N2MzYjQxOTY3OWE0YTcwMzczZmVkZmIxNDZmMDE0ODk0Nzg4YjUxMmIzMjE4MDdiYw==", // base64, SHA256 HMAC with your signing key - "cardano": { - "credentials": { - "payment": ["script...", "addr_vkh..."], // this field follows CIP-0005 - "stake": ["script...", "addr_vkh..."] // this field follows CIP-0005 - }, - "config": { - "resolveTxInput": true, - "assetMetadata": true - } - } - } - ], - "timestamp": "2024-06-27T12:34:56Z" -} -``` - -## Subscription Object - -The subscription object is blockchain specific, because there are different networks, credentials or other fields required. -Chain specific fields are grouped under a specific field named after the given chain. -In the case of Cardano blockchain, all the relevant fields are placed under a field named `cardano`. -Below we list the currently supported subscriptions: - -### Cardano - -This is the format for a Cardano specific object in the `topics` array. A client can subscribe to multiple accounts at once by providing multiple objects or one by one at any given time. - -```json -{ - "blockchain": { - "name": "cardano", - "network": { - "type": "string", - "enum": ["mainnet", "preprod", "preview"] - } - }, - "publicKey": { - "type": "string" - }, - "signature": { - "type": "string", - "pattern": "^[A-Za-z0-9+/=]*$" - }, - "cardano": { - "type": "object", - "properties": { - "credentials": { - "type": "object", - "properties": { - "payment": { - "type": "array", - "items": { - "type": "string", - "pattern": "^[A-Za-z0-9+/=]*$" - } - }, - "stake": { - "type": "array", - "items": { - "type": "string", - "pattern": "^[A-Za-z0-9+/=]*$" - } - } - }, - "required": [ - "payment", "stake" - ] - }, - "config": { - "type": "object", - "properties": { - "resolveTxInput": { - "type": "boolean", - "default": false - }, - "assetMetadata": { - "type": "boolean", - "default": false - } - } - } - }, - "required": ["credentials", "config"] - }, - "required": ["blockchain", "publicKey", "signature", "cardano"] -} -``` - -### Other blockchains - -Any other blockchain schema should follow the same structure as the above Cardano example by applying these: - -* Group blockchain specific properties under one property named after the blockchain (see `cardano` field) -* Keep chain specific authentication fields under `credentials` property -* Keep chain specific configuration fields under `config` property -* Define a JSON Schema in the documentation diff --git a/docs/messages/index.md b/docs/messages/index.md deleted file mode 100644 index 068d7ca..0000000 --- a/docs/messages/index.md +++ /dev/null @@ -1,165 +0,0 @@ -# Message Types - -This index categorizes supported message types in our event-driven communication protocol, distinguishing between client-sent and server-sent messages. It serves as the foundation for understanding the protocol's structure and functionality. - -> [!NOTE] -> We use the schema to define messages. - -## Client Messages - -- [`heartbeat`](./client/heartbeat.md): Connection management/ keep alive -- [`subscribe`](./client/subscribe.md): Define topics of interest & authentication -- [`unsubscribe`](./client/unsubscribe.md): Graceful disconnect from server - -## Server Messages - -Server-sent messages are sent either **periodically** or **by trigger** due to newly available, relevant data for a specific client. - -### Message Partials - -Different blockchains have different **periodic** events that are not related to specific transactions, but are required for either transaction construction or necessary to derive the correct wallet state. These events get represented by additional top level message keys (message partial) correlated to a chain [`point`](#event-sequencing-and-synchronization). - -We differentiate between [Ledger](#ledger-events) and [Network](#network-events) events. - -#### Ledger events - -Ledger events are specific occurrences that directly affect the state of a blockchain's ledger. These events typically involve changes to account balances, transaction histories, and other financial records. - -##### Examples - -- new transactions -- smart contract interactions -- token transfers -- stake delegation changes - -#### Network events - -Network events are occurrences that impact the overall operation and configuration of the blockchain network but do not directly alter the ledger's state. These events often involve changes to the network's consensus mechanism, protocol parameters, or infrastructure. - -##### Examples - -- Difficulty Adjustments (Bitcoin): Changes to the mining difficulty to maintain a consistent block production rate. -- Halving Events (Bitcoin): Reductions in the block reward given to miners, which occur approximately every four years. -- Epoch Transitions (Cardano): Periodic changes in the network's epoch, which can involve updates to protocol parameters and staking rewards. -- Era/ Network Transitions (Cardano): Cardano has transitioned through different [eras](https://roadmap.cardano.org/en/) which add/ remove or change certain network properties -- Network Upgrades (Ethereum): Implementation of new protocol features or improvements, such as the transition to Proof-of-Stake or gas price adjustments. -- Slot Leader Schedule Updates (Solana): Changes to the schedule of validators responsible for producing blocks. - -## Server Message Partials - -- [`genesis`](./server/genesis.md): If applicable for the subscribed blockchain, it serves static data used as part of the network bootstrap -- [`transaction`](./server/transaction.md): A new transaction -- [`tip`](./server/tip.md): A new block was appended - -### Cardano - -- [`protocol parameters`](./server/cardano/protocol-parameters.md): Epoch based message for updated protocol parameters -- [`era summary`](./server/cardano/era-summary.md): Era based message for updated slot length - -## Multichain Support - -Each server-sent message incorporates a unique chain-specific identifier. This identifier serves to precisely indicate the blockchain and network to which the message pertains. - -```json -"chain": { - "blockchain": "cardano", - "network": "mainnet", - /* ... */ -} -``` - -> [!NOTE] -> The fields of `chain` may slightly vary because some networks require additional metadata. -> For example, Ethereum-compatible chains, the `chain_id` field is particularly important as it corresponds to the [EIP-155](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-155.md) chain ID.
- -A client may define multiple [`subscription`](./messages/client/subscribe.md) objects (topics of interest) to receive events from different blockchains. -This is done via subscriptions by proving authenticity depending on the blockchain. - -A client has the capability to initiate multiple subscriptions either in a single message at once or sequentially over time via a unified WebSocket connection. -When multiple subscriptions are grouped within a single message, any authentication failure in one subscription results in the intentional failure of all subscriptions included in that message. - -Alternatively, when subscriptions are executed sequentially, each subscription operation remains independent and does not impact the others. - -Furthermore, a client may opt for a dedicated WebSocket connection per subscription, restricting each WebSocket connection to handle only one subscription at a time. - -## Event Sequencing and Synchronization - -As described on a high-level before, each event server-sent message includes a `point` object. This object is crucial for tracking a client's progress in synchronizing with the current tip of a **specific** blockchain. -This type may vary depending on the blockchain, the following shows a few examples: - -## Schema - -```json -"point": { - "type": "object", - "oneOf": [ - { - "properties": { - "height": { - "type": "integer" - }, - "hash": { - "type": "string" - } - }, - "required": ["height", "hash"], - "additionalProperties": false - }, - { - "properties": { - "slot": { - "type": "integer" - }, - "hash": { - "type": "string" - } - }, - "required": ["slot", "hash"], - "additionalProperties": false - } - ] -} -``` - -#### Cardano Point - -```json -"point": { - "slot": 127838345, // absolute slot number - "hash": "9f06ab6ecce25041b3a55db6c1abe225a65d46f7ff9237a8287a78925b86d10e" // block hash -} -``` - -#### Bitcoin Point - -```json -"point": { - "height": 849561, // block height - "hash": "00000000000000000000e51f4863683fdcf1ab41eb1eb0d0ab53ee1e69df11bb" // block hash -} -``` - -#### Ethereum Point - -```json -"point": { - "height": 20175853, // block height - "hash": "0xc5ae7e8e0107fe45f7f31ddb2c0456ec92547ce288eb606ddda4aec738e3c8ec" // block hash -} -``` - -### `Point` Uses - -The `point` object serves several important functions: - -1. **Synchronization**: - - It allows clients to keep track of their current position in the blockchain, enabling them to request only new events since their last known point. More details can be found in [02-Synchronization](../02-Synchronization.md). - -2. **Consistency**: - - In case of chain reorganizations, clients can use the blockchain-specific point to detect and handle any changes in the blockchain history by handling `rollback` events properly. - -3. **Resumability**: - - If a client disconnects and reconnects, it can provide its last known point to resume event processing from where it left off for a particular blockchain.