Number | Title | Category | Status |
---|---|---|---|
17 |
API Version 6's major rewrite |
Server |
accepted |
Ogmios was created circa 2018, prior to the Shelley era, as a way to bridge Cardano to developers. One of Ogmios' main goal has been to maximise the developer experience and ease of interaction with the Cardano blockchain.
True to its core goal, its API hasn't changed much in the course of the parse years. However, having crossed all the Cardano eras, it has accumulated few technical debts. Ogmios often integrate eras and features ahead of time and often, before they properly settle in the ledger and the ecosystem. Hence, some names chosen initially now sound a little awkward (e.g. extra data hash
instead of script integrity hash
or, metadata
vs auxiliary data
).
In addition, the Byron era which was integrated first and well before the Shelley era came out still present some oddities. While most eras that followed Shelley were built upon Shelley and shared an extensive part of its source code, Byron has been seggregated out and is different. At the time, it wasn't clear that all eras following Shelley would be similar so we did not bother making Byron and Shelley too similar. Now however, Byron feels very much disconnected from other eras.
More, there were also regrettable decisions made from the start regarding choices of serialization in the API. In particular, the use of singleton object instead of discriminated unions made it hard down the line for client to parse data. This particularly the case for recursive structures such as timelocks scripts from the Allegra/Mary eras.
Finally, over time, Ogmios has tried to abstract away the complexity of the hard-fork combinator. While all mini-protocols in Cardano are era-dependent, Ogmios tries as much as possible to be era-independant. Yet, Ogmios would still return different responses based on era sometimes which create a strange duality between the input interface and the server's output. This era separation on the client side also adds complexity as it forces client to consider era differences even when it isn't needed (e.g. accessing a block header hash that is present in ALL eras shouldn't require a complex dance).
The outer API of Ogmios will be rewritten, aiming to solve the various quirks that have been identified since it was created. In particular, we will follow a few principles while doing so:
-
The API should maximise consistency.
If two fields have same names, they should refer to the same entity. And vice-versa, two entities that are semantically equivalent should have the same name. Similarly, if a particular approach is used for structuring a certain object, it is reasonable to expect any object that are structurally equivalent to also be serialised in the same way. -
The API should be as unambiguous and as self-explanatory as possible.
This means, for example, avoiding acronyms such astx
,vk
,blk
and favor their complete alternativestransaction
,verificationKey
,block
. The only two exceptions in the API to this are:- id (→ identifier);
- vrf (→ verifiable random function);
because they are more often seen in these forms than in their full form. The new API shall also try to avoid ambiguity arising from context. If something is called hash
or id
, it should be nested in a parent object that removes any ambiguity. For example: { "transaction": { "id": "..." } }
, { "header": { "hash": "..." } }
.
- The API should favor composability.
By composability, we mean composability within the API itself. Thus, if a method of the API expects a an objectX
, and some other method returns an objectX
, it should be straightforward to lift the result as an input. On a concrete example, this means avoiding coumpound keys such astransactionId
orheaderHash
and prefer them the form of a partial object (e.g.{ "transaction": { "id": "..." } }
). This also means that whenever possible, the API should try to align the representation of objects that share similar fields. This is particularly the case for blocks and transactions which are now represented in one common format, with optional fields.
-
The API is easier to parse and comprehend overall.
-
The JSON-schema definition has been split into two. One file still lives in this repository, while the other has been moved to CardanoSolutions/cardanonical. The latter contains definitions of ALL data-types present in the Cardano blockchain for reuse across multiple services of the ecosystem.
-
Every single client application is now broken :) ... but here is a (hopefully exhaustive) migration guide:
Tip
There are still many test vectors available for every element of the Ogmios API. Use them!
JSON-WSP has been ditched and replaced by JSON-RPC 2.0 with which Ogmios is now fully compatible. In particular, this means that request and response payloads are a bit more lightweight.
Old | New |
---|---|
{
"type": "jsonwsp/request",
"version": "1.0",
"servicename": "ogmios",
"methodname": "query",
"args": { "query": "genesisConfig" },
"mirror": { "id": "foo" }
} |
{
"jsonrpc": "2.0",
"method": "queryNetwork/genesisConfiguration",
"params": { "era": "shelley" },
"id": "foo"
} |
Note
Ogmios' implementation of JSON-RPC 2.0 is slightly more flexible as the specification w.r.t to the id
field. While the specification indicates that this field should be a string
, Ogmios will still accept anything as an id
field (string
, number
, object
, etc..). So it is essentially a drop-in replacement for mirror
.
Similarly, Ogmios will always return the method
as part of the response field as it often help with context and parsing.
All methods names have been adjusted as well. Here's a translation table (beware the casing, methods are now lowerCamelCase
):
Old | New |
---|---|
RequestNext |
nextBlock |
FindIntersect |
findIntersection |
--- | --- |
SubmitTx |
submitTransaction |
EvaluateTx |
evaluateTransaction |
--- | --- |
Acquire |
acquireLedgerState |
Query |
queryLedgerState/* queryNetwork/* |
Release |
releaseLedgerState |
--- | --- |
AwaitAcquire |
acquireMempool |
NextTx |
nextTransaction |
HasTx |
hasTransaction |
SizeAndCapacity |
sizeOfMempool |
ReleaseMempool |
releaseMempool |
- Similarly, many ledger state queries have been renamed. Here's a recap table:
Old | New |
---|---|
currentEpoch |
epoch |
currentProtocolParameters |
protocolParameters |
delegationsAndRewards |
rewardAccountSummaries |
eraStart |
eraStart |
eraSummaries |
eraSummaries |
ledgerTip |
tip |
nonMyopicMemberRewards |
projectedRewards |
proposedProtocolParameters |
proposedProtocolParameters |
rewardsProvenance |
N/A |
rewardsProvenance' |
rewardsProvenance |
stakeDistribution |
liveStakeDistribution |
utxo |
utxo |
Also, some queries have been moved under queryNetwork
and are always available, in any era:
Old | New |
---|---|
blockHeight |
blockHeight |
chainTip |
tip |
genesisConfig |
genesisConfiguration |
systemStart |
startTime |
Warning
The queryNetwork/genesis
local-state-query now expects one era as argument (either 'byron', 'shelley' or 'alonzo') to retrieve the corresponding genesis configuration.
Query responses from the local-state-query protocol are now properly linked to their parent query. In prior version of ogmios, the response will simply have query
as a method, not giving much information about which query was it a response for.
Request | Response |
---|---|
{
"jsonrpc": "2.0",
"method": "queryLedgerState/tip",
} |
{
"jsonrpc": "2.0",
"method": "queryLedgerState/tip",
"result": {
"slot": 1234,
"id": "1234567890abcdef"
}
} |
All protocol errors are now identified using unique error codes. Beyond the codes specific to JSON-RPC 2.0 (i.e. -32700, -32600, -32601, -32602, -32603), Ogmios returns errors in different ranges depending on the mini-protocol. Hence the range 1000-1999
is reserved to the chain synchronization, 2000-2999
for the ledger/network state queries, 3000-3999
for the transaction submission / evaluation and 4000-4999 for the mempool monitoring.
You don't have to worry too much about those ranges as it sufficient to know that each error has a unique code. The code can thus be used as a discriminant for parsing the error details, if any. With this, all the submission errors have been greatly reworked to provide extensive descriptions as 'message' and useful details when possible. In addition, Ogmios no longer returns a list of ledger errors. The list turned out to be often quite confusing as some error would trigger more error in cascade. To cope with this, Ogmios now has a built-in heuristic to figure out which error came first and is the most relevant to tackle next and thus, will only show one error at a time.
The block model has also been reworked and merged together into one comprehensive block type. Fields have been renamed, some have been nested and other unnested. Here's a recap:
Old | New |
---|---|
hash |
N/A (removed) |
header.blockSize |
size |
header.blockHeight |
height |
header.slot |
slot |
header.blockHash |
N/A (removed) |
headerHash |
id |
header.previousHash |
ancestor |
header.opCert |
issuer.operationalCertificate |
header.issuerVk |
issuer.verificationKey |
header.issuerVrf |
issuer.vrfVerificationKey |
body |
transactions |
header.signature |
N/A (removed) |
Byron blocks are also now less "weird" than the rest of the blocks. So few changes concern only (old) Byron blocks to align them with the new model:
Old | New |
---|---|
header.genesisKey |
issuer.verificationKey |
header.prevHash |
ancestor |
header.signature.signature |
N/A (removed) |
header.signature.dlgCertificate.delegateVk |
delegate.verificationKey |
header.protocolVersion |
protocol.version |
header.protocolMagicId |
protocol.magic |
header.softwareVersion |
protocol.software |
header.proof |
N/A (removed) |
body.txPayload |
transactions |
body.dlgPayload |
operationalCertificates |
body.updatePayload |
governanceAction |
The transaction model has been greatly reworked. The main changes are:
-
The fields previously nested under
body
andwitnesses
have been flattened out and are now part of the top level object (e.g.inputs
are no longer nested underbody
). -
Metadata now only contains user-defined labelled metadata instead of also containing extra scripts. Extra scripts have been moved to the
scripts
field and merged with witness scripts. The hash digests of those metadata-scripts are now listed asrequiredExtraScripts
. The naming of metadata is now also a bit less awkward asbody → blob
for accessing user-defined metadata is now simplylabels
. -
Few fields have been renamed, here's a recap:
Old New body.certificates
certificates
body.collateralReturn
collateralReturn
body.collaterals
collaterals
body.fee
fee
body.inputs
inputs
body.mint
mint
body.network
network
body.outputs
outputs
body.references
references
body.requiredExtraSignatures
requiredExtraSignatories
body.scriptIntegrityHash
scriptIntegrityHash
body.totalCollateral
totalCollateral
body.update
proposals
body.validityInterval
validityInterval
body.withdrawals
withdrawals
body
N/A id
id
inputSource
spends
metadata
metadata
raw
cbor
witness.bootstrap
signatories
1witness.datums
datums
witness.redeemers
redeemers
2witness.scripts
scripts
witness.signatures
signatories
1witness
N/A N/A requiredExtraScripts
N/A votes
Tip
[1] The bootstrap
and signatures
fields have been merged under a single signatories
since they were both Ed25519 signatures of the transaction body. However, a bootstrap
signature contains some extra field needed to verify the signature. Hence the chainCode
and addressAttributes
are only present on bootstrap signatures (when spending from a Byron/Bootstrap address)."
Warning
[2] The format of the redeemers field has changed from an map to an array. Map keys have been turned into object, nested under a field validator
.
Transaction's output references have been aligned to follow the new context-nesting strategy on Ogmios and avoid acronyms. Thus, there's no txId
field anymore but instead a transaction
and nested id
fields. Similarly, the index is now scoped under output
.
Old | New |
---|---|
txId |
transaction.id |
index |
output.index |
Starting from version 6, Ogmios has only one protocol parameter data model. Era-dependent models from Shelley, Alonzo and Babbage have been merged into one. Names have been made more uniform across eras. Consequently, some parameters have been made optional as they may only be present after a certain era. And some only exist in old eras. Yet overall, if something is meant to exist across all eras it will be present and has one single name.
Old | New | Presence |
---|---|---|
decentralizationParameter |
federatedBlockProductionRatio |
Up to Babbage |
desiredNumberOfPools |
desiredNumberOfStakePools |
All eras |
extraEntropy |
extraEntropy |
Up to Babbage |
maxBlockBodySize |
maxBlockBodySize |
All eras |
maxBlockHeaderSize |
maxBlockHeaderSize |
All eras |
maxTxSize |
maxTransactionSize |
All eras |
minFeeCoefficient |
minFeeCoefficient |
All eras |
minFeeConstant |
minFeeConstant |
All eras |
minPoolCost |
minStakePoolCost |
All eras |
minUtxoValue |
minUtxoDepositConstant |
All eras |
monetaryExpansion |
monetaryExpansion |
All eras |
poolDeposit |
stakePoolDeposit |
All eras |
poolInfluence |
stakePoolPledgeInfluence |
All eras |
poolRetirementEpochBound |
stakePoolRetirementEpochBound |
All eras |
protocolVersion |
version |
All eras |
stakeKeyDeposit |
stakeCredentialDeposit |
All eras |
treasuryExpansion |
treasuryExpansion |
All eras |
Old | New | Presence |
---|---|---|
coinsPerUtxoWord |
minUtxoDepositCoefficient |
All eras |
collateralPercentage |
collateralPercentage |
Alonzo onwards |
costModels |
plutusCostModels |
Alonzo onwards |
maxCollateralInputs |
maxCollateralInputs |
Alonzo onwards |
maxExecutionUnitsPerBlock |
maxExecutionUnitsPerBlock |
Alonzo onwards |
maxExecutionUnitsPerTransaction |
maxExecutionUnitsPerTransaction |
Alonzo onwards |
maxValueSize |
maxValueSize |
Alonzo onwards |
prices |
scriptExecutionPrices |
Alonzo onwards |
Old | New | Presence |
---|---|---|
coinsPerUtxoByte |
minUtxoDepositCoefficient |
All eras |
The stake pools data model has been unified across the entire API. In particular, queries that used to return only a list of stake pool identifiers will now return a more complete list of stake pool objects, with the registered pool parameters. Parameters have therefore been lifted up to the stake pool model. Some have been slightly renamed too. So here's a translation table of the new 'StakePool' object:
Old | New |
---|---|
cost |
cost |
id |
id |
margin |
margin |
metadata |
metadata |
owners |
owners |
pledge |
pledge |
rewardAccount |
rewardAccount |
vrf |
vrfVerificationKeyHash |
Great rework of the representation of phase-1 monetary scripts (a.k.a native script). They are now easier to parse thanks to a type discriminant clause
at the fields level which indicates how the overall object should be interpret; rather than being at the key level. Below is shown how the old model translates to the new.
In addition, scripts (including Plutus scripts) are now wrapped in an object with three properties: language
, json
and cbor
. The language
is used as a discriminant for the whole script. The field json
is only present for native scripts and contains the format detailed below. The field cbor
contains the serialized representation of that script.
Old | New |
---|---|
"<credential-digest>" |
{
"clause": "signature",
"from": "<credential-digest>",
} |
Old | New |
---|---|
{
"all": [ "<native-script>" ]
} |
{
"clause": "all",
"from": [ "<native-script>" ]
} |
Old | New |
---|---|
{
"any": [ "<native-script>" ]
} |
{
"clause": "any",
"from": [ "<native-script>" ]
} |
Old | New |
---|---|
{
"NOf": {
"<integer>": [ "<native-script>" ]
}
} |
{
"clause": "some",
"atLeast": "<integer>",
"from": [ "<native-script>" ]
} |
Old | New |
---|---|
{
"expiresAt": "<slot>"
} |
{
"clause": "before",
"slot": "<slot>"
} |
Old | New |
---|---|
{
"startsAt": "<slot>"
} |
{
"clause": "after",
"slot": "<slot>"
} |
A discriminant value field (type
) has been introduced to all certificate to allow parsing them with more ease. Consequently, certificates are no longer prefixed with a discriminant key. For fields have also been renamed along the way.
Old | New |
---|---|
{
"stakeDelegation": {
"delegate": "<credential-digest>",
"delegatee": "<stake-pool-id>"
}
} |
{
"type": "stakeDelegation",
"credential": "<credential-digest>",
"stakePool": {
"id": "<stake-pool-id>"
}
} |
Old | New |
---|---|
{
"stakeKeyRegistration": "<credential-digest>"
} |
{
"type": "stakeCredentialRegistration",
"credential": "<credential-digest>",
} |
Old | New |
---|---|
{
"stakeKeyDeregistration": "<credential-digest>"
} |
{
"type": "stakeCredentialDeregistration",
"credential": "<credential-digest>",
} |
Old | New |
---|---|
{
"poolRegistration": "<stake-pool-parameters>"
} |
{
"type": "stakePoolRegistration",
"stakePool": {
"id": "<stake-pool-id>",
"parameters": "<stake-pool-parameters>"
}
} |
Old | New |
---|---|
{
"poolRetirement": {
"poolId": "<stake-pool-id>",
"retirementEpoch": "<epoch>"
}
} |
{
"type": "stakePoolRetirement",
"stakePool": {
"id": "<stake-pool-id>",
"retirementEpoch": "<epoch>"
}
} |
Old | New |
---|---|
{
"genesisDelegation": {
"delegateKeyHash": "<credential-digest>",
"verificationKeyHash": "<credential-digest>",
"vrfVerificationKeyHash": "<vrf-digest>"
}
} |
{
"type": "genesisDelegation",
"issuer": {
"verificationKeyHash": "<credential-digest>",
"vrfVerificationKeyHash": "<vrf-digest>"
},
"delegate": {
"verificationKeyHash": "<credential-digest>"
}
} |
Treasury transfers have been converted into governance actions, so you'll now find them in the proposals
field of transactions.
The representation of Value
has been changed to be more compact, more extensible and clearer. Values are now encoded as nested objects, where keys are respectively asset's policy id and asset name. Leaves are plain integers. The special case of Ada is encoded as a special policy id ada
and asset name lovelace
. This behavior is consistently applied to any amount that refers to a lovelace quantity. Transaction fees for example are now encoded as: { "ada": { "lovelace": 1234 } }
.
The representation of transaction metadata has been both simplified and made more user-friendly, while remaining safe for more complex use-cases. In fact, many people in the community have grown to expect transaction metadata to be JSON objects. However, they aren't. Or more specifically, they aren't necessarily. There are actually plenty of transaction metadata on-chain that aren't representable as valid JSON. Prior to version 6, Ogmios would give a so-called detailed JSON schema representation of those metadata, by encoding the binary encoding as a JSON object. This has created a lot of confusion for rookie users not yet familiar with Cardano entrails who would be expecting a plain JSON object. Plus, the format was unpractical to parse for client down the line as it used object keys as type discriminant, leaving decoders no choice to try various encoding alternatively.
Starting from version 6, by default (see note below) Ogmios returns transaction metadata as JSON object when possible and fallback to CBOR otherwise. In fact, when metadata aren't representable as JSON object, this is probably because they are some elaborated binary encoding and users consuming them are most seemingly capable of decoding that themselves in the way they intended. Ogmios can be configured to always return the CBOR output using the --include-metadata-cbor
flag on start.
To give a concrete example:
Old | New |
---|---|
{
"14": {
"map": [
{
"k": { "string": "foo" },
"v": { "int": 42 }
},
{
"k": { "string": "bar" },
"v": { "list": [ { "int": 1 }, { "int": 2 } ] }
}
]
}
} |
{
"14": {
"cbor": "A263666F6F182A63626172820102",
"json": {
"foo": 42,
"bar": [ 1, 2 ]
},
}
}
|
When it isn't possible to represent the metadata as a plain JSON object, the json
field is simply omitted and the metadata is only provided as CBOR.
[!INFO]
The old behavior can be requested on-demand by enabling the
--metadata-detailed-schema
flag. When enabled, thejson
metadata will always be present and use the old declarative representation. This can be used in combination with the new--include-metadata-cbor
flag as well.