Skip to content

Commit

Permalink
redo the Specification section
Browse files Browse the repository at this point in the history 
* The only thing that ledger CIPs require that are not already included
  in the CIP process is adding them to a table.
* There is no more STS.md, instead we link to a PDF.
* There is no mere or Serialization.md, instead we link to CIP-80.
* The heart of the new Specification section is a definition of what is
  in scope for ledger CIPs, which is agreement by every possible
  implementation.
  • Loading branch information
@JaredCorduan
JaredCorduan committed Jan 30, 2023
1 parent e815314 commit 535e029
Showing 1 changed file with 97 additions and 193 deletions.
290 changes: 97 additions & 193 deletions CIP-ledger-evolution/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,23 +6,21 @@ Comments-Summary: No comments
Comments-URI:
Status: Active
Type: Process
Created: 2022-12-XX
Created: 2023-01-XX
License: CC-BY-4.0
---

# Evolution of the Cardano ledger

## Abstract

This CIP proposes a process for proposing changes to the Cardano ledger.
This CIP provides guidance for future CIPs concerning the Cardano ledger.

## Motivation

The ledger is responsible for processing transactions and updating the shared state of the network.
It also processes block headers and handles the state transformation from one epoch to the next
(e.g. computing the staking rewards).
It is a very user-facing component;
adding new functionality to the Cardano network often involves adding features to the ledger.

Most of the state maintained by the ledger relates to the
[Extended UTxO accounting model](https://iohk.io/en/research/library/papers/the-extended-utxo-model/) and
Expand All @@ -31,9 +29,12 @@ the proof-of-stake consensus mechanism used in Cardano.

This CIP aims to give guidance for future CIPs related to the ledger,
making it a registered category of the CIP process[^1].
[^1]: See [CIP-1](https://github.com/cardano-foundation/CIPs/blob/cip-cps-rework/CIP-0001/README.md#categories).
While nothing new is added to the usual CIP process (except for a ledger registry),
expectations for ledger CIPs are made explicit and some background information is provided.

Many thanks Michael Peyton Jones to writing [CIP-35] and forging this path for us.
Some of the language here has been taken directly from it.
Many thanks to Arnaud Bailly and Michael Peyton Jones for all their help reviewing and providing
feedback on the first versions of this CIP.

## Background

Expand All @@ -44,72 +45,76 @@ Context for the terminology used in this document is given in [CIP-59].
### Specifications

The ledger is specified as a state transition system using a
[small step operational semantics](STS.md).
[small step operational semantics](https://github.com/input-output-hk/cardano-ledger/releases/latest/download/small-step-semantics.pdf).
We refer to this framework as the *Small Step Semantics for Cardano*, or the *STS* for short.
An understanding of the existing STS specifications for the
[existing ledger eras](https://github.com/input-output-hk/cardano-ledger#cardano-ledger)
is often required to fully understand the implications of changes to the ledger
existing ledger eras is often required to fully understand the implications of changes to the ledger
(though an understanding of the Haskell implementation is a fair substitute).

The STS framework leaves both cryptographic primitives and the serialization format abstract.
These specifications need to be complete enough to realize a full implementation of the ledger
The STS specifications need to be complete enough to realize a full implementation of the ledger
given the cryptographic primitives and serialization format.
The cryptographic primitives are described as appendices to the STS specifications,
and the serialization format is given as a
[CDDL file](https://www.rfc-editor.org/rfc/rfc8610).
There SHOULD be one STS specification per ledger era
(the Allegra ledger era is currently combined with the Mary era).
There SHOULD be one STS specification per ledger era.

From the Byron to the Babbage ledger eras, the STS frameworks were written in $\LaTeX$.
Starting in the Conway ledger era, literate Agda will be used.
During the transition from $\LaTeX$ to literate Agda, we will take advantage
of the ability to substitute $\LaTeX$ in place of Agda code when needed for expedience.
With time, the Agda specification will gloriously blur the line between specification
and reference implementation,
just as the type system upon which Agda is built blurs the line between proof and program.
With time, the Agda specification will not only be used to provide PDF specifications,
but also reference implementations.

### Ledger eras

A ledger era is a collection of features added to the ledger which are introduced at a hard fork.
The existing ledger eras, with a very simplistic description is given below.

|name|new features|
| --- | --- |
|Byron|initial UTxO ledger|
|Shelley|decentralized block production, stake delegation|
|Allegra|timelock scripts|
|Mary|multi-assets|
|Alonzo|Plutus scripts|
|Babbage|improved Plutus script contexts|
|Conway|constitutional committee replacement, SPO voting on hard forks|


### Ledger state

The ledger state is the source of truth from which the network derives meaning.
For performance reasons, we require it to be the minimal state needed
to derive this value and validate blocks.

There is currently one exception to the minimality, which will be deprecated in the days ahead,
namely the stake pool ranking information which is used by Daedalus.
The existing ledger eras, with very simplistic descriptions, are given below.

|name|new features|link|
| --- | --- | --- |
|Byron|initial UTxO ledger|[spec](https://github.com/input-output-hk/cardano-ledger/releases/latest/download/byron-ledger.pdf)|
|Shelley|decentralized block production, stake delegation|[spec](https://github.com/input-output-hk/cardano-ledger/releases/latest/download/shelley-ledger.pdf)|
|Allegra|timelock scripts|-|
|Mary|multi-assets|[spec](https://github.com/input-output-hk/cardano-ledger/releases/latest/download/mary-ledger.pdf)|
|Alonzo|Plutus scripts|[spec](https://github.com/input-output-hk/cardano-ledger/releases/latest/download/alonzo-ledger.pdf)|
|Babbage|improved Plutus script contexts|[spec](https://github.com/input-output-hk/cardano-ledger/releases/latest/download/babbage-ledger.pdf)|
|Conway|governance|[spec WIP](https://github.com/input-output-hk/formal-ledger-specifications)|

Note that there is no Allegra specification.
The Allegra era consists entirely of the addition of timelocks to the MultiSig script
introduced in the Shelley ledger era
(See figure 12 of the Mary specification).

Note that small, isolated changes can be made within a ledger era
by way of an intra-era hard fork. See [CIP-59] for more details.

#### Ledger events

Some provenance about the ledger calculations is provided by ledger events.
These events come with zero cost to a running node if not used and are not stored in the ledger state.
Sometimes these events have clear triggers (e.g. Plutus script execution)
and sometimes they provide intermediate calculations performed by the ledger rules
(e.g. the reweard calculation).
The events come with zero cost to a running node if not used and are not stored in the ledger state.
Documentation about the existing events can be found
[here](https://github.com/input-output-hk/cardano-ledger/blob/master/docs/LedgerEvents.md).

### Soft forks and Hard forks

We make the following definitions:
Since most ledger CIPs will involve backwards incompatible changes,
the following two definitions are helpful:

**Hard fork** - A *hard fork* is a change to the protocol (not restricted to the ledger)
resulting in a single new (CBOR) block being valid.
resulting in a single new block definition becoming valid.

Alternatively, a hard fork is a backwards incompatible change for both
block producers and block validators.

**Soft fork** - A *soft fork* is a change to the protocol (not restricted to the ledger)
resulting in fewer blocks being valid.
A good way to think about soft forks is the property that a node which only validates
(and does not produce blocks) does not need a software upgrade in order to continue following the block chain.
Soft forks do not seem to come up often in practice.

Alternatively, a soft fork is a backwards incompatible change for
block producers, but a backwards compatible change for block validators.

### Serialization

Expand All @@ -118,184 +123,83 @@ Transactions and blocks are serialized with
and specified with
[CDDL file](https://www.rfc-editor.org/rfc/rfc8610).

The ledger state is also currently also serialized with CBOR,
but this may change in the future.

### Plutus script context
Serialization changes to the ledger are discussed in
[CIP-80](https://github.com/cardano-foundation/CIPs/pull/372).

The ledger and Plutus scripts have a common interface, namely the script context.
See [CIP-35] for more information.
Note that CIPs relating to the script context are relevant to both the ledger and to the Plutus CIP categories.

## Specification
Note that the serialisation format of the ledger state is unspecified and left as an
implementation detail (unlike the format of blocks).

This proposal deals only with the types of change listed in
[Types of change](#types-of-change), all others are out of scope.
### The ledger-script interface

### Changes that require a CIP
The ledger and Plutus scripts have a common interface, described in [CIP-35].
CIPs relating to this inteface are relevant to both the ledger and to the Plutus CIP categories.

This proposal recommends that some of the changes listed in "Types of change" (specified below) should:

* Be proposed in a CIP.
* Go through additional process in addition to the usual CIP process.

The additional process mostly takes the form of additional information that should be present in
the CIP before it moves to particular stages. As such, it is up to the CIP Editors to enforce this.

The requirement to propose a change via a CIP is, as all CIPs are, advisory.
In exceptional circumstances or where swift action is required, we expect that changes may still
be made without following this process. In such circumstances,
a retrospective CIP SHOULD be made after the fact to record the changes and the rationale for them.
## Specification

Changes that require a CIP do not have to each be in an individual CIP, they can be included in batches or in other CIPs.
So, for example, a single CIP could propose multiple new related fields in the transaction.
### What merits a ledger CIP?

It is very likely the case that if a change does not require a soft or hard fork,
it does not require a CIP, except in the case of bugs.
The criterion for deciding if a change to the ledger merits a CIP is as follows:
changes to the ledger require going through the CIP process whenever
every implementation of the Cardano ledger needs to be standardized on the details.

Bug fixes are an exception to this criterion, they do not merit a CIP except in the case
that the fix is substantially complicated.
A "bug fix" is a change to behavior where:
- The implemented behavior does not match the specification; or
- The specified behavior is clearly wrong (in the judgment of relevant experts)

#### Changes that *do not* require a CIP

* performance improvements
* bug fixes
* ledger events
* changes to the protocol parameter values (this is a governance issue, see [CIP-1694])

### Types of change

- [Ledger features](#ledger-features)
- [Ledger-script interface](#ledger-script-interface)
- [Ledger intra-era switch](#ledger-intra-era-switch)
- [Ledger protocol](#ledger-protocol)
- [Ledger eras](#ledger-eras-1)
- [Serialization](#serialization)

#### Ledger features

This is a very broad category.
Any conceptual group of changes to the ledger semantics,
involving a single idea or a group of closely related ideas
(such as adding, modifying, or removing existing features)
is a `ledger-feature`.
This category excludes bug fixes and changes to the [ledger protocol](#ledger-protocol).
Ledger features are assumed to be included in a future ledger era.

To become active, the ledger feature must be included in an active [ledger era](#ledger-eras-1).
Serialization changes are another possible exception to the criterion.
Many serialization changes can be handled as a part of the normal development process
without the need for a CIP.
Dramatic changes to the serialization, however, may benefit from the CIP process.

##### Ledger-script interface
The ledger rules MUST be standardized in order for consensus to be maintained,
but things like the ledger events are more open to debate.

Any conceptual group of changes to the script context is a `ledger-script-interface`.
This is a shared type with Plutus, so the process in [CIP-35] MUST also be followed.
Changes to the protocol parameter values do not require a CIP since they are
a governance issue (see [CIP-1694]).

To become active, the interface change must be included in an active [ledger era](#ledger-eras-1).

#### Ledger intra-era switch

Sometimes a ledger change is so small and targeted that it can be implemented by
switching on the major protocol version (and not introducing a new ledger era).
These must be used *very* sparingly, as they can quickly lead to unintended complexity and
unclear semantics[^2].
A ledger intra-era switch does not necessarily require a CIP;
sometimes it might be more appropriate to just include it in a [ledger era](#ledger-eras-1) CIP.

When a hard fork involves only intra-era switching, a new ledger era can be avoided and we
refer to the hard fork as an intra-era hard fork.

To become active, the ledger intra-era switch must be included in an active [ledger era](#ledger-eras-1).

#### Ledger protocol

The `ledger-protocol` category corresponds closely to block header validation.
It corresponds precisely to all the functionality in the Shelley ledger specification
not captured by the `BBODY` and `TICK` transitions.
This mostly involves handling the nonces for Ouroboros, the VRF checks,
the KES certificates, and block leadership checks.

Historically, the ledger protocols have been specified alongside the ledger eras.
The `TPraos` ledger protocol was specified alongside Shelley, and
`Praos` was specified alongside Babbage.

To become active, a ledger protocol change MUST be
specified with the STS framework and MUST be included in the software update for a soft/hard fork.
If the fork coincides with a ledger era, the specifications can be combined.

#### Ledger eras

A `ledger-era` is a collection of ledger-features, bug fixes, intra-era switches, and other minor changes
grouped together with a name and activated together at a fork.
In the event of interaction between the features, details may need to be provided.

To become active, the ledger era MUST:
* specify the semantics with the [STS](STS.md) framework
* specify cryptographic details in an appendix to the STS specification
* specify the CBOR serialization schema with a CDDL file
* be included in the software update for a soft/hard fork.

### Serialization changes

See [Serialization](Serialization.md).

### The Small Step Semantics Framework for Cardano

See [STS](STS.md).

### Implementing and releasing changes

This CIP does not cover the process of implementing changes.
As usual, the CIP process covers the design phase, and it is up to the implementer to ensure that
their proposal is implemented, which may require additional work to meet the requirements of the
maintainers of the Cardano code repositories (testing, implementation quality, approach), and so on.
### Ledger Core CIP registry

Changes can be released after their CIPs have reached Active status.
Different changes will require different releases as described in "Types of release".
This CIP does not cover the process by which changes are actually incorporated into releases after having been implemented.
In particular, there is NO assumption that a feature which requires a particular release will be
included in the next such release, even after it has been implemented.
Any CIP which proposes a change to the ledger MUST be added to this table:

### Ledger Core CIP registry
| # | Title | Status |
|--- | --- | --- |
| 31 | Reference inputs | Active |
| 32 | Inline datums | Active |
| 33 | Reference scripts | Active |
| 59 | Terminology Surrounding Core Features | Active |
| 80 | Transaction Serialization Deprecation Cycle | Proposed |
| 1694 | A proposal for entering the Voltaire phase | Proposed |

Any CIP which proposes a type of change listed in "Types of change" MUST also add itself to this registry (in addition to the main registry).
### Expectations for ledger CIPs

| # | Title | Type of change | Status |
|----|-------------------|-------------------------|--------|
| 31 | Reference inputs | Ledger-script interface | Draft |
| 32 | Inline datums | Ledger-script interface | Draft |
| 33 | Reference scripts | Ledger-script interface | Draft |
* Familiarity with the existing ledger specifications is required to propose changes to the ledger.
* The CIP specifications for ledger CIPs must be sufficiently detailed for inclusion in
a formal ledger specification.
* Some changes may require getting an expert opinion.
For example, changes to the incentives model (such as changing the reward calculation)
should include a proper game theoretic analysis.

## Rationale

### Why have a public process for changes?
### There is only one implementation, why limit the scope of ledger CIPs in this way?

Cardano is continuing to move towards decentralized governance within the Voltaire phase of development.
Historically, key development and implementation decisions have been made by the core development team.
This was important in the earliest stages of the platform’s evolution.
However, this becomes less so as the platform starts to mature and is neither sustainable nor desirable in the long term.
Even though there is currently only one implementation, this provides us with a clear
definition of what is essential to the ledger.
It also provides a clear path for future implementations.

Furthermore, while many changes to Cardano are obscure or not of interest to many community members,
there is a much larger community who have a keen interest in changes to Plutus Core: dApp developers.
Hence it is especially important to have a clear way for this community to be able to propose changes and see how they are progressing.
### Why is the specification vague about the role of ledger events in the CIP process?

### Why include a CIP registry?
This decision should be left to the community as more use cases emerge.

This is just to make it easy for those considering proposing a CIP following this process to see which CIPs have already been submitted.
An alternative would be a standard title for CIPs, or perhaps some kind of CIP metadata to indicate that it follows the process in this CIP.
### Why is familiarity with the formal specifications required?

### Out of scope
It is not always clear which seemingly small details can make a large difference
to the many consumers of the ledger.
It is better that the CIP process achieve consensus on all the details than for
these decisions to be made during the implementation phase.

* Changes that cover _using_ the ledger, but not _changing_ the ledger.
* Changes to the node-to-client queries, which are used to get information from the ledger.
* Changes about how clients consume the ledger state, unless it involves changes to the ledger state.
* Unless something exceptional arises,
storing additional information in the ledger state that is not needed for block validation.

[^1]: See [CIP-1](https://github.com/cardano-foundation/CIPs/blob/cip-cps-rework/CIP-0001/README.md#categories).
[^2]: On an early testnet for the Mary era, the network managed to enter the Mary era while remaining
on major protocol version 2 (the mainnet version for the Shelley era).
The network exhibited strange behavior due to this fact, which was perplexing to understand until
we realized the mismatch. We dubbed it the Mary Shelley's Frankenstein era.

[CIP-35]: https://github.com/cardano-foundation/CIPs/tree/master/CIP-0035
[CIP-59]: https://github.com/cardano-foundation/CIPs/tree/master/CIP-0059
Expand Down

0 comments on commit 535e029

Please sign in to comment.