Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

guide: validation data refactoring #1576

Merged
5 commits merged into from
Aug 13, 2020
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions roadmap/implementers-guide/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@
- [Validators](runtime-api/validators.md)
- [Validator Groups](runtime-api/validator-groups.md)
- [Availability Cores](runtime-api/availability-cores.md)
- [Global Validation Data](runtime-api/global-validation-data.md)
- [Local Validation Data](runtime-api/local-validation-data.md)
- [Persisted Validation Data](runtime-api/persisted-validation-data.md)
- [Full Validation Data](runtime-api/full-validation-data.md)
- [Session Index](runtime-api/session-index.md)
- [Validation Code](runtime-api/validation-code.md)
- [Candidate Pending Availability](runtime-api/candidate-pending-availability.md)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ On `ActiveLeavesUpdate`:
* Otherwise, for each `activated` head in the update:
* Determine if the para is scheduled or is next up on any occupied core by fetching the `availability_cores` Runtime API.
* Determine an occupied core assumption to make about the para. The simplest thing to do is to always assume that if the para occupies a core, that the candidate will become available. Further on, this might be determined based on bitfields seen or validator requests.
* Use the Runtime API subsystem to fetch the global validation data and local validation data.
* Use the Runtime API subsystem to fetch the full validation data.
* Construct validation function params based on validation data.
* Invoke the `collation_producer`.
* Construct a `CommittedCandidateReceipt` using the outputs of the `collation_producer` and signing with the `key`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,7 @@ Parachain candidates are validated against their validation function: A piece of
Upon receiving a validation request, the first thing the candidate validation subsystem should do is make sure it has all the necessary parameters to the validation function. These are:
* The Validation Function itself.
* The [`CandidateDescriptor`](../../types/candidate.md#candidatedescriptor).
* The [`LocalValidationData`](../../types/candidate.md#localvalidationdata).
* The [`GlobalValidationSchedule](../../types/candidate.md#globalvalidationschedule).
* The [`ValidationData`](../../types/candidate.md#validationdata).
* The [`PoV`](../../types/availability.md#proofofvalidity).

### Determining Parameters
Expand Down
2 changes: 1 addition & 1 deletion roadmap/implementers-guide/src/node/utility/provisioner.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ To determine availability:
- If the bitfields indicate availability and there is a scheduled `next_up_on_available`, then we can make an `OccupiedCoreAssumption::Included`.
- If the bitfields do not indicate availability, and there is a scheduled `next_up_on_time_out`, and `occupied_core.time_out_at == block_number_under_production`, then we can make an `OccupiedCoreAssumption::TimedOut`.
- If we did not make an `OccupiedCoreAssumption`, then continue on to the next core.
- Now compute the core's `validation_data_hash`: get the `LocalValidationData` from the runtime, given the known `ParaId` and `OccupiedCoreAssumption`; this can be combined with a cached `GlobalValidationData` to compute the hash.
- Now compute the core's `validation_data_hash`: get the `PersistedValidationData` from the runtime, given the known `ParaId` and `OccupiedCoreAssumption`;
- Find an appropriate candidate for the core.
- There are two constraints: `backed_candidate.candidate.descriptor.para_id == scheduled_core.para_id && candidate.candidate.descriptor.validation_data_hash == computed_validation_data_hash`.
- In the event that more than one candidate meets the constraints, selection between the candidates is arbitrary. However, not more than one candidate can be selected per core.
Expand Down
16 changes: 16 additions & 0 deletions roadmap/implementers-guide/src/runtime-api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,19 @@ The next sections will contain information on specific runtime APIs. The format
/// best for the implementation to return an error indicating the failure mode.
fn some_runtime_api(at: Block, arg1: Type1, arg2: Type2, ...) -> ReturnValue;
```

Certain runtime APIs concerning the state of a para require the caller to provide an `OccupiedCoreAssumption`. This indicates how the result of the runtime API should be computed if there is a candidate from the para occupying an availability core in the [Inclusion Module](../runtime/inclusion.md).

The choices of assumption are whether the candidate occupying that core should be assumed to have been made available and included or timed out and discarded, along with a third option to assert that the core was not occupied. This choice affects everything from the parent head-data, the validation code, and the state of message-queues. Typically, users will take the assumption that either the core was free or that the occupying candidate was included, as timeouts are expected only in adversarial circumstances and even so, only in a small minority of blocks directly following validator set rotations.

```rust
/// An assumption being made about the state of an occupied core.
enum OccupiedCoreAssumption {
/// The candidate occupying the core was made available and included to free the core.
Included,
/// The candidate occupying the core timed out and freed the core without advancing the para.
TimedOut,
/// The core was not occupied to begin with.
Free,
}
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Full Validation Data

Yields the full [`ValidationData`](../types/candidate.md#validationdata) at the state of a given block.

```rust
fn full_validation_data(at: Block, ParaId, OccupiedCoreAssumption) -> Option<ValidationData>;
```

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Persisted Validation Data

Yields the [`PersistedValidationData`](../types/candidate.md#persistedvalidationdata) for the given [`ParaId`](../types/candidate.md#paraid) along with an assumption that should be used if the para currently occupies a core:

```rust
/// Returns the persisted validation data for the given para and occupied core assumption.
///
/// Returns `None` if either the para is not registered or the assumption is `Freed`
/// and the para already occupies a core.
fn persisted_validation_data(at: Block, ParaId, OccupiedCoreAssumption) -> Option<PersistedValidationData>;
```
3 changes: 0 additions & 3 deletions roadmap/implementers-guide/src/runtime/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,6 @@ fn update_configuration(f: impl FnOnce(&mut HostConfiguration)) {
*pending = Some(x);
})
}

/// Get the GlobalValidationData, assuming the context is the parent block.
fn global_validation_data() -> GlobalValidationData;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

local_validation_data was queried from paras.md. global_validation_data was declared here, in configuration.md.

In the presented changes, persisted_validation_data is queried from paras.md but where do I query full_validation_data to implement the runtime API call with the same name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The runtime API will need to draw this information out from many different modules. It wasn't obvious where its home should be.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I actually was triggered by that: it wasn't for me where would be the home and then I thought that the chances are it might be not obvious for the implementer either.

but that's fine, we can figure out this later

```

## Entry-points
Expand Down
4 changes: 2 additions & 2 deletions roadmap/implementers-guide/src/runtime/inclusion.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,8 @@ All failed checks should lead to an unrecoverable error making the block invalid
1. check that each candidate corresponds to a scheduled core and that they are ordered in the same order the cores appear in assignments in `scheduled`.
1. check that `scheduled` is sorted ascending by `CoreIndex`, without duplicates.
1. check that there is no candidate pending availability for any scheduled `ParaId`.
1. check that each candidate's `validation_data_hash` corresponds to a `(LocalValidationData, GlobalValidationData)` computed from the current state.
> NOTE: With contextual execution in place, local and global validation data will be obtained as of the state of the context block. However, only the state of the current block can be used for such a query.
1. check that each candidate's `validation_data_hash` corresponds to a `PersistedValidationData` computed from the current state.
rphmeier marked this conversation as resolved.
Show resolved Hide resolved
> NOTE: With contextual execution in place, validation data will be obtained as of the state of the context block. However, only the state of the current block can be used for such a query.
1. If the core assignment includes a specific collator, ensure the backed candidate is issued by that collator.
1. Ensure that any code upgrade scheduled by the candidate does not happen within `config.validation_upgrade_frequency` of `Paras::last_code_upgrade(para_id, true)`, if any, comparing against the value of `Paras::FutureCodeUpgrades` for the given para ID.
1. Check the collator's signature on the candidate data.
Expand Down
2 changes: 1 addition & 1 deletion roadmap/implementers-guide/src/runtime/paras.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ OutgoingParas: Vec<ParaId>;
* `is_parathread(ParaId) -> bool`: Returns true if the para ID references any live parathread.

* `last_code_upgrade(id: ParaId, include_future: bool) -> Option<BlockNumber>`: The block number of the last scheduled upgrade of the requested para. Includes future upgrades if the flag is set. This is the `expected_at` number, not the `activated_at` number.
* `local_validation_data(id: ParaId) -> Option<LocalValidationData>`: Get the LocalValidationData of the given para, assuming the context is the parent block. Returns `None` if the para is not known.
* `persisted_validation_data(id: ParaId) -> Option<PersistedValidationData>`: Get the PersistedValidationData of the given para, assuming the context is the parent block. Returns `None` if the para is not known.

## Finalization

Expand Down
19 changes: 2 additions & 17 deletions roadmap/implementers-guide/src/types/availability.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,32 +32,17 @@ Often referred to as PoV, this is a type-safe wrapper around bytes (`Vec<u8>`) w
struct PoV(Vec<u8>);
```

## Omitted Validation Data

Validation data that is often omitted from types describing candidates as it can be derived from the relay-parent of the candidate. However, with the expectation of state pruning, these are best kept available elsewhere as well.

This contains the [`GlobalValidationData`](candidate.md#globalvalidationschedule) and [`LocalValidationData`](candidate.md#localvalidationdata)

```rust
struct OmittedValidationData {
/// The global validation schedule.
global_validation: GlobalValidationData,
/// The local validation data.
local_validation: LocalValidationData,
}
```


## Available Data

This is the data we want to keep available for each [candidate](candidate.md) included in the relay chain.
This is the data we want to keep available for each [candidate](candidate.md) included in the relay chain. This is the PoV of the block, as well as the [`PersistedValidationData`](candidate.md#persistedvalidationdata)

```rust
struct AvailableData {
/// The Proof-of-Validation of the candidate.
pov: PoV,
/// The omitted validation data.
omitted_validation: OmittedValidationData,
omitted_validation: PersistedValidationData,
rphmeier marked this conversation as resolved.
Show resolved Hide resolved
}
```

Expand Down
Loading