Session Delayed Para Changes / Actions Queue (#2406)

* initial implementation of lifecycles and upgrades

* clean up a bit

* fix doc comment

* more rigid lifecycle checks

* include paras which are transitioning, and lifecycle query

* format guide

* update api

* update guide

* explicit outgoing state, fix genesis

* handle outgoing with transitioning paras

* do not include transitioning paras in identifier

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Update roadmap/implementers-guide/src/runtime/paras.md

* Apply suggestions from code review

* Use matches macro

* Correct terms

* Apply suggestions from code review

* actions queue

* Revert "actions queue"

This reverts commit b2e9011.

* collapse onboarding state

* starting actions queue

* consolidate actions queue

* schedule para initialize result

* more actions queue for upgrade/downgrade

* clean up with fully implemented actions queue

* fix tests

* fix scheduler tests

* fix hrmp tests

* fix test

* doc fixes

* fix hrmp test w/ valid para

* Update paras.md

* fix paras registrar

* Update propose_parachain.rs

* fix merge

* Introduce "shared" module

* fix rococo build

* fix up and use shared

* guide updates

* add shared config to common tests

* add shared to test-runtime

* remove println

* fix note

Co-authored-by: Gavin Wood <gavin@parity.io>

roadmap/implementers-guide/src/runtime/README.md

@@ -15,11 +15,15 @@ There is some functionality of the relay chain relating to parachains that we al
 We will split the logic of the runtime up into these modules:
 
 * Initializer: manage initialization order of the other modules.
+* Shared: manages shared storage and configurations for other modules.
 * Configuration: manage configuration and configuration updates in a non-racy manner.
 * Paras: manage chain-head and validation code for parachains and parathreads.
 * Scheduler: manages parachain and parathread scheduling as well as validator assignments.
 * Inclusion: handles the inclusion and availability of scheduled parachains and parathreads.
 * Validity: handles secondary checks and dispute resolution for included, available parablocks.
+* Hrmp: handles horizontal messages between paras.
+* Ump: Handles upward messages from a para to the relay chain.
+* Dmp: Handles downward messages from the relay chain to the para.
 
 The [Initializer module](initializer.md) is special - it's responsible for handling the initialization logic of the other modules to ensure that the correct initialization order and related invariants are maintained. The other modules won't specify a on-initialize logic, but will instead expose a special semi-private routine that the initialization module will call. The other modules are relatively straightforward and perform the roles described above.
 

roadmap/implementers-guide/src/runtime/inclusion.md

@@ -36,17 +36,13 @@ PendingAvailabilityCommitments: map ParaId => CandidateCommitments;
 
 /// The current validators, by their parachain session keys.
 Validators: Vec<ValidatorId>;
-
-/// The current session index.
-CurrentSessionIndex: SessionIndex;
 ```
 
 ## Session Change
 
 1. Clear out all candidates pending availability.
 1. Clear out all validator bitfields.
 1. Update `Validators` with the validators from the session change notification.
-1. Update `CurrentSessionIndex` with the session index from the session change notification.
 
 ## Routines
 

roadmap/implementers-guide/src/runtime/paras.md

@@ -1,9 +1,9 @@
 # Paras Module
 
 The Paras module is responsible for storing information on parachains and parathreads. Registered
-parachains and parathreads cannot change except at session boundaries. This is primarily to ensure
-that the number and meaning of bits required for the availability bitfields does not change except at session
-boundaries.
+parachains and parathreads cannot change except at session boundaries and after at least a full
+session has passed. This is primarily to ensure that the number and meaning of bits required for the
+availability bitfields does not change except at session boundaries.
 
 It's also responsible for managing parachain validation code upgrades as well as maintaining
 availability of old parachain code and its pruning.
@@ -63,9 +63,9 @@ pub enum ParaLifecycle {
   /// Para is a Parachain.
   Parachain,
   /// Para is a Parathread which is upgrading to a Parachain.
-  UpgradingToParachain,
+  UpgradingParathread,
   /// Para is a Parachain which is downgrading to a Parathread.
-  DowngradingToParathread,
+  DowngradingParachain,
   /// Parathread is being offboarded.
   OutgoingParathread,
   /// Parachain is being offboarded.
@@ -82,7 +82,7 @@ state of the para using the `ParaLifecycle` enum.
 None                 Parathread                  Parachain
  +                        +                          +
  |                        |                          |
- |    (Session Delay)     |                          |
+ |   (2 Session Delay)    |                          |
  |                        |                          |
  +----------------------->+                          |
  |       Onboarding       |                          |
@@ -91,10 +91,10 @@ None                 Parathread                  Parachain
  |       Onboarding       |                          |
  |                        |                          |
  |                        +------------------------->+
- |                        |  UpgradingToParachain    |
+ |                        |   UpgradingParathread    |
  |                        |                          |
  |                        +<-------------------------+
- |                        |  DowngradingToParathread |
+ |                        |   DowngradingParachain   |
  |                        |                          |
  |<-----------------------+                          |
  |   OutgoingParathread   |                          |
@@ -137,38 +137,31 @@ PastCodePruning: Vec<(ParaId, BlockNumber)>;
 FutureCodeUpgrades: map ParaId => Option<BlockNumber>;
 /// The actual future code of a para.
 FutureCode: map ParaId => Option<ValidationCode>;
-
-/// Upcoming paras (chains and threads). These are only updated on session change. Corresponds to an
-/// entry in the upcoming-genesis map. Ordered ascending by ParaId.
-UpcomingParas: Vec<ParaId>;
+/// The actions to perform during the start of a specific session index.
+ActionsQueue: map SessionIndex => Vec<ParaId>;
 /// Upcoming paras instantiation arguments.
 UpcomingParasGenesis: map ParaId => Option<ParaGenesisArgs>;
-/// Paras that are to be cleaned up at the end of the session. Ordered ascending by ParaId.
-OutgoingParas: Vec<ParaId>;
-/// Existing Parathreads that should upgrade to be a Parachain. Ordered ascending by ParaId.
-UpcomingUpgrades: Vec<ParaId>;
-/// Existing Parachains that should downgrade to be a Parathread. Ordered ascending by ParaId.
-UpcomingDowngrades: Vec<ParaId>;
 ```
 
 ## Session Change
 
-1. Clean up outgoing paras.
-  1. This means removing the entries under `Heads`, `ValidationCode`, `FutureCodeUpgrades`, and
-     `FutureCode`. An according entry should be added to `PastCode`, `PastCodeMeta`, and
-     `PastCodePruning` using the outgoing `ParaId` and removed `ValidationCode` value. This is
-     because any outdated validation code must remain available on-chain for a determined amount of
-     blocks, and validation code outdated by de-registering the para is still subject to that
-     invariant.
-1. Apply all incoming paras by initializing the `Heads` and `ValidationCode` using the genesis
-   parameters.
-1. Amend the `Parachains` list and `ParaLifecycle` to reflect changes in registered parachains.
-1. Amend the `ParaLifecycle` set to reflect changes in registered parathreads.
-1. Upgrade all parathreads that should become parachains, updating the `Parachains` list and
-   `ParaLifecycle`.
-1. Downgrade all parachains that should become parathreads, updating the `Parachains` list and
-   `ParaLifecycle`.
-1. Return list of outgoing paras to the initializer for use by other modules.
+1. Execute all queued actions for paralifecycle changes:
+  1. Clean up outgoing paras.
+     1. This means removing the entries under `Heads`, `ValidationCode`, `FutureCodeUpgrades`, and
+        `FutureCode`. An according entry should be added to `PastCode`, `PastCodeMeta`, and
+        `PastCodePruning` using the outgoing `ParaId` and removed `ValidationCode` value. This is
+        because any outdated validation code must remain available on-chain for a determined amount
+        of blocks, and validation code outdated by de-registering the para is still subject to that
+        invariant.
+  1. Apply all incoming paras by initializing the `Heads` and `ValidationCode` using the genesis
+     parameters.
+  1. Amend the `Parachains` list and `ParaLifecycle` to reflect changes in registered parachains.
+  1. Amend the `ParaLifecycle` set to reflect changes in registered parathreads.
+  1. Upgrade all parathreads that should become parachains, updating the `Parachains` list and
+     `ParaLifecycle`.
+  1. Downgrade all parachains that should become parathreads, updating the `Parachains` list and
+     `ParaLifecycle`.
+  1. Return list of outgoing paras to the initializer for use by other modules.
 
 ## Initialization
 
@@ -179,11 +172,9 @@ UpcomingDowngrades: Vec<ParaId>;
 
 * `schedule_para_initialize(ParaId, ParaGenesisArgs)`: Schedule a para to be initialized at the next
   session. Noop if para is already registered in the system with some `ParaLifecycle`.
-* `schedule_para_cleanup(ParaId)`: Schedule a para to be cleaned up at the next session.
-* `schedule_parathread_upgrade(ParaId)`: Schedule a parathread to be upgraded to a parachain. Noop
-  if `ParaLifecycle` is not `Parathread`.
+* `schedule_para_cleanup(ParaId)`: Schedule a para to be cleaned up after the next full session.
+* `schedule_parathread_upgrade(ParaId)`: Schedule a parathread to be upgraded to a parachain.
 * `schedule_parachain_downgrade(ParaId)`: Schedule a parachain to be downgraded to a parathread.
-  Noop if `ParaLifecycle` is not `Parachain`.
 * `schedule_code_upgrade(ParaId, ValidationCode, expected_at: BlockNumber)`: Schedule a future code
   upgrade of the given parachain, to be applied after inclusion of a block of the same parachain
   executed in the context of a relay-chain block with number >= `expected_at`.
@@ -197,8 +188,8 @@ UpcomingDowngrades: Vec<ParaId>;
   current, or (with certain choices of `assume_intermediate`) future code. `assume_intermediate`, if
   provided, must be before `at`. If the validation code has been pruned, this will return `None`.
 * `lifecycle(ParaId) -> Option<ParaLifecycle>`: Return the `ParaLifecycle` of a para.
-* `is_parachain(ParaId) -> bool`: Returns true if the para ID references any live parachain, including
-  those which may be transitioning to a parathread in the future.
+* `is_parachain(ParaId) -> bool`: Returns true if the para ID references any live parachain,
+  including those which may be transitioning to a parathread in the future.
 * `is_parathread(ParaId) -> bool`: Returns true if the para ID references any live parathread,
   including those which may be transitioning to a parachain in the future.
 * `is_valid_para(ParaId) -> bool`: Returns true if the para ID references either a live parathread

roadmap/implementers-guide/src/runtime/shared.md

@@ -0,0 +1,55 @@
+# Shared Module
+
+This module is responsible for managing shared storage and configuration for other modules.
+
+It is important that other pallets are able to use the Shared Module, so it should not have a
+dependency on any other modules in the Parachains Runtime.
+
+For the moment, it is used exclusively to track the current session index across the Parachains
+Runtime system, and when it should be allowed to schedule future changes to Paras or Configurations.
+
+## Constants
+
+```rust
+// `SESSION_DELAY` is used to delay any changes to Paras registration or configurations.
+// Wait until the session index is 2 larger then the current index to apply any changes,
+// which guarantees that at least one full session has passed before any changes are applied.
+pub(crate) const SESSION_DELAY: SessionIndex = 2;
+```
+
+## Storage
+
+```rust
+// The current session index within the Parachains Runtime system.
+CurrentSessionIndex: SessionIndex;
+```
+
+## Initialization
+
+The Shared Module currently has no initialization routines.
+
+The Shared Module is initialized directly after the Configuration module, but before all other
+modules. It is important to update the Shared Module before any other module since its state may be
+used within the logic of other modules, and it is important that the state is consistent across
+them.
+
+## Session Change
+
+During a session change, the Shared Module receives and stores the current Session Index for that
+block through the Session Change Notification.
+
+This information is used in the:
+
+* Configuration Module: For delaying updates to configurations until at lease one full session has
+  passed.
+* Paras Module: For delaying updates to paras until at least one full session has passed.
+
+## Finalization
+
+The Shared Module currently has no finalization routines.
+
+## Functions
+
+* `scheduled_sessions() -> SessionIndex`: Return the next session index where updates to the
+  Parachains Runtime system would be safe to apply.
+* `set_session_index(SessionIndex)`: For tests. Set the current session index in the Shared Module.