Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AHM] Stage management #584

Open
wants to merge 7 commits into
base: dev-asset-hub-migration
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions pallets/ah-migrator/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ sp-runtime = { workspace = true }
sp-std = { workspace = true }
hex-literal = { workspace = true }
hex = { workspace = true }
xcm = { workspace = true }

[features]
default = ["std"]
Expand Down Expand Up @@ -79,6 +80,7 @@ std = [
"sp-io/std",
"sp-runtime/std",
"sp-std/std",
"xcm/std",
]
runtime-benchmarks = [
"frame-benchmarking/runtime-benchmarks",
Expand Down
100 changes: 99 additions & 1 deletion pallets/ah-migrator/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ use sp_runtime::{
AccountId32, FixedU128,
};
use sp_std::prelude::*;
use xcm::prelude::*;

/// The log target of this pallet.
pub const LOG_TARGET: &str = "runtime::ah-migrator";
Expand All @@ -103,6 +104,20 @@ pub enum PalletEventName {
Bounties,
}

/// The migration stage on the Asset Hub.
#[derive(Encode, Decode, Clone, Default, RuntimeDebug, TypeInfo, MaxEncodedLen, PartialEq, Eq)]
pub enum MigrationStage {
/// The migration has not started but will start in the future.
#[default]
Pending,
/// Migrating data from the Relay Chain.
DataMigrationOngoing,
/// Migrating data from the Relay Chain is completed.
DataMigrationDone,
/// The migration is done.
MigrationDone,
}

#[frame_support::pallet(dev_mode)]
pub mod pallet {
use super::*;
Expand Down Expand Up @@ -130,6 +145,10 @@ pub mod pallet {
{
/// The overarching event type.
type RuntimeEvent: From<Event<Self>> + IsType<<Self as frame_system::Config>::RuntimeEvent>;
/// The origin that can perform permissioned operations like setting the migration stage.
///
/// This is generally root and Fellows origins.
type ManagerOrigin: EnsureOrigin<<Self as frame_system::Config>::RuntimeOrigin>;
/// Native asset registry type.
type Currency: Mutate<Self::AccountId, Balance = u128>
+ MutateHold<Self::AccountId, Reason = Self::RuntimeHoldReason>
Expand Down Expand Up @@ -169,6 +188,8 @@ pub mod pallet {
type Preimage: QueryPreimage<H = <Self as frame_system::Config>::Hashing> + StorePreimage;
/// Convert a Relay Chain Call to a local AH one.
type RcToAhCall: for<'a> TryConvert<&'a [u8], <Self as frame_system::Config>::RuntimeCall>;
/// Send UMP message.
type SendXcm: SendXcm;
}

/// RC accounts that failed to migrate when were received on the Asset Hub.
Expand All @@ -178,6 +199,10 @@ pub mod pallet {
pub type RcAccounts<T: Config> =
StorageMap<_, Twox64Concat, T::AccountId, RcAccountFor<T>, OptionQuery>;

/// The Asset Hub migration state.
#[pallet::storage]
pub type AhMigrationStage<T: Config> = StorageValue<_, MigrationStage, ValueQuery>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if ValueQuery makes sense if our default is Pending which says that the migration will start in the next block.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the doc for Pending in both pallets (...will start in the future). In both pallets it's ValueQuery. I think its better than the alternative with OptionQuery and no Pending variants at all (otherwise its never reachable stage).


#[pallet::error]
pub enum Error<T> {
/// The error that should to be replaced by something meaningful.
Expand All @@ -195,14 +220,22 @@ pub mod pallet {
FailedToConvertCall,
/// Failed to bound a call.
FailedToBoundCall,
/// Failed to send XCM message.
XcmError,
}

#[pallet::event]
#[pallet::generate_deposit(pub(crate) fn deposit_event)]
pub enum Event<T: Config> {
/// The event that should to be replaced by something meaningful.
TODO,

/// A stage transition has occurred.
StageTransition {
/// The old stage before the transition.
old: MigrationStage,
/// The new stage after the transition.
new: MigrationStage,
},
/// We received a batch of accounts that we are going to integrate.
AccountBatchReceived {
/// How many accounts are in the batch.
Expand Down Expand Up @@ -570,6 +603,29 @@ pub mod pallet {

Self::do_receive_asset_rates(rates).map_err(Into::into)
}

/// Set the migration stage.
///
/// This call is intended for emergency use only and is guarded by the
/// [`Config::ManagerOrigin`].
#[pallet::call_index(100)]
pub fn force_set_stage(origin: OriginFor<T>, stage: MigrationStage) -> DispatchResult {
<T as Config>::ManagerOrigin::ensure_origin(origin)?;
Self::transition(stage);
Ok(())
}

/// Start the data migration.
///
/// This is typically called by the Relay Chain to start the migration on the Asset Hub and
/// receive a handshake message indicating the Asset Hub's readiness.
#[pallet::call_index(101)]
pub fn start_migration(origin: OriginFor<T>) -> DispatchResult {
<T as Config>::ManagerOrigin::ensure_origin(origin)?;
Self::send_xcm(types::RcMigratorCall::StartDataMigration)?;
Self::transition(MigrationStage::DataMigrationOngoing);
Ok(())
}
}

#[pallet::hooks]
Expand All @@ -579,6 +635,48 @@ pub mod pallet {
}
}

impl<T: Config> Pallet<T> {
/// Execute a stage transition and log it.
fn transition(new: MigrationStage) {
let old = AhMigrationStage::<T>::get();
AhMigrationStage::<T>::put(&new);
log::info!(
target: LOG_TARGET,
"[Block {:?}] Stage transition: {:?} -> {:?}",
frame_system::Pallet::<T>::block_number(),
&old,
&new
);
Self::deposit_event(Event::StageTransition { old, new });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this should send an UMP each time there is a stage transition? Then the relay can keep track of progress and we only have to send it from one spot in the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But what we achieve by that? AH does not really know much about the stage (it does not know if all pallets data is sent).
I can see that we can have AH sending an XCM message after it processes every XCM message, but it might increase the migration time significantly.
Or we can let AH know that this message with the data is the last on this pallet, but AH still can get congested just on one pallet with a lot of data.

}

/// Send a single XCM message.
pub fn send_xcm(call: types::RcMigratorCall) -> Result<(), Error<T>> {
log::info!(target: LOG_TARGET, "Sending XCM message");

let call = types::RcPalletConfig::RcmController(call);

let message = Xcm(vec![
Instruction::UnpaidExecution {
weight_limit: WeightLimit::Unlimited,
check_origin: None,
},
Instruction::Transact {
origin_kind: OriginKind::Superuser,
require_weight_at_most: Weight::from_all(1), // TODO
call: call.encode().into(),
},
]);

if let Err(err) = send_xcm::<T::SendXcm>(Location::parent(), message.clone()) {
log::error!(target: LOG_TARGET, "Error while sending XCM message: {:?}", err);
return Err(Error::XcmError);
};

Ok(())
}
}

impl<T: Config> pallet_rc_migrator::types::MigrationStatus for Pallet<T> {
fn is_ongoing() -> bool {
// TODO: implement
Expand Down
14 changes: 14 additions & 0 deletions pallets/ah-migrator/src/types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,17 @@ pub fn map_lock_reason(reasons: LockReasons) -> LockWithdrawReasons {
LockReasons::Misc => LockWithdrawReasons::TIP,
}
}

/// Relay Chain pallet list with indexes.
#[derive(Encode, Decode)]
pub enum RcPalletConfig {
#[codec(index = 255)]
RcmController(RcMigratorCall),
}

/// Call encoding for the calls needed from the rc-migrator pallet.
#[derive(Encode, Decode)]
pub enum RcMigratorCall {
#[codec(index = 2)]
StartDataMigration,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should maybe be something like AhStageChanged(stage) and then define the stage type either in the RC pallet or in a shared crate.

Copy link
Contributor Author

@muharem muharem Feb 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, can be put this way. but if we have only few (e.g. three) stage notification from ah, a separate calls might be simpler

}
78 changes: 75 additions & 3 deletions pallets/rc-migrator/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -122,9 +122,18 @@ pub enum PalletEventName {
#[derive(Encode, Decode, Clone, Default, RuntimeDebug, TypeInfo, MaxEncodedLen, PartialEq, Eq)]
pub enum MigrationStage<AccountId, BlockNumber, BagsListScore, AccountIndex, VotingClass, AssetKind>
{
/// The migration has not yet started but will start in the next block.
/// The migration has not yet started but will start in the future.
#[default]
Pending,
/// The migration has been scheduled to start at the given block number.
Scheduled {
block_number: BlockNumber,
},
/// The migration is initializing.
///
/// This stage involves waiting for the notification from the Asset Hub that it is ready to
/// receive the migration data.
Initializing,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Initializing,
WaitingForAhReadiness,

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may be I better update the doc? It will be more general and if needed we can include more into it.

/// Initializing the account migration process.
AccountsMigrationInit,
// TODO: Initializing?
Expand Down Expand Up @@ -272,6 +281,8 @@ type AccountInfoFor<T> =

#[frame_support::pallet(dev_mode)]
pub mod pallet {
use frame_support::traits::schedule::DispatchTime;

use super::*;

/// Paras Registrar Pallet
Expand Down Expand Up @@ -302,6 +313,10 @@ pub mod pallet {
{
/// The overarching event type.
type RuntimeEvent: From<Event<Self>> + IsType<<Self as frame_system::Config>::RuntimeEvent>;
/// The origin that can perform permissioned operations like setting the migration stage.
///
/// This is generally root, Asset Hub and Fellows origins.
type ManagerOrigin: EnsureOrigin<<Self as frame_system::Config>::RuntimeOrigin>;
/// Native asset registry type.
type Currency: Mutate<Self::AccountId, Balance = u128>
+ MutateHold<Self::AccountId, Reason = Self::RuntimeHoldReason>
Expand Down Expand Up @@ -384,18 +399,75 @@ pub mod pallet {
#[pallet::pallet]
pub struct Pallet<T>(_);

#[pallet::call]
impl<T: Config> Pallet<T> {
/// Set the migration stage.
///
/// This call is intended for emergency use only and is guarded by the
/// [`Config::ManagerOrigin`].
#[pallet::call_index(0)]
pub fn force_set_stage(origin: OriginFor<T>, stage: MigrationStageOf<T>) -> DispatchResult {
<T as Config>::ManagerOrigin::ensure_origin(origin)?;
Self::transition(stage);
Ok(())
}

/// Schedule the migration to start at a given moment.
#[pallet::call_index(1)]
pub fn schedule_migration(
origin: OriginFor<T>,
start_moment: DispatchTime<BlockNumberFor<T>>,
) -> DispatchResult {
<T as Config>::ManagerOrigin::ensure_origin(origin)?;
let now = frame_system::Pallet::<T>::block_number();
let block_number = start_moment.evaluate(now);
Self::transition(MigrationStage::Scheduled { block_number });
Ok(())
}

/// Start the data migration.
///
/// This is typically called by the Asset Hub to indicate it's readiness to receive the
/// migration data.
#[pallet::call_index(2)]
pub fn start_data_migration(origin: OriginFor<T>) -> DispatchResult {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pub fn start_data_migration(origin: OriginFor<T>) -> DispatchResult {
pub fn ah_ready_for_data(origin: OriginFor<T>) -> DispatchResult {

? I think it could help to include ah in there to show that it comes from the AH.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think can be more general, we might include more. But also no strong opinion. I just trying to not create too many stages to not waste time.

<T as Config>::ManagerOrigin::ensure_origin(origin)?;
Self::transition(MigrationStage::AccountsMigrationInit);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe defensive assert to check that we are in the right phase.

Ok(())
}
}

#[pallet::hooks]
impl<T: Config> Hooks<BlockNumberFor<T>> for Pallet<T> {
fn on_initialize(_: BlockNumberFor<T>) -> Weight {
fn on_initialize(now: BlockNumberFor<T>) -> Weight {
let mut weight_counter = WeightMeter::with_limit(T::MaxRcWeight::get());
let stage = RcMigrationStage::<T>::get();
weight_counter.consume(T::DbWeight::get().reads(1));

match stage {
MigrationStage::Pending => {
// TODO: not complete
// TODO: we should do nothing on pending stage.
// On production the AH will send a message and initialize the migration.
// Now we transition to `AccountsMigrationInit` to run tests
Self::transition(MigrationStage::AccountsMigrationInit);
},
MigrationStage::Scheduled { block_number } =>
if now >= block_number {
match Self::send_xcm(types::AhMigratorCall::<T>::StartMigration) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
match Self::send_xcm(types::AhMigratorCall::<T>::StartMigration) {
match Self::send_xcm(types::AhMigratorCall::<T>::IsAhReadyToMigrate) {

Sorry for the silly naming suggestions, but i think it can help later to see what is happening.

Ok(_) => {
Self::transition(MigrationStage::Initializing);
},
Err(_) => {
defensive!(
"Failed to send StartMigration message to AH, \
retry with the next block"
);
},
}
},
MigrationStage::Initializing => {
// waiting AH to send a message and to start sending the data
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could add some timeout in case AH does not send back the Ok, but should not do now.

},
MigrationStage::AccountsMigrationInit => {
// TODO: weights
let _ = AccountsMigrator::<T>::obtain_rc_accounts();
Expand Down
7 changes: 5 additions & 2 deletions pallets/rc-migrator/src/types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,14 @@ use sp_runtime::FixedU128;

pub type AccountIdOf<T> = <T as frame_system::Config>::AccountId;

/// Relay Chain Freeze Reason
/// Asset Hub Pallet list with indexes.
#[derive(Encode, Decode)]
pub enum AssetHubPalletConfig<T: Config> {
#[codec(index = 255)]
AhmController(AhMigratorCall<T>),
}

/// Call encoding for the calls needed from the Broker pallet.
/// Call encoding for the calls needed from the ah-migrator pallet.
#[derive(Encode, Decode)]
pub enum AhMigratorCall<T: Config> {
#[codec(index = 0)]
Expand Down Expand Up @@ -74,6 +74,9 @@ pub enum AhMigratorCall<T: Config> {
ReceiveBountiesMessages { messages: Vec<bounties::RcBountiesMessageOf<T>> },
#[codec(index = 17)]
ReceiveAssetRates { asset_rates: Vec<(<T as pallet_asset_rate::Config>::AssetKind, FixedU128)> },

#[codec(index = 101)]
StartMigration,
}

/// Copy of `ParaInfo` type from `paras_registrar` pallet.
Expand Down
6 changes: 3 additions & 3 deletions relay/polkadot/src/ah_migration/phase1.rs
Original file line number Diff line number Diff line change
Expand Up @@ -124,8 +124,8 @@ pub fn call_allowed_status(call: &<Runtime as frame_system::Config>::RuntimeCall
XcmPallet(..) => (OFF, ON), /* TODO allow para origins and root to call this during the migration, see https://github.com/polkadot-fellows/runtimes/pull/559#discussion_r1928789463 */
MessageQueue(..) => (ON, ON), // TODO think about this
AssetRate(..) => (OFF, OFF),
Beefy(..) => (OFF, ON), /* TODO @claravanstaden @bkontur
* RcMigrator has no calls currently
* Exhaustive match. Compiler ensures that we did not miss any. */
Beefy(..) => (OFF, ON), /* TODO @claravanstaden @bkontur */
RcMigrator(..) => (ON, ON),
// Exhaustive match. Compiler ensures that we did not miss any.
}
}
9 changes: 1 addition & 8 deletions relay/polkadot/src/governance/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,9 @@
//! New governance configurations for the Polkadot runtime.

use super::*;
use crate::xcm_config::CollectivesLocation;
use crate::xcm_config::{CollectivesLocation, FellowsBodyId};
use frame_support::parameter_types;
use frame_system::EnsureRootWithSuccess;
use pallet_xcm::{EnsureXcm, IsVoiceOfBody};
use xcm::latest::BodyId;

mod origins;
pub use origins::{
Expand Down Expand Up @@ -59,11 +57,6 @@ pub type TreasurySpender = EitherOf<EnsureRootWithSuccess<AccountId, MaxBalance>

impl origins::pallet_custom_origins::Config for Runtime {}

parameter_types! {
// Fellows pluralistic body.
pub const FellowsBodyId: BodyId = BodyId::Technical;
}

impl pallet_whitelist::Config for Runtime {
type WeightInfo = weights::pallet_whitelist::WeightInfo<Self>;
type RuntimeCall = RuntimeCall;
Expand Down
Loading
Loading