Update polkadot-v0.9.43 #456

imstar15 · 2023-10-18T01:12:00Z

No description provided.

chrisli30

The code is able to compile and unit tests are passing now.

Subsequently, we will need to run

upgrade test script
make sure tasks still run before and after upgrade
make sure staking can work in dev environment.

chrisli30 · 2023-10-19T01:22:47Z

pallets/automation-time/src/tests.rs

@@ -2160,7 +2160,7 @@ fn trigger_tasks_limits_missed_slots() {
 			AutomationTime::get_last_slot()
 		{
 			assert_eq!(updated_last_time_slot, SCHEDULED_TIME);
-			assert_eq!(updated_last_missed_slot, SCHEDULED_TIME - SLOT_SIZE_SECONDS * 3);
+			assert_eq!(updated_last_missed_slot, SCHEDULED_TIME - SLOT_SIZE_SECONDS * 2);


Add a comment here.

This line ensures when given a total weight of 9_769_423 + 200_000, missing_task_id5, missing_task_id4, missing_task_id3 and missing_task_id2 will be discarded from the missed_queue in the current version of code. // TODO: we should examine the tasks in missed_queue instead of examing the timestamp of missing_task_id2

code change from 3 to 2? could be some regression ?

Because scheduled_call.get_dispatch_info().weight have changed with polkadot-v.9.43, We need to change the test code.

https://github.com/OAK-Foundation/OAK-blockchain/blob/0769fcd72f738d83a5c299bf978ace308d423f0b/pallets/automation-time/src/lib.rs#L1130

imstar15 · 2023-10-19T13:35:18Z

Xcm demo status:

Mangata: OK
Astar: OK
Moonbeam: OK

Make sure tasks still run before and after upgrade
OK

imstar15 · 2023-10-24T14:07:41Z

Migrations:

1. Bounties

The migration of Bounties is to change the prefix of pallet_bounties storage from Treasury to Bounties in Polkadot runtime. In Turing's storage, it's already set as Bounties, and we don't need to migrate it.

We only need to set the storageVersion to 4.

Reference: https://github.com/paritytech/polkadot/blob/ba42b9ce51d25bdaf52d2c61e0763a6e3da50d25/runtime/rococo/src/lib.rs#L1601

2. PolkadotXcm

V0: (u64, u64, u32)

V1: (u64, Weight, u32)

Some data in the on-chain storage structure is at version 0.

Therefore, we need to perform migration.

3. Preimage

v0:

https://github.com/paritytech/substrate/blob/033d4e86cc7eff0066cd376b9375f815761d653c/frame/preimage/src/migration.rs#L40

#[derive(Clone, Eq, PartialEq, Encode, Decode, TypeInfo, MaxEncodedLen, RuntimeDebug)]
pub enum RequestStatus<AccountId, Balance> {
	Unrequested(Option<(AccountId, Balance)>),
	Requested(u32),
}

v1:

https://github.com/paritytech/substrate/blob/033d4e86cc7eff0066cd376b9375f815761d653c/frame/preimage/src/lib.rs#L64

/// A type to note whether a preimage is owned by a user or the system.
#[derive(Clone, Eq, PartialEq, Encode, Decode, TypeInfo, MaxEncodedLen, RuntimeDebug)]
pub enum RequestStatus<AccountId, Balance> {
	/// The associated preimage has not yet been requested by the system. The given deposit (if
	/// some) is being held until either it becomes requested or the user retracts the preimage.
	Unrequested { deposit: (AccountId, Balance), len: u32 },
	/// There are a non-zero number of outstanding requests for this hash by this chain. If there
	/// is a preimage registered, then `len` is `Some` and it may be removed iff this counter
	/// becomes zero.
	Requested { deposit: Option<(AccountId, Balance)>, count: u32, len: Option<u32> },
}

The on-chain storage structure is already at version 1.

We only need to set the storageVersion to 1.

4. Scheduler

v3:

https://github.com/paritytech/substrate/blob/5e49f6e44820affccaf517fd22af564f4b495d40/frame/scheduler/src/lib.rs#L97

struct ScheduledV1<Call, BlockNumber> {
	maybe_id: Option<Vec<u8>>,
	priority: schedule::Priority,
	call: Call,
	maybe_periodic: Option<schedule::Period<BlockNumber>>,
}

v4:

https://github.com/paritytech/substrate/blob/5e49f6e44820affccaf517fd22af564f4b495d40/frame/scheduler/src/lib.rs#L107

pub struct Scheduled<Name, Call, BlockNumber, PalletsOrigin, AccountId> {
	/// The unique identity for this task, if there is one.
	maybe_id: Option<Name>,
	/// This task's priority.
	priority: schedule::Priority,
	/// The call to be dispatched.
	call: Call,
	/// If the call is periodic, then this points to the information concerning that.
	maybe_periodic: Option<schedule::Period<BlockNumber>>,
	/// The origin with which to dispatch the call.
	origin: PalletsOrigin,
	_phantom: PhantomData<AccountId>,
}

The on-chain storage structure is already at version 4.

We only need to set the storageVersion to 4.

5. Multisig

v0:
https://github.com/paritytech/substrate/blob/5e49f6e44820affccaf517fd22af564f4b495d40/frame/multisig/src/migrations.rs#L36

#[frame_support::storage_alias]
type Calls<T: Config> = StorageMap<
  Pallet<T>,
  Identity,
  [u8; 32],
  (OpaqueCall<T>, <T as frame_system::Config>::AccountId, BalanceOf<T>),
>;

Version 1 migration Remove Calls stroage and refund.

2023-10-26 14:54:46.933  INFO main runtime::multisig: [3733596] ✍️ Number of calls to refund and delete: 0

No Calls record in the chain storage.

We only need to set the storageVersion to 1.

6. Democracy

v0:

https://github.com/paritytech/substrate/blob/5e49f6e44820affccaf517fd22af564f4b495d40/frame/democracy/src/migrations.rs#L39

#[pallet::storage]
pub type NextExternal<T: Config> = StorageValue<_, (BoundedCallOf<T>, VoteThreshold)>;

v1:

https://github.com/paritytech/substrate/blob/5e49f6e44820affccaf517fd22af564f4b495d40/frame/democracy/src/lib.rs#L416

#[storage_alias]
pub type NextExternal<T: Config> = StorageValue<Pallet<T>, (<T as frame_system::Config>::Hash, VoteThreshold)>;

All data in the on-chain storage are at version 1.

We only need to set the storageVersion to 1.

7. AssetRegistry

v0:

pub type LocationToAssetId<T: Config> = StorageMap<_, Twox64Concat, xcm::v2::MultiLocation, T::AssetId, OptionQuery>;

https://github.com/open-web3-stack/open-runtime-module-library/blob/83a76c2bf66c0b1236c31077e6fb24bb760a3535/asset-registry/src/migrations.rs#L30

v1:

https://github.com/open-web3-stack/open-runtime-module-library/blob/83a76c2bf66c0b1236c31077e6fb24bb760a3535/asset-registry/src/lib.rs#L114

pub type LocationToAssetId<T: Config> = StorageMap<_, Twox64Concat, xcm::v3::MultiLocation, T::AssetId, OptionQuery>;

Some on-chain data is of type xcm::v2::Multilocation and cannot be decoded.
Therefore, we need to perform migration.

8. UnknownTokens

v0:

https://github.com/open-web3-stack/open-runtime-module-library/blob/83a76c2bf66c0b1236c31077e6fb24bb760a3535/unknown-tokens/src/migrations.rs#L43

pub(crate) type ConcreteFungibleBalances<T> =
		StorageDoubleMap<_, Blake2_128Concat, xcm::v2::MultiLocation, Blake2_128Concat, xcm::v2::MultiLocation, u128, ValueQuery>;
		
pub(crate) type AbstractFungibleBalances<T> =
		StorageDoubleMap<_, Blake2_128Concat, xcm::v2::MultiLocation, Blake2_128Concat, Vec<u8>, u128, ValueQuery>;

v1:

https://github.com/open-web3-stack/open-runtime-module-library/blob/83a76c2bf66c0b1236c31077e6fb24bb760a3535/unknown-tokens/src/lib.rs#L59

pub(crate) type ConcreteFungibleBalances<T> =
		StorageDoubleMap<_, Blake2_128Concat, xcm::v3::MultiLocation, Blake2_128Concat, xcm::v3::MultiLocation, u128, ValueQuery>;
		
pub(crate) type AbstractFungibleBalances<T> =
		StorageDoubleMap<_, Blake2_128Concat, xcm::v3::MultiLocation, Blake2_128Concat, Vec<u8>, u128, ValueQuery>;

We need to perform migration to migrate MultiLocation

9. XcmpQueue, DmpQueue

Omit

The reason for this error is a bug in the Cumulus code. In the code, the version is defined as 1, but in the migration process, it is being upgraded to 2.

https://github.com/paritytech/cumulus/blob/b8999fce0f61fb757f9e57e326cda48e70137019/pallets/dmp-queue/src/migration.rs#L42

/// The current storage version.
pub const STORAGE_VERSION: StorageVersion = StorageVersion::new(1);

/// Migrates the pallet storage to the most recent version, checking and setting the
/// `StorageVersion`.
pub fn migrate_to_latest<T: Config>() -> Weight {
	...
	if StorageVersion::get::<Pallet<T>>() == 1 {
		weight.saturating_accrue(migrate_to_v2::<T>());
		StorageVersion::new(2).put::<Pallet<T>>();
		weight.saturating_accrue(T::DbWeight::get().writes(1));
	}
    ...
}

v9n · 2023-10-25T00:03:08Z

node/src/chain_spec/oak.rs

@@ -343,8 +343,7 @@ fn testnet_genesis(
 	general_councils: Vec<AccountId>,
 	technical_memberships: Vec<AccountId>,
 ) -> oak_runtime::GenesisConfig {
-	let candidate_stake =
-		std::cmp::max(oak_runtime::MinCollatorStk::get(), oak_runtime::MinCandidateStk::get());
+	let candidate_stake = oak_runtime::MinCandidateStk::get();


this logic is changed. is this internally done this way?

Yes.

The new version of the parachain-staking pallet has removed the MinCollatorStk setting and instead uses MinCandidateStk to limit the threshold of candidate.

A new set of collators is chosen from the candidates.

moonbeam-foundation/moonbeam@25d02f3

In fact, in the implementation of the join_candidates function, it uses the MinCandidateStk value to restrict the entry threshold. It requires bond >= T::MinCandidateStk::get() and locks these bonded assets.

Join candidates function:
https://github.com/moonbeam-foundation/moonbeam/blob/25d02f3c952c9827bb6d938da5c6a60ed5ccf420/pallets/parachain-staking/src/lib.rs#L936C1-L936C1

Lock bond:
https://github.com/moonbeam-foundation/moonbeam/blob/25d02f3c952c9827bb6d938da5c6a60ed5ccf420/pallets/parachain-staking/src/lib.rs#L956C1-L956C1

Good call! @v9n

I sync’ed with Charles. The value of MinCandidateStk was updated from 400K to 1M 5 months ago but MinCollatorStk wasn’t, so we haven’t been using MinCollatorStk for a while. Therefore, since MinCollatorStk's value has been already out-of-date I think it’s safe to delete its logic now. As Charles said, the MinCandidateStk alone can safe guard the candidacy entrance.

v9n · 2023-10-25T00:03:42Z

node/src/chain_spec/turing.rs

-		turing_runtime::MinCollatorStk::get(),
-		turing_runtime::MinCandidateStk::get(),
-	);
+	let candidate_stake = turing_runtime::MinCandidateStk::get();


same as above

v9n · 2023-10-26T03:47:57Z

pallets/automation-time/rpc/runtime-api/src/lib.rs

 #[cfg_attr(feature = "std", derive(Serialize, Deserialize))]
 #[cfg_attr(feature = "std", serde(rename_all = "camelCase"))]
 #[cfg_attr(feature = "std", serde(deny_unknown_fields))]
 pub struct AutostakingResult {
 	pub period: i32,
-	pub apy: f64,
+	pub apy: String,


this change from a number to a string? is that right?

Yes, AutostakingResult must implement TypeInfo trait in polkadot-v0.9.43.
But f64 does not implement TypeInfo.
So I change it to String instead.

v9n · 2023-10-26T03:51:41Z

runtime/neumann/src/weights/pallet_xcm.rs

+		//  Measured:  `0`
+		//  Estimated: `0`
+		// Minimum execution time: 2_972_000 picoseconds.
+		Weight::from_parts(3_082_000, 0)


should this number be pull froma benchmark ?

Yes, we should run a benchmark on a standard machine to generate this value.
However, this weight value isn't very critical at the moment because it is typically only called by AdminOrigin.

https://github.com/paritytech/polkadot/blob/ba42b9ce51d25bdaf52d2c61e0763a6e3da50d25/xcm/pallet-xcm/src/lib.rs#L1131

In kusama, the weight is 3_078_000.

https://github.com/paritytech/polkadot/blob/ba42b9ce51d25bdaf52d2c61e0763a6e3da50d25/runtime/kusama/src/weights/pallet_xcm.rs#L179

v9n · 2023-10-26T04:12:29Z

runtime/turing/src/lib.rs

+	/// We opted to not check exactly what on-chain versions each pallet is at, since it would be
+	/// an involved effort, this is testnet, and no one has complained
+	/// (https://github.com/paritytech/polkadot/issues/6657#issuecomment-1552956439).
+	pub struct SetStorageVersions;


love the comment and background.

v9n · 2023-10-26T04:13:16Z

runtime/turing/src/lib.rs

+				log::info!("The pallet_xcm is migrating to version 1");
+				let mut weight = T::DbWeight::get().reads(1);
+
+				let translate = |pre: (u64, Weight, u32)| -> Option<(u64, Weight, u32)> {


we should explain on why we do (u64, Weight, u32) -> (u64, Weight, u32) to restore the proof size

v9n

looks good to me so far. i had some general comment but want to point out so we can understand that it is. but don't think there is any blocker.

v9n

looks great. let merge 🥳

imstar15 force-pushed the oak-polkadot-v0.9.43 branch 2 times, most recently from d7a23f6 to 29f3c97 Compare October 18, 2023 17:28

chrisli30 reviewed Oct 19, 2023

View reviewed changes

imstar15 force-pushed the oak-polkadot-v0.9.43 branch from 90649c7 to 736f7df Compare October 21, 2023 03:49

v9n reviewed Oct 25, 2023

View reviewed changes

v9n reviewed Oct 26, 2023

View reviewed changes

v9n approved these changes Oct 26, 2023

View reviewed changes

imstar15 and others added 17 commits October 26, 2023 16:38

Update polkadot-v0.9.43

a300e63

Fix compliant error

ce64098

Fix compliant errors in oak-node crate

de3e45a

fixup: auto format Rust code

31c933e

Remove unused code

fbdb2ce

Fix automation-time RPC

7caafc8

Fix compliant erros with --all-features

26aba53

fixup: auto format Rust code

2ff0c61

Update frame-weight-template.hbs

e6d326f

Replace Weight::from_ref_time with Weight::from_parts

513ed05

fixup: auto format Rust code

50adf5e

Remove pallet::generate_store marco

b9bd49a

Replace set_balance with force_set_balance

a091378

Fix test cases

6298698

Set pallet-parachain-staking branch to oak-polkadot-0.9.43

84fd678

Fix trigger_tasks_limits_missed_slots unit test

a81631d

fixup: auto format Rust code

90762ec

imstar15 and others added 16 commits October 26, 2023 16:40

Update rust version to nightly-2023-05-02

e39de74

Format Rust code

bb36e22

Add pallet::storage_version to Pallet definition

7e6b43a

Add pallet migrations in turing runtime

b2711c2

Upgrade Bounties storage version

19f51c4

fixup: auto format Rust code

6c3ffe6

Set PolkadotXcm storage version to 1

7eacf6f

Upgrade Preimage , Scheduler storage version

3ef6b82

Upgrade Multisig storage version

cfdc50e

Upgrade Democracy storage version

db3c406

Add pallet_xcm migration

afafa0e

fixup: auto format Rust code

a9b1acc

Add comment for trigger_tasks_limits_missed_slots test case

a607105

fixup: auto format Rust code

871d8eb

Fix automation-price with polkadot-v0.9.43

fc90557

Update pallet-parachain-staking

0769fcd

imstar15 force-pushed the oak-polkadot-v0.9.43 branch from 9fd56e6 to 0769fcd Compare October 26, 2023 14:04

imstar15 marked this pull request as ready for review October 26, 2023 14:27

Add comment for PalletXcmMigrateToV1

e489674

v9n approved these changes Oct 27, 2023

View reviewed changes

imstar15 merged commit 34ba6b4 into master Oct 27, 2023

imstar15 deleted the oak-polkadot-v0.9.43 branch October 27, 2023 07:37

imstar15 mentioned this pull request Nov 29, 2023

Update weight template #428

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update polkadot-v0.9.43 #456

Update polkadot-v0.9.43 #456

imstar15 commented Oct 18, 2023

chrisli30 left a comment

chrisli30 Oct 19, 2023

v9n Oct 26, 2023

imstar15 Oct 26, 2023 •

edited

Loading

imstar15 commented Oct 19, 2023 •

edited

Loading

imstar15 commented Oct 24, 2023 •

edited

Loading

v9n Oct 25, 2023

imstar15 Oct 25, 2023 •

edited

Loading

chrisli30 Oct 25, 2023 •

edited

Loading

v9n Oct 25, 2023

v9n Oct 26, 2023

imstar15 Oct 27, 2023 •

edited

Loading

v9n Oct 26, 2023

imstar15 Oct 27, 2023 •

edited

Loading

v9n Oct 26, 2023

v9n Oct 26, 2023

imstar15 Oct 27, 2023

v9n left a comment

v9n left a comment

Update polkadot-v0.9.43 #456

Update polkadot-v0.9.43 #456

Conversation

imstar15 commented Oct 18, 2023

chrisli30 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

imstar15 Oct 26, 2023 • edited Loading

Choose a reason for hiding this comment

imstar15 commented Oct 19, 2023 • edited Loading

imstar15 commented Oct 24, 2023 • edited Loading

Choose a reason for hiding this comment

imstar15 Oct 25, 2023 • edited Loading

Choose a reason for hiding this comment

chrisli30 Oct 25, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

imstar15 Oct 27, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

imstar15 Oct 27, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

v9n left a comment

Choose a reason for hiding this comment

v9n left a comment

Choose a reason for hiding this comment

imstar15 Oct 26, 2023 •

edited

Loading

imstar15 commented Oct 19, 2023 •

edited

Loading

imstar15 commented Oct 24, 2023 •

edited

Loading

imstar15 Oct 25, 2023 •

edited

Loading

chrisli30 Oct 25, 2023 •

edited

Loading

imstar15 Oct 27, 2023 •

edited

Loading

imstar15 Oct 27, 2023 •

edited

Loading