Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: ✨ Implement Forest root changes catch up in Blockchain Service #312

Merged
merged 47 commits into from
Jan 23, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
7e501b6
fix: :rotating_light: Remove error rust analyzer was complaining abou…
ffarall Jan 3, 2025
f5267b5
chore: :see_no_evil: Gitignore cSpell config file
ffarall Jan 6, 2025
af7b41b
feat: :construction: Emit events of mutations applied always
ffarall Jan 6, 2025
02880e6
feat: :construction: Half way implementing apply forest root changes
ffarall Jan 7, 2025
f3644e7
Merge branch 'main' into feat/forest-root-cath-ups-impl
ffarall Jan 9, 2025
67e175b
chore: :see_no_evil: Typo in cSpell gitignore
ffarall Jan 9, 2025
60048fc
fix: :rotating_light: Fix event fields after merge
ffarall Jan 9, 2025
bdedae5
feat: :construction: Implement applying `TrieMutation`s in reverse
ffarall Jan 9, 2025
1190017
feat: :construction: Implement forest root catch ups for BSPs
ffarall Jan 10, 2025
61ab757
chore: :label: Run `pnpm typegen`
ffarall Jan 10, 2025
4e1d9a2
fix: :white_check_mark: Use new `MutationsAppliedForProvider` event i…
ffarall Jan 10, 2025
c2db1de
fix: :bug: Create tree route with old and new best block in regular n…
ffarall Jan 10, 2025
1d038d0
feat: :loud_sound: Improve logs for apply forest root changes
ffarall Jan 10, 2025
3034eab
feat: :fire: Remove now unused code of applying forest root changes i…
ffarall Jan 10, 2025
3b52ab0
feat: :fire: Remove forest root changing from `bsp_charge_fees` and `…
ffarall Jan 10, 2025
c24fa3c
fix: :white_check_mark: Wait for new log from blockchain service to c…
ffarall Jan 10, 2025
891672c
fix: :white_check_mark: Fix race condition in onboarding by using `wa…
ffarall Jan 10, 2025
ab108f0
test: :white_check_mark: Cleanup reorgs test
ffarall Jan 10, 2025
57c1cde
style: :art: Apply cargo format
ffarall Jan 11, 2025
af21adf
feat: :loud_sound: Improve error logs for failed extrinsics in `sealB…
ffarall Jan 13, 2025
c512c8e
test: :bulb: Improve comment docs in reorg utilities
ffarall Jan 13, 2025
de2e8af
fix: :adhesive_bandage: Remove type parameter to please rust analyzer
ffarall Jan 13, 2025
fd220fb
feat: :sparkles: Add optional block sealing to `wait.bspStored`
ffarall Jan 13, 2025
32872b0
fix: :bug: Fix `Multiaddresses` type in `types-bundle`
ffarall Jan 13, 2025
a458cc0
fix: 🐛 Fix `BackupStorageProvider` type in `types-bundle`
ffarall Jan 13, 2025
be8da1b
chore: :label: Fix wrong `StorageDataType` in types-bundle
ffarall Jan 13, 2025
9dd2f5a
Merge branch 'main' into feat/forest-root-cath-ups-impl
ffarall Jan 13, 2025
abffd18
fix: :label: Fix `StorageDataType` type in `types-bundle`
ffarall Jan 13, 2025
d462936
test: :construction: Add tests for reorgs with forest root changes
ffarall Jan 13, 2025
0c35b0d
feat: :sparkles: Add optional `nonce` to `block.seal` testing functio…
ffarall Jan 14, 2025
b9a7371
test: :construction: Test failing in reorg with file deletion confirm…
ffarall Jan 15, 2025
226d61f
fix: :bug: Fix building of `tree_route` both for reorgs and new best …
ffarall Jan 16, 2025
90382b0
test: :white_check_mark: Test scenario with reorgs with changes retra…
ffarall Jan 16, 2025
ed774e1
fix: :adhesive_bandage: Fix wrong error logging when reverting change…
ffarall Jan 16, 2025
f7245b5
test: :white_check_mark: Remove `only` flag leftover
ffarall Jan 16, 2025
77da01f
Merge branch 'main' into feat/forest-root-cath-ups-impl
ffarall Jan 16, 2025
fdf92d8
chore: :label: Run typegen after merge
ffarall Jan 16, 2025
3f891fb
docs: :memo: Add test diagram for reorgs test
ffarall Jan 16, 2025
3723f6f
docs: :bulb: Update comments with suggested changes
ffarall Jan 23, 2025
0888316
Merge branch 'main' into feat/forest-root-cath-ups-impl
ffarall Jan 23, 2025
a842105
chore: :label: Update `api-augment` after merge
ffarall Jan 23, 2025
acbf708
Merge branch 'main' into feat/forest-root-cath-ups-impl
ffarall Jan 23, 2025
7a14948
chore: :label: Update `api-augment` after merge
ffarall Jan 23, 2025
5fe410b
fix: :rotating_light: Use new paramter in pallet params as a replacem…
ffarall Jan 23, 2025
1344e85
revert: :fire: Remove file that should have been ignored
ffarall Jan 23, 2025
ceb5416
refactor: :recycle: Merge `super` and `crate` imports
ffarall Jan 23, 2025
e072239
fix: :bug: Revert events order when reverting changes in a block
ffarall Jan 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ res/benchmarking
test/dist/*

# Ignore cSpell config files
cSpell.json
cspell.json

# Don't commit your config file
config.toml
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 7 additions & 1 deletion api-augment/dist/interfaces/lookup.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api-augment/dist/interfaces/lookup.js.map

Large diffs are not rendered by default.

27 changes: 24 additions & 3 deletions api-augment/dist/types/interfaces/augment-api-events.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2121,14 +2121,35 @@ declare module "@polkadot/api-base/types/events" {
}
>;
/**
* A set of mutations has been applied to the Forest.
* A set of mutations has been applied to a given Forest.
* This is the generic version of [`MutationsAppliedForProvider`](Event::MutationsAppliedForProvider)
* when [`generic_apply_delta`](ProofsDealerInterface::generic_apply_delta) is used
* and the root is not necessarily linked to a specific Provider.
**/
MutationsApplied: AugmentedEvent<
ApiType,
[provider: H256, mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>, newRoot: H256],
[mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>, oldRoot: H256, newRoot: H256],
{
provider: H256;
mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>;
oldRoot: H256;
newRoot: H256;
}
>;
/**
* A set of mutations has been applied to the Forest of a given Provider.
**/
MutationsAppliedForProvider: AugmentedEvent<
ApiType,
[
providerId: H256,
mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>,
oldRoot: H256,
newRoot: H256
],
{
providerId: H256;
mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>;
oldRoot: H256;
newRoot: H256;
}
>;
Expand Down
2 changes: 2 additions & 0 deletions api-augment/dist/types/interfaces/augment-types.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1348,6 +1348,7 @@ import type {
QueryProviderMultiaddressesError,
QueryStorageProviderCapacityError,
RandomnessOutput,
ReputationWeightType,
SaveFileToDisk,
ShouldRemoveFile,
StorageDataUnit,
Expand Down Expand Up @@ -2271,6 +2272,7 @@ declare module "@polkadot/types/types/registry" {
ReportedRoundStates: ReportedRoundStates;
Reporter: Reporter;
ReportIdOf: ReportIdOf;
ReputationWeightType: ReputationWeightType;
ReserveData: ReserveData;
ReserveIdentifier: ReserveIdentifier;
Response: Response;
Expand Down
8 changes: 7 additions & 1 deletion api-augment/dist/types/interfaces/lookup.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1668,9 +1668,15 @@ declare const _default: {
provider: string;
maybeProviderAccount: string;
};
MutationsAppliedForProvider: {
providerId: string;
mutations: string;
oldRoot: string;
newRoot: string;
};
MutationsApplied: {
provider: string;
mutations: string;
oldRoot: string;
newRoot: string;
};
ChallengesTickerSet: {
Expand Down
28 changes: 21 additions & 7 deletions api-augment/dist/types/interfaces/storagehubclient/types.d.ts
Original file line number Diff line number Diff line change
@@ -1,14 +1,26 @@
import type { Bytes, Enum, Null, Struct, U8aFixed, bool, u32, u64 } from "@polkadot/types-codec";
import type {
Bytes,
Enum,
Null,
Struct,
U8aFixed,
Vec,
bool,
u32,
u64
} from "@polkadot/types-codec";
import type { AccountId, BlockNumber, H256 } from "@polkadot/types/interfaces/runtime";
/** @name BackupStorageProvider */
export interface BackupStorageProvider extends Struct {
readonly capacity: StorageData;
readonly data_used: StorageData;
readonly multiaddresses: Bytes;
readonly capacity: StorageDataUnit;
readonly capacity_used: StorageDataUnit;
readonly multiaddresses: Multiaddresses;
readonly root: MerklePatriciaRoot;
readonly last_capacity_change: BlockNumber;
readonly owner_account: AccountId;
readonly payment_account: AccountId;
readonly reputation_weight: ReputationWeightType;
readonly sign_up_block: BlockNumber;
}
/** @name BackupStorageProviderId */
export interface BackupStorageProviderId extends H256 {}
Expand Down Expand Up @@ -126,7 +138,7 @@ export interface MainStorageProviderId extends H256 {}
/** @name MerklePatriciaRoot */
export interface MerklePatriciaRoot extends H256 {}
/** @name Multiaddresses */
export interface Multiaddresses extends Bytes {}
export interface Multiaddresses extends Vec<Bytes> {}
/** @name ProviderId */
export interface ProviderId extends H256 {}
/** @name QueryAvailableStorageCapacityError */
Expand Down Expand Up @@ -196,6 +208,8 @@ export interface QueryStorageProviderCapacityError extends Enum {
}
/** @name RandomnessOutput */
export interface RandomnessOutput extends H256 {}
/** @name ReputationWeightType */
export interface ReputationWeightType extends u32 {}
/** @name SaveFileToDisk */
export interface SaveFileToDisk extends Enum {
readonly isFileNotFound: boolean;
Expand All @@ -208,9 +222,9 @@ export interface SaveFileToDisk extends Enum {
/** @name ShouldRemoveFile */
export interface ShouldRemoveFile extends bool {}
/** @name StorageData */
export interface StorageData extends u32 {}
export interface StorageData extends u64 {}
/** @name StorageDataUnit */
export interface StorageDataUnit extends u32 {}
export interface StorageDataUnit extends u64 {}
/** @name StorageProviderId */
export interface StorageProviderId extends Enum {
readonly isBackupStorageProvider: boolean;
Expand Down
10 changes: 9 additions & 1 deletion api-augment/dist/types/interfaces/types-lookup.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2156,10 +2156,17 @@ declare module "@polkadot/types/lookup" {
readonly provider: H256;
readonly maybeProviderAccount: Option<AccountId32>;
} & Struct;
readonly isMutationsAppliedForProvider: boolean;
readonly asMutationsAppliedForProvider: {
readonly providerId: H256;
readonly mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>;
readonly oldRoot: H256;
readonly newRoot: H256;
} & Struct;
readonly isMutationsApplied: boolean;
readonly asMutationsApplied: {
readonly provider: H256;
readonly mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>;
readonly oldRoot: H256;
readonly newRoot: H256;
} & Struct;
readonly isChallengesTickerSet: boolean;
Expand All @@ -2174,6 +2181,7 @@ declare module "@polkadot/types/lookup" {
| "SlashableProvider"
| "NoRecordOfLastSubmittedProof"
| "NewChallengeCycleInitialised"
| "MutationsAppliedForProvider"
| "MutationsApplied"
| "ChallengesTickerSet";
}
Expand Down
27 changes: 24 additions & 3 deletions api-augment/src/interfaces/augment-api-events.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1643,12 +1643,33 @@ declare module "@polkadot/api-base/types/events" {
**/
ChallengesTickerSet: AugmentedEvent<ApiType, [paused: bool], { paused: bool }>;
/**
* A set of mutations has been applied to the Forest.
* A set of mutations has been applied to a given Forest.
* This is the generic version of [`MutationsAppliedForProvider`](Event::MutationsAppliedForProvider)
* when [`generic_apply_delta`](ProofsDealerInterface::generic_apply_delta) is used
* and the root is not necessarily linked to a specific Provider.
**/
MutationsApplied: AugmentedEvent<
ApiType,
[provider: H256, mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>, newRoot: H256],
{ provider: H256; mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>; newRoot: H256 }
[mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>, oldRoot: H256, newRoot: H256],
{ mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>; oldRoot: H256; newRoot: H256 }
>;
/**
* A set of mutations has been applied to the Forest of a given Provider.
**/
MutationsAppliedForProvider: AugmentedEvent<
ApiType,
[
providerId: H256,
mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>,
oldRoot: H256,
newRoot: H256
],
{
providerId: H256;
mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>;
oldRoot: H256;
newRoot: H256;
}
>;
/**
* A manual challenge was submitted.
Expand Down
2 changes: 2 additions & 0 deletions api-augment/src/interfaces/augment-types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1354,6 +1354,7 @@ import type {
QueryProviderMultiaddressesError,
QueryStorageProviderCapacityError,
RandomnessOutput,
ReputationWeightType,
SaveFileToDisk,
ShouldRemoveFile,
StorageDataUnit,
Expand Down Expand Up @@ -2278,6 +2279,7 @@ declare module "@polkadot/types/types/registry" {
ReportedRoundStates: ReportedRoundStates;
Reporter: Reporter;
ReportIdOf: ReportIdOf;
ReputationWeightType: ReputationWeightType;
ReserveData: ReserveData;
ReserveIdentifier: ReserveIdentifier;
Response: Response;
Expand Down
8 changes: 7 additions & 1 deletion api-augment/src/interfaces/lookup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1690,9 +1690,15 @@ export default {
provider: "H256",
maybeProviderAccount: "Option<AccountId32>"
},
MutationsAppliedForProvider: {
providerId: "H256",
mutations: "Vec<(H256,ShpTraitsTrieMutation)>",
oldRoot: "H256",
newRoot: "H256"
},
MutationsApplied: {
provider: "H256",
mutations: "Vec<(H256,ShpTraitsTrieMutation)>",
oldRoot: "H256",
newRoot: "H256"
},
ChallengesTickerSet: {
Expand Down
29 changes: 22 additions & 7 deletions api-augment/src/interfaces/storagehubclient/types.ts
Original file line number Diff line number Diff line change
@@ -1,18 +1,30 @@
// Auto-generated via `yarn polkadot-types-from-defs`, do not edit
/* eslint-disable */

import type { Bytes, Enum, Null, Struct, U8aFixed, bool, u32, u64 } from "@polkadot/types-codec";
import type {
Bytes,
Enum,
Null,
Struct,
U8aFixed,
Vec,
bool,
u32,
u64
} from "@polkadot/types-codec";
import type { AccountId, BlockNumber, H256 } from "@polkadot/types/interfaces/runtime";

/** @name BackupStorageProvider */
export interface BackupStorageProvider extends Struct {
readonly capacity: StorageData;
readonly data_used: StorageData;
readonly multiaddresses: Bytes;
readonly capacity: StorageDataUnit;
readonly capacity_used: StorageDataUnit;
readonly multiaddresses: Multiaddresses;
readonly root: MerklePatriciaRoot;
readonly last_capacity_change: BlockNumber;
readonly owner_account: AccountId;
readonly payment_account: AccountId;
readonly reputation_weight: ReputationWeightType;
readonly sign_up_block: BlockNumber;
}

/** @name BackupStorageProviderId */
Expand Down Expand Up @@ -149,7 +161,7 @@ export interface MainStorageProviderId extends H256 {}
export interface MerklePatriciaRoot extends H256 {}

/** @name Multiaddresses */
export interface Multiaddresses extends Bytes {}
export interface Multiaddresses extends Vec<Bytes> {}

/** @name ProviderId */
export interface ProviderId extends H256 {}
Expand Down Expand Up @@ -231,6 +243,9 @@ export interface QueryStorageProviderCapacityError extends Enum {
/** @name RandomnessOutput */
export interface RandomnessOutput extends H256 {}

/** @name ReputationWeightType */
export interface ReputationWeightType extends u32 {}

/** @name SaveFileToDisk */
export interface SaveFileToDisk extends Enum {
readonly isFileNotFound: boolean;
Expand All @@ -245,10 +260,10 @@ export interface SaveFileToDisk extends Enum {
export interface ShouldRemoveFile extends bool {}

/** @name StorageData */
export interface StorageData extends u32 {}
export interface StorageData extends u64 {}

/** @name StorageDataUnit */
export interface StorageDataUnit extends u32 {}
export interface StorageDataUnit extends u64 {}

/** @name StorageProviderId */
export interface StorageProviderId extends Enum {
Expand Down
10 changes: 9 additions & 1 deletion api-augment/src/interfaces/types-lookup.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2247,10 +2247,17 @@ declare module "@polkadot/types/lookup" {
readonly provider: H256;
readonly maybeProviderAccount: Option<AccountId32>;
} & Struct;
readonly isMutationsAppliedForProvider: boolean;
readonly asMutationsAppliedForProvider: {
readonly providerId: H256;
readonly mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>;
readonly oldRoot: H256;
readonly newRoot: H256;
} & Struct;
readonly isMutationsApplied: boolean;
readonly asMutationsApplied: {
readonly provider: H256;
readonly mutations: Vec<ITuple<[H256, ShpTraitsTrieMutation]>>;
readonly oldRoot: H256;
readonly newRoot: H256;
} & Struct;
readonly isChallengesTickerSet: boolean;
Expand All @@ -2265,6 +2272,7 @@ declare module "@polkadot/types/lookup" {
| "SlashableProvider"
| "NoRecordOfLastSubmittedProof"
| "NewChallengeCycleInitialised"
| "MutationsAppliedForProvider"
| "MutationsApplied"
| "ChallengesTickerSet";
}
Expand Down
2 changes: 1 addition & 1 deletion api-augment/storagehub.json

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions client/blockchain-service/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ sc-network = { workspace = true }
sc-utils = { workspace = true }
sc-tracing = { workspace = true }
sp-api = { workspace = true }
sp-blockchain = { workspace = true }
sp-core = { workspace = true, default-features = true }
sp-keystore = { workspace = true }
sp-runtime = { workspace = true, default-features = true }
Expand Down
4 changes: 2 additions & 2 deletions client/blockchain-service/src/commands.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ use super::{
handler::BlockchainService,
transaction::SubmittedTransaction,
types::{
BestBlockInfo, ConfirmStoringRequest, Extrinsic, ExtrinsicResult, RespondStorageRequest,
ConfirmStoringRequest, Extrinsic, ExtrinsicResult, MinimalBlockInfo, RespondStorageRequest,
RetryStrategy, StopStoringForInsolventUserRequest, SubmitProofRequest, Tip,
},
};
Expand All @@ -55,7 +55,7 @@ pub enum BlockchainServiceCommand {
callback: tokio::sync::oneshot::Sender<Result<()>>,
},
GetBestBlockInfo {
callback: tokio::sync::oneshot::Sender<BestBlockInfo>,
callback: tokio::sync::oneshot::Sender<MinimalBlockInfo>,
},
WaitForBlock {
block_number: BlockNumber,
Expand Down
Loading
Loading