This repository has been archived by the owner on Nov 15, 2023. It is now read-only.
Suggestions for #3413 (Store voters in unsorted bags companion) #3488
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* remove duplicate thiserror annotations * chore: cargo spellcheck
* Fix weights * try something to unbreak companion * update Substrate Co-authored-by: Andronik Ordian <write@reusable.software> Co-authored-by: parity-processbot <>
* add a from_backing_statement to SignedDisputeStatement * inform dispute coordinator of all backing statements * add dispute coordinator message to backing tests * send positive dispute statement with every approval * issue disputes when encountering invalid candidates. * try to fix flaky test for CI (passed locally) * guide: keep track of concluded-positive disputes until pruned * guide: block implications * guide: new dispute inherent flow * mostly implement recency changes for dispute coordinator * add a clock to dispute coordinator * adjust DB tests * fix and add new dispute coordinator tests * provisioner: select disputes * import all validators' approvals * address nit: refactor backing statement submission * gracefully handle disconnected dispute coordinator * remove `review` comment * fix up old_tests * fix approval-voting compilation * fix backing compilation * use known-leaves in WaitForActivation * follow-up test fixing * add back allow(dead_code)
* Move on to next validator after timeout. * Better naming. * Wrong implementation of validator fetch timeouts. * Validator side: Move on to next collator if download takes too long. * Drop multiple requests from same validator. * Add test that next response is sent after timeout. * Multiple requests by same validator should get dropped. * Test that another collator is tried after exclusive download time. * Add dep. * Cleanup. * Merge fix. * Review remarks. * Fixes. * Add log targets to trace logs Co-authored-by: Andronik Ordian <write@reusable.software>
* finer grained runtime-api caching * fix av-store * simplify a request * remove unused imports * fix tests
* disputes: Allow batch queries in dispute-coordinator This commit moves to batch queries when responding to QueryCandidateVotes messages. This simplifies the code in the provisioner and dispute-coordinator by no longer requiring to make use of a FuturesOrdered when awaiting multiple quries. Instead, the provisioner need only request the batch itself. * node/approval-voting: Address Feedback to fail on query element missing. * Address feedback * Fix implementer's guide
* Fix miner * fix miner * Update utils/staking-miner/src/monitor.rs * Fix width
* node/dispute-coordinator: Modify db to return SubsystemResult. In preparation of moving to the overlayed backend pattern, this commit moves the db to return SubsystemResult values. * node/dispute-coordinator: Add the Backend and OverlayedBackend. This commit adds the backend and overlayed backend structs to the dispute-coordinator subsystem. * node/dispute-coordinator: Implement backend and overlayed-backend. This commit finalizes the move from the previous transactional model to the common overlay pattern in subsystem persistency. This can be observed in the ApprovalVoting and ChainSelection subsystems. * Add module docs + license * Touchup merge
* Companion for substrate#9197 Signed-off-by: koushiro <koushiro.cqx@gmail.com> * update Substrate Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: parity-processbot <>
* adds test-runner boilerplate * revert to master * Update node/test/runtime/Cargo.toml Co-authored-by: Andronik Ordian <write@reusable.software> * fix warning * use polkadot_development_config * remove vestigial code * ... * remove unused dependencies * adds simnet binary * adds simnet binary * merged with remote * dummy to check pipeline * add 2 docker files and a build cmd * adds logging * atempt to use binary from build host * fix simnet-binary * fix docker commands * switch branches * ... * update docker file * update the dockerfile 2 * add some message in the cheatsheet * add repo to chaches stage also * update paths * do only 1 stage build * add time when build cmd started * remove debugg commands * polkadot-simnet-substrate-working-version-v1 * reduce size of polkadot-simnet image * update test runner api * update test-runner * ... * revert to master * Merge branch 'master' of github.com:paritytech/polkadot into substrate-test-runner * bump impl version * remove unused imports, fix test * was_binary.to_vec() * Apply suggestions from code review Co-authored-by: Andronik Ordian <write@reusable.software> * ... * remove unused import * remove unused import * adds post upgrade test * dry code * revert spec_version * update Cargo.lock * tested and it works * compare runtime spec version * fix spaces, remove docker files * replace spaces with tabs * Update runtime/polkadot/src/lib.rs Co-authored-by: Andronik Ordian <write@reusable.software> * ... * revert Cargo.lock * bump cargo.lock Co-authored-by: Andronik Ordian <write@reusable.software> Co-authored-by: radupopa2010 <radupopa2010@yahoo.com> Co-authored-by: CI system <>
Signed-off-by: koushiro <koushiro.cqx@gmail.com>
* CI: add spellcheck * revert me * CI: explicit command for spellchecker * spellcheck: edit misspells * CI: run spellcheck on diff * spellcheck: edits * spellcheck: edit misspells * spellcheck: add rules * spellcheck: mv configs * spellcheck: more edits * spellcheck: chore * spellcheck: one more thing * spellcheck: and another one * spellcheck: seems like it doesn't get to an end * spellcheck: new words after rebase * spellcheck: new words appearing out of nowhere * chore * review edits * more review edits * more edits * wonky behavior * wonky behavior 2 * wonky behavior 3 * change git behavior * spellcheck: another bunch of new edits * spellcheck: new words are koming out of nowhere * CI: finding the master * CI: fetching master implicitly * CI: undebug * new errors * a bunch of new edits * and some more * Update node/core/approval-voting/src/approval_db/v1/mod.rs Co-authored-by: Andronik Ordian <write@reusable.software> * Update xcm/xcm-executor/src/assets.rs Co-authored-by: Andronik Ordian <write@reusable.software> * Apply suggestions from code review Co-authored-by: Andronik Ordian <write@reusable.software> * Suggestions from the code review * CI: scan only changed files Co-authored-by: Andronik Ordian <write@reusable.software>
* ci: use chevdor/srtool-actions to build runtimes * cleanup
Co-authored-by: CI system <>
* Update MMR leaf. * Revert to older substrate. * Add version docs. * Fix spellcheck.
Bumps [async-process](https://github.com/smol-rs/async-process) from 1.0.1 to 1.1.0. - [Release notes](https://github.com/smol-rs/async-process/releases) - [Changelog](https://github.com/smol-rs/async-process/blob/master/CHANGELOG.md) - [Commits](smol-rs/async-process@v1.0.1...v1.1.0) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* disputes module skeleton and storage * implement dispute module initialization logic * implement disputes session change logic * provide dispute skeletons * deduplication & ancient check * fix a couple of warnings * begin provide_dispute_data impl * flesh out statement set import somewhat * move ApprovalVote to shared primitives * add a signing-payload API to explicit dispute statements * implement statement signature checking * some bitflags glue for observing changes in disputes * implement dispute vote import logic * flesh out everything except slashing * guide: tweaks * declare and use punishment trait * punish validators for inconclusive disputes * guide: tiny fix * guide: update docs * add disputes getter fn * guide: small change to spam slots handling * improve spam slots handling and fix some bugs * finish API of disputes runtime * define and deposit `RevertTo` log * begin integrating disputes into para_inherent * use precomputed slash_for/against * return candidate hash from process_bitfields * implement inclusion::collect_disputed * finish integration into rest of runtime * add Disputes to initializer * address suggestions * use pallet macro * fix typo * Update runtime/parachains/src/disputes.rs * add test: fix pruning * document specific behavior * deposit events on dispute changes * add an allow(unused) on fn disputes * add a dummy PunishValidators implementation * add disputes module to Rococo * add disputes module to westend runtime * add disputes module to test runtime * add disputes module to kusama runtime * guide: prepare for runtime API for checking frozenness * remove revert digests in favor of state variable * merge reversions * Update runtime/parachains/src/disputes.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * Update runtime/parachains/src/disputes.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * Update runtime/parachains/src/disputes.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> * add byzantine_threshold and supermajority_threshold utilities to primitives * use primitive helpers * deposit revert event when freezing chain * deposit revert log when freezing chain * test revert event and log are generated when freezing * add trait to decouple disputes handling from paras inherent handling * runtime: fix compilation and setup dispute handler * disputes: add hook for filtering out dispute statements * disputes: add initializer hooks to DisputesHandler * runtime: remove disputes pallet from all runtimes * tag TODOs * don't import any dispute statements just yet... * address grumbles * fix spellcheck, hopefully * maybe now? * last spellcheck round * fix runtime tests * fix test-runtime Co-authored-by: thiolliere <gui.thiolliere@gmail.com> Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> Co-authored-by: André Silva <andrerfosilva@gmail.com>
* feat: rustfmt.toml Copied from substrate. * avoid normalize It has some odd side effects converting // to /* */ instead of the other way round. See rust-lang/rustfmt#4909 .
Bumps [slotmap](https://github.com/orlp/slotmap) from 1.0.2 to 1.0.5. - [Release notes](https://github.com/orlp/slotmap/releases) - [Changelog](https://github.com/orlp/slotmap/blob/master/RELEASES.md) - [Commits](orlp/slotmap@v1.0.2...v1.0.5) --- updated-dependencies: - dependency-name: slotmap dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.91 to 0.2.98. - [Release notes](https://github.com/rust-lang/libc/releases) - [Commits](rust-lang/libc@0.2.91...0.2.98) --- updated-dependencies: - dependency-name: libc dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
kianenigma
reviewed
Jul 20, 2021
kianenigma
reviewed
Jul 20, 2021
* Update secp256k1 and remove unrequired usage * Rename missed old crate names * Enable required feature
* fix * update
Bumps [indexmap](https://github.com/bluss/indexmap) from 1.6.1 to 1.7.0. - [Release notes](https://github.com/bluss/indexmap/releases) - [Commits](indexmap-rs/indexmap@1.6.1...1.7.0) --- updated-dependencies: - dependency-name: indexmap dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* guide: filter_multi_dispute_data * guide: elaborate * Implementation of dispute data filtering * tests for filtering * don't use std, you fool! * use swap_remove * Update runtime/parachains/src/disputes.rs Co-authored-by: Andronik Ordian <write@reusable.software> * use btreeste * address API nit Co-authored-by: Andronik Ordian <write@reusable.software>
From the GitHub docs: > # The `docs/*` pattern will match files like > # `docs/getting-started.md` but not further nested files like > # `docs/build-app/troubleshooting.md`. > docs/* docs@example.com > > # In this example, @octocat owns any file in an apps directory > # anywhere in your repository. > apps/ @octocat
* node/dispute-coordinator: Introduce resume capability This commit introduces a resume capability for the dispute coordinator subsystem. Specifically, this will allow to recover data for disputes for which we have no local statements. * node/dispute-coordinator: Add resume function to TestState and modify Harness This commit modifies the TestHarness to return a TestState. We subsequently define a resume function on TestState that allows to interrupt the test and test specifically for behavior on startup of the subsystem. * node/dispute-coordinator: Implement resume functionality This commit implements the resume functionality for the subsystem. In addition, we will forward any DisputeParticipation::Participate message in order to ensure that disputes for which we do not have local statements may be recovered in due time. * Address Feedback * Modify to run handle_leaf on first import * Modify missing_local_statement logic * node/dispute-coordinator: Add simple test to ensure we adequately handle local_statements that are not missing. * Add missing keystore tests
* frame v2 * migrate runtimes * migrate more * comments * docs * cleanup * Adjust visibility on storage items Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
* migration * fix warning * change runtimes * extra line * another double space lol * add dmp in test runtime * test runtime * Adjust visibility on storage items Co-authored-by: Keith Yeung <kungfukeith11@gmail.com>
* improve test * better doc
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
ghost
pushed a commit
that referenced
this pull request
Sep 17, 2021
…on properties (#3413) * add voter bags generation script and generated values for all runtimes * VoterBagThresholds for test-runtime * semicolon * turns out the header is called file_header.txt * perform benchmarks to get real numbers * add note about when to re-run this script * Suggestions for #3413 (Store voters in unsorted bags companion) (#3488) Co-authored-by: Andronik Ordian <write@reusable.software> Co-authored-by: parity-processbot <> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: radupopa2010 <radupopa2010@yahoo.com> Co-authored-by: CI system <> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> Co-authored-by: thiolliere <gui.thiolliere@gmail.com> Co-authored-by: André Silva <andrerfosilva@gmail.com> Co-authored-by: Keith Yeung <kungfukeith11@gmail.com> Co-authored-by: Bernhard Schuster <bernhard@ahoi.io> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com> Co-authored-by: Alexander Theißen <alex.theissen@me.com> Co-authored-by: Lldenaurois <ljdenaurois@gmail.com> Co-authored-by: Qinxuan Chen <koushiro.cqx@gmail.com> Co-authored-by: Seun Lanlege <seun@parity.io> Co-authored-by: Denis Pisarev <denis.pisarev@parity.io> Co-authored-by: Andreas Doerr <adoerr@users.noreply.github.com> Co-authored-by: Chevdor <chevdor@users.noreply.github.com> Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com> Co-authored-by: Denis Tsai <denistsai@aetheras.io> Co-authored-by: Pierre Besson <pierre.besson@parity.io> Co-authored-by: kianenigma <kian@parity.io> Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> Co-authored-by: Xiliang Chen <xlchen1291@gmail.com> Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com> Co-authored-by: ferrell-code <automatedcharlesferrell@gmail.com> Co-authored-by: Shaun W <spxwang@gmail.com> * Try resolve Cargo.lock conflicts * Add temp VoterSnapshotPerBlock value to compile * Improve voter_bags test with nominator/validator breakdown * Remove miner.log * Update README * Module doc comments * Build working * remote-ext-tests very WIP * Remove system module query * formatting and log target * clean up * Save * Fix issues with polkadot runtime after mergin * Save * Some updates to runtimes to make sure things compile * save * udpate weights * Use UseNominatorsAndUpdateBagsList * make remote-ext tests a bin * Adjust remote-ext test to use pub; update some comments * Use list_bags_get * Try fix cargo.lock * Gate kusama feat code * Carg.lock .. again * point bags-list deps to master * Update voter-bags CLI to take total issuance and min balance as args * Switch polkadot runtime to UseNominatorMap * Some tweaks to the remote-test * Delete some stuff for polkadot; comments * Various fixes * Add comment for VoterSnapshotPerBlock * Remove bags-list import from polakdot cargo * use benchmarking feature for pallet-bags-list * Add custom migration for westend and kusama for bags-list * :facepalm * Fix kusama runtime * Small clean up * Be more alphabetical ordered * Use 22_500 for VoterSnapshotPerBlock * point back to master * remove unused type * SortedListProvider for staking runtime * Remove polkadot for remote-ext bags test * Revert "Remove polkadot for remote-ext bags test" This reverts commit 3608c46. * Remove polkadot for remote-ext bags test * update Substrate Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com> Co-authored-by: Andronik Ordian <write@reusable.software> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: radupopa2010 <radupopa2010@yahoo.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> Co-authored-by: thiolliere <gui.thiolliere@gmail.com> Co-authored-by: André Silva <andrerfosilva@gmail.com> Co-authored-by: Keith Yeung <kungfukeith11@gmail.com> Co-authored-by: Bernhard Schuster <bernhard@ahoi.io> Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Co-authored-by: Robert Habermeier <rphmeier@gmail.com> Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com> Co-authored-by: Alexander Theißen <alex.theissen@me.com> Co-authored-by: Lldenaurois <ljdenaurois@gmail.com> Co-authored-by: Qinxuan Chen <koushiro.cqx@gmail.com> Co-authored-by: Seun Lanlege <seun@parity.io> Co-authored-by: Denis Pisarev <denis.pisarev@parity.io> Co-authored-by: Andreas Doerr <adoerr@users.noreply.github.com> Co-authored-by: Chevdor <chevdor@users.noreply.github.com> Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com> Co-authored-by: Denis Tsai <denistsai@aetheras.io> Co-authored-by: Pierre Besson <pierre.besson@parity.io> Co-authored-by: kianenigma <kian@parity.io> Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> Co-authored-by: Xiliang Chen <xlchen1291@gmail.com> Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com> Co-authored-by: ferrell-code <automatedcharlesferrell@gmail.com> Co-authored-by: Shaun W <spxwang@gmail.com> Co-authored-by: parity-processbot <>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
target branch: prgn-companion-for-9081
target pr: #3413
cc @kianenigma
This introduces some feature gated integrated tests for the bags migration using
remote-extension.Example for running feature gated tests:
To run the polkadot tests
Note:
--nocaptureis helpful because we print out useful metrics like voter distribution among bagsFollow up work / TODO
get_npos_voters