[Feature] Part 1: add TargetList for validator ranking #12034

ruseinov · 2022-08-15T09:29:32Z

This is the first of a series of PRs aimed at reviving #11013

Since we've introduced 2 instances of bagslist effectively overriding VoterList with an Instance1 - does that mean we have to migrate storage to make sure the list contains what it needs to contain?
Gotta figure out where bag thresholds are being generated and make sure we generate balances? Or just reuse the same thresholds
Why reverse?

This seems to introduce all the moving parts without actually touching any of the current business-logic.

polkadot companion: paritytech/polkadot#5930

paritytech-cicd-pr · 2022-08-15T09:30:55Z

The CI pipeline was cancelled due to failure one of the required jobs.
The job name - cargo-check-benches
The job logs - https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1744862

paritytech-cicd-pr · 2022-08-15T09:38:13Z

The CI pipeline was cancelled due to failure one of the required jobs.
The job name - cargo-check-benches
The job logs - https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1744978

paritytech-cicd-pr · 2022-08-15T09:40:49Z

The CI pipeline was cancelled due to failure one of the required jobs.
The job name - cargo-check-benches
The job logs - https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1745014

paritytech-cicd-pr · 2022-08-15T16:37:17Z

The CI pipeline was cancelled due to failure one of the required jobs.
The job name - cargo-check-benches
The job logs - https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1746014

paritytech-cicd-pr · 2022-08-15T17:47:10Z

The CI pipeline was cancelled due to failure one of the required jobs.
The job name - cargo-check-benches
The job logs - https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1746316

paritytech-cicd-pr · 2022-08-15T17:47:16Z

The CI pipeline was cancelled due to failure one of the required jobs.
The job name - cargo-check-benches
The job logs - https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1746315

paritytech-cicd-pr · 2022-08-15T19:32:06Z

The CI pipeline was cancelled due to failure one of the required jobs.
The job name - cargo-check-benches
The job logs - https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1746433

frame/staking/src/pallet/mod.rs

kianenigma · 2022-08-16T05:31:00Z

Since we've introduced 2 instances of bagslist effectively overriding VoterList with an Instance1 - does that mean we have to migrate storage to make sure the list contains what it needs to contain?

We sure do, but let's leave that for the end of the work -- for now don't bother with it.

Gotta figure out where bag thresholds are being generated and make sure we generate balances? Or just reuse the same thresholds

utils/frame/generate-bags

Why reverse?

This Compat type is there only to make the tests pass. If you remove it and just use type TargetList = TargetBagsList, you will see that a gzillion tests will fail. You can and should double check but I think this is only because the order of targets that are returned differs.

kianenigma · 2022-08-16T06:00:57Z

frame/staking/src/lib.rs

@@ -685,6 +685,16 @@ pub struct Nominations<T: Config> {
 	pub suppressed: bool,
 }

+/// An unbounded version of `Nominations`, use for some really wacky hacks.
+#[derive(PartialEqNoBound, EqNoBound, Clone, Encode, Decode, RuntimeDebugNoBound, TypeInfo)]


This is not needed for this PR.

frame/staking/src/pallet/impls.rs

bin/node/runtime/src/lib.rs

paritytech-cicd-pr · 2022-08-16T07:33:30Z

The CI pipeline was cancelled due to failure one of the required jobs.
The job name - cargo-check-benches
The job logs - https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1748008

paritytech-cicd-pr · 2022-08-16T07:48:51Z

The CI pipeline was cancelled due to failure one of the required jobs.
The job name - cargo-check-benches
The job logs - https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1748045

paritytech-cicd-pr · 2022-08-16T07:49:00Z

The CI pipeline was cancelled due to failure one of the required jobs.
The job name - cargo-check-benches
The job logs - https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1748044

paritytech-cicd-pr · 2022-08-16T07:55:21Z

The CI pipeline was cancelled due to failure one of the required jobs.
The job name - cargo-check-benches
The job logs - https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/1748082

frame/staking/src/pallet/impls.rs

gilescope · 2022-09-14T10:03:02Z

frame/staking/src/pallet/impls.rs

+
+		let mut targets_iter = T::TargetList::iter();
+		while all_targets.len() < max_allowed_len &&
+			targets_seen < (NPOS_MAX_ITERATIONS_COEFFICIENT * max_allowed_len as u32)


would it be worth doing .try_from rather than as and then .unwrap_or(u32::MAX)?

Sounds like a good idea!

I guess same as below - we will see what's possible in P2, the staking code is full of as, which is not necessarily an excuse, but if this is checked elsewhere - no need to overcomplicate the code just yet.

Thanks so much for the review and those nits, it was helpful for me to go over the codebase again to see if we can indeed assume that u32 never overflows here.

gilescope · 2022-09-14T10:14:05Z

frame/staking/src/pallet/impls.rs


-		Self::register_weight(T::WeightInfo::get_npos_targets(validator_count));
+		Self::register_weight(T::WeightInfo::get_npos_targets(all_targets.len() as u32));


again here maybe we want to saturate with a try_into?

It's a good spot, but at the moment it seems unnecessary to complicate things.
We will revisit this upon introduction of the P2.

gilescope

The staking code is pretty new to me but aside from a couple of nits looks good.

frame/staking/src/pallet/impls.rs

Co-authored-by: Squirrel <[email protected]>

frame/staking/src/pallet/impls.rs

ruseinov · 2022-09-18T09:03:17Z

bot merge

paritytech-processbot · 2022-09-18T09:03:24Z

Error: "Check reviews" status is not passing for paritytech/polkadot#5930

ggwpez · 2022-09-19T12:20:19Z

frame/staking/src/migrations.rs

+		for MigrateToV11<T, P, N>
+	{
+		#[cfg(feature = "try-runtime")]
+		fn pre_upgrade() -> Result<(), &'static str> {


Should not compile.

#### Intro Upgrade polkadot from v0.9.27 to v0.9.30 as a checkpoint, then to v0.9.33 (latest version without workspace dependencies) Require ComposableFi/composable-ibc#176 to be merged first. #### Migrations - v0.9.28 - [x] paritytech/polkadot#5582 - Nomination not present in our runtimes - v0.9.29 - [x] paritytech/substrate#12095 - Nomination not present in our runtimes - v0.9.30 - [x] paritytech/substrate#12034 - BagList/Staking not present in our runtimes - [x] paritytech/polkadot#5930 - Nomination/BagList/Staking not present in our runtimes - [x] paritytech/substrate#12230 - Staking not present in our runtimes - [x] paritytech/polkadot#5996 - Staking not present in our runtimes - [x] paritytech/substrate#12083 - Contracts not present in our runtimes Signed-off-by: cor <[email protected]> Co-authored-by: cor <[email protected]>

) * [Feature] Part 1: add TargetList for validator ranking * remove redundant todo * remove typo * cleanup * implement score * more fixes * fix thresholds * fmt * Remove the stuff that has to come in the next PR, some fixes * extended balance import * Change all the references from VoteWeight to Self::Score * Add a migration for VoterBagsList * fix score * add targetList to nomination-pools tests * fix bench * address review comments * change get_npos_targets * address more comments * remove thresholds for the time being * fix instance reference * VoterBagsListInstance * reus * remove params that are not used yet * Introduced pre/post upgrade try-runtime checks * fix * fixes * fix migration * fix migration * fix post_upgrade * change * Fix * eloquent PhantomData * fix PD * more fixes * Update frame/staking/src/pallet/impls.rs Co-authored-by: Squirrel <[email protected]> * is_nominator now works * fix test-staking * build fixes * fix remote-tests * Apply suggestions from code review Co-authored-by: parity-processbot <> Co-authored-by: kianenigma <[email protected]> Co-authored-by: Squirrel <[email protected]> Co-authored-by: Kian Paimani <[email protected]>

ruseinov added 2 commits August 15, 2022 11:18

[Feature] Part 1: add TargetList for validator ranking

64b5c6e

remove redundant todo

a6b07a5

ruseinov requested review from kianenigma and andresilva as code owners August 15, 2022 09:29

github-actions bot added the A0-please_review Pull request needs code review. label Aug 15, 2022

remove typo

ec61b29

cleanup

281e5d0

implement score

73b47cb

more fixes

76034bb

fix thresholds

86631ed

kianenigma reviewed Aug 16, 2022

View reviewed changes

frame/staking/src/pallet/mod.rs Show resolved Hide resolved

kianenigma reviewed Aug 16, 2022

View reviewed changes

frame/staking/src/pallet/impls.rs Show resolved Hide resolved

kianenigma reviewed Aug 16, 2022

View reviewed changes

bin/node/runtime/src/lib.rs Outdated Show resolved Hide resolved

fmt

3a82504

Remove the stuff that has to come in the next PR, some fixes

74b6fba

extended balance import

8a2fc42

Change all the references from VoteWeight to Self::Score

e676398

ruseinov added 2 commits September 14, 2022 10:40

fix PD

94efa6e

more fixes

c1547a2

gilescope reviewed Sep 14, 2022

View reviewed changes

frame/staking/src/pallet/impls.rs Outdated Show resolved Hide resolved

gilescope reviewed Sep 14, 2022

View reviewed changes

gilescope approved these changes Sep 14, 2022

View reviewed changes

kianenigma reviewed Sep 14, 2022

View reviewed changes

frame/staking/src/pallet/impls.rs Outdated Show resolved Hide resolved

ruseinov and others added 5 commits September 16, 2022 10:52

Update frame/staking/src/pallet/impls.rs

194e0e2

Co-authored-by: Squirrel <[email protected]>

is_nominator now works

ed98cca

fix test-staking

ba0c063

build fixes

e6a3871

fix remote-tests

d019f1d

kianenigma reviewed Sep 18, 2022

View reviewed changes

frame/staking/src/pallet/impls.rs Outdated Show resolved Hide resolved

kianenigma reviewed Sep 18, 2022

View reviewed changes

frame/staking/src/pallet/impls.rs Outdated Show resolved Hide resolved

Apply suggestions from code review

93491e2

kianenigma added A8-mergeoncegreen and removed A0-please_review Pull request needs code review. labels Sep 18, 2022

ruseinov merged commit 03eb807 into master Sep 18, 2022

ruseinov deleted the ru/feature/p1-target-list branch September 18, 2022 09:28

ggwpez reviewed Sep 19, 2022

View reviewed changes

4meta5 mentioned this pull request Oct 17, 2022

Update substrate/polkadot from v0.9.29 to v0.9.30 moonbeam-foundation/moonbeam#1874

Closed

1 task

github-actions bot mentioned this pull request Nov 10, 2022

Update substrate/polkadot/cumulus from v0.9.29 to v0.9.31 moonbeam-foundation/moonbeam#1942

Closed

ruseinov mentioned this pull request Nov 16, 2022

[Staking] Approval Stake tracking paritytech/polkadot-sdk#443

Open

github-actions bot mentioned this pull request Nov 23, 2022

Update substrate/polkadot/cumulus from v0.9.29 to v0.9.32 moonbeam-foundation/moonbeam#1965

Closed

hussein-aitlahcen mentioned this pull request Jan 16, 2023

[PICA-34] - Upgrade to polkadot v0.9.30 ComposableFi/composable#2913

Merged

7 tasks

vovac12 mentioned this pull request Feb 28, 2023

Prepare migrations for 1.9.0 sora-xor/sora2-network#294

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Part 1: add TargetList for validator ranking #12034

[Feature] Part 1: add TargetList for validator ranking #12034

ruseinov commented Aug 15, 2022 •

edited

Loading

paritytech-cicd-pr commented Aug 15, 2022

paritytech-cicd-pr commented Aug 15, 2022

paritytech-cicd-pr commented Aug 15, 2022

paritytech-cicd-pr commented Aug 15, 2022

paritytech-cicd-pr commented Aug 15, 2022

paritytech-cicd-pr commented Aug 15, 2022

paritytech-cicd-pr commented Aug 15, 2022

kianenigma commented Aug 16, 2022

kianenigma Aug 16, 2022

paritytech-cicd-pr commented Aug 16, 2022

paritytech-cicd-pr commented Aug 16, 2022

paritytech-cicd-pr commented Aug 16, 2022

paritytech-cicd-pr commented Aug 16, 2022

gilescope Sep 14, 2022

ruseinov Sep 16, 2022

ruseinov Sep 16, 2022

gilescope Sep 14, 2022

ruseinov Sep 16, 2022

gilescope left a comment

ruseinov commented Sep 18, 2022

paritytech-processbot bot commented Sep 18, 2022

ggwpez Sep 19, 2022


		Self::register_weight(T::WeightInfo::get_npos_targets(validator_count));
		Self::register_weight(T::WeightInfo::get_npos_targets(all_targets.len() as u32));

[Feature] Part 1: add TargetList for validator ranking #12034

[Feature] Part 1: add TargetList for validator ranking #12034

Conversation

ruseinov commented Aug 15, 2022 • edited Loading

paritytech-cicd-pr commented Aug 15, 2022

paritytech-cicd-pr commented Aug 15, 2022

paritytech-cicd-pr commented Aug 15, 2022

paritytech-cicd-pr commented Aug 15, 2022

paritytech-cicd-pr commented Aug 15, 2022

paritytech-cicd-pr commented Aug 15, 2022

paritytech-cicd-pr commented Aug 15, 2022

kianenigma commented Aug 16, 2022

kianenigma Aug 16, 2022

Choose a reason for hiding this comment

paritytech-cicd-pr commented Aug 16, 2022

paritytech-cicd-pr commented Aug 16, 2022

paritytech-cicd-pr commented Aug 16, 2022

paritytech-cicd-pr commented Aug 16, 2022

gilescope Sep 14, 2022

Choose a reason for hiding this comment

ruseinov Sep 16, 2022

Choose a reason for hiding this comment

ruseinov Sep 16, 2022

Choose a reason for hiding this comment

gilescope Sep 14, 2022

Choose a reason for hiding this comment

ruseinov Sep 16, 2022

Choose a reason for hiding this comment

gilescope left a comment

Choose a reason for hiding this comment

ruseinov commented Sep 18, 2022

paritytech-processbot bot commented Sep 18, 2022

ggwpez Sep 19, 2022

Choose a reason for hiding this comment

ruseinov commented Aug 15, 2022 •

edited

Loading