Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Commit

Permalink
Use Message Queue pallet for UMP dispatch (#6271)
Browse files Browse the repository at this point in the history
* Add ProcessXcmMessage struct

* Migrate away from weights in host config

* New well-known key to report UMPQ capacity

* Add missing file

* Fixes

* Remove original UMP files

* Docs

* Update runtime/parachains/src/inclusion/mod.rs

Co-authored-by: asynchronous rob <[email protected]>

* Add benchmarking

* Benchmarks

* Mock example of using the QueueChangeHandler to update the WKK

* Use master Cargo.lock

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Merge remote-tracking branch 'origin/master' into gav-message-queue

* Update Cargo.lock

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update remove-weight migration

The migration got touched on master; just resolving conflicts here.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add message- to dispatch-origin conversion for XCM processing

Just using the `impl Into<MultiLocation>` was a bit inflexible.
Like this, the Relaychain can convert `UMP(para)` to a MultiLocation `para`.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* DNM: Temporarily comment code since XCMv3 is not merged yet

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use u64 for queue-wide limits on UmpAcceptanceCheckErr

Using u32 here was one audit finding for the queue pallet.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Define one sub-queue per *MP queue

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Harden check_upward_messages

Using safe math and casts.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add type-safe well_known_keys

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add message-queue weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Deploy MessageQueue to Polkadot

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update Cargo.toml

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Migrate to parachain config V5

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update UMP tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert messed up merge 🤦

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update remove-weight migration

The migration got touched on master; just resolving conflicts here.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add message- to dispatch-origin conversion for XCM processing

Just using the `impl Into<MultiLocation>` was a bit inflexible.
Like this, the Relaychain can convert `UMP(para)` to a MultiLocation `para`.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* DNM: Temporarily comment code since XCMv3 is not merged yet

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use u64 for queue-wide limits on UmpAcceptanceCheckErr

Using u32 here was one audit finding for the queue pallet.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Define one sub-queue per *MP queue

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Harden check_upward_messages

Using safe math and casts.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add type-safe well_known_keys

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add message-queue weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Deploy MessageQueue to Polkadot

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update Cargo.toml

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Migrate to parachain config V5

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update UMP tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Make stuff compile

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Move DMP and HRMP messages to the MessageQueue

It currently does not compile in the CIbecause of some local
tweaks to Substrate.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Diener for CI

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* diener update cargo.lock

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert wrong changes

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "DNM: Temporarily comment code since XCMv3 is not merged yet"

This reverts commit 130a48a.

* Make compile

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fixup runtimes

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Define benchmarks

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix migration

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use master Cargo.lock

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Lockfile

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add AggregateMessageOrigin

This enum currently only holds one value, but having it will make
it easier in the future to extend.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Forbid UMP for off-boarding paras

- Reject candidates with UMP messages for off-boarding paras
- Forbid scheduling off-boarding when a para has unprocess UMPs

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Delete stupid test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use BoundedVec for upward messages

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add weights and fix MessageProcessor

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bound receive_upward_messages and check bound in configuration pallet

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bound Debug impl

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* clippy

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix test runtime

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix xcm-simulator

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Properly fix xcm-simulator and fuzzer

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* cargo update -p sp-io

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Adapt to upstream Substrate changes

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix ProcesseMessage impls

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Some tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use master Cargo.lock

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* cargo update -p sp-io

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use new MQ API

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix migration

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update Cargo.lock

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add UMP while Para offboarding tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use Mocked message processor for benchmarking

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use variables for constants

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add MQ pallet weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use MQ pallet weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Configure QueueChangeHandler

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add config test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix MQ serive weight

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Cleanup

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Cleanup outgoing UMP dispatch queues

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use Master Cargo.lock

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update Cargo.lock

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Weight mul is not const

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Clippy

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove merge marker

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update runtime/parachains/src/inclusion/mod.rs

Co-authored-by: Gavin Wood <[email protected]>

* Update runtime/kusama/src/lib.rs

Co-authored-by: Gavin Wood <[email protected]>

* Use lowercase UMP

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Clarify comment

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use Weight::from_parts

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix doc

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Emit event after the fact

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add defensive_proof to receive_upward_messages

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Reapply "Remove original UMP files"

Looks like they came back from the dead. Re-apply commit 34b8a40

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove old files

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Rename MaxUmpMessageLen -> MaxUmpMessageLenOf

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Test defensive message dropping of receive_upward_messages

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fixup imports

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update implementors guide

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove FAIL-CI mark

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Delete unused code

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add another test for MQ change hook

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Imports

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Keep Kusama runtime formatting

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Delete unused code"

This reverts commit dd76bca.

* Feature gate mock functions

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Review: Use saturating_add

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Test RelayDispatchQueueSize storage key

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Move migration to own file to avoid merge conflicts

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Migration in own file

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fixup migration

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Rococo: configure MQ pallet

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fixup tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Rococo: configure MQ pallet"

Going to do this as follow up, since it needs Substrate changes
and i dont want to stall this MR any longer.

This reverts commit b9c15e8.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Fixup tests"

This reverts commit 88f1cbe.

* Fixup migration

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix migration

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix CI

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix migration

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix other migration

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bump MAX_CODE_SIZE to 10MiB

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add ForceUpdateUmpLimits migration

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* clippy

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* clippy

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* imports

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use defensive instead of defensive_proof

'defensive_proof' also prints the 'self', which spams the console
too much when running the tests. Just the length is enough.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Rename to ScheduleConfigUpdate

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fixup migration checks

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add MAX_CODE_SIZE to ScheduleConfigUpdate

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Set MAX_CODE_SIZE to 4MiB

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix benchmark

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix formatting

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Add MAX_CODE_SIZE to ScheduleConfigUpdate"

This reverts commit 7caffb0.

* Revert "Set MAX_CODE_SIZE to 4MiB"

This reverts commit 103ffba.

* Revert "Bump MAX_CODE_SIZE to 10MiB"

This reverts commit 530734b.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove consistency check from migration

Re-addig these checks is blocked on #7108

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix constants

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bump MAX_UPWARD_MESSAGE_SIZE_BOUND for Westend

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix migrations

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use old nightly for fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fixes

* cargo fmt

* Fix tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fixes

* Add MQ pallet to fuzzer

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix XMC simulator example

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove runtime-benchmarks from fuzzers

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Remove runtime-benchmarks from fuzzers"

This reverts commit e1f2bb0.

* Fix example simulator

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add V6 migration and remove old ones

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Actually make old migrations reusable

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Dont delete old migrations

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Future proof AggregateMessageOrigin and review fixes

There are indications that Loopback and Bridged will be needed soon.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* More cleanup

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fix benchmarks

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix fuzzer build

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Review

Co-authored-by: muharem <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove old migration

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Set MQ service weight to 20%

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix tabs in Markdown

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: asynchronous rob <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Keith Yeung <[email protected]>
  • Loading branch information
4 people authored May 19, 2023
1 parent 29a4fb1 commit 7c1dc94
Show file tree
Hide file tree
Showing 74 changed files with 4,033 additions and 2,535 deletions.
29 changes: 29 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 5 additions & 5 deletions node/network/approval-distribution/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1270,11 +1270,11 @@ impl State {
let candidate_entry = match block_entry.candidates.get(index as usize) {
None => {
gum::debug!(
target: LOG_TARGET,
?hash,
?index,
"`get_approval_signatures`: could not find candidate entry for given hash and index!"
);
target: LOG_TARGET,
?hash,
?index,
"`get_approval_signatures`: could not find candidate entry for given hash and index!"
);
continue
},
Some(e) => e,
Expand Down
2 changes: 0 additions & 2 deletions node/service/src/chain_spec.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@
//! Polkadot chain configurations.

use beefy_primitives::crypto::AuthorityId as BeefyId;
use frame_support::weights::Weight;
use grandpa::AuthorityId as GrandpaId;
#[cfg(feature = "kusama-native")]
use kusama_runtime as kusama;
Expand Down Expand Up @@ -189,7 +188,6 @@ fn default_parachains_host_configuration(
max_upward_queue_count: 8,
max_upward_queue_size: 1024 * 1024,
max_downward_message_size: 1024 * 1024,
ump_service_total_weight: Weight::from_parts(100_000_000_000, MAX_POV_SIZE as u64),
max_upward_message_size: 50 * 1024,
max_upward_message_num_per_candidate: 5,
hrmp_sender_deposit: 0,
Expand Down
57 changes: 56 additions & 1 deletion primitives/src/v4/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ pub type ValidatorSignature = validator_app::Signature;

/// A declarations of storage keys where an external observer can find some interesting data.
pub mod well_known_keys {
use super::{HrmpChannelId, Id};
use super::{HrmpChannelId, Id, WellKnownKey};
use hex_literal::hex;
use parity_scale_codec::Encode as _;
use sp_io::hashing::twox_64;
Expand Down Expand Up @@ -210,6 +210,7 @@ pub mod well_known_keys {
///
/// - `count: u32`, the number of messages currently in the queue for given para,
/// - `total_size: u32`, the total size of all messages in the queue.
#[deprecated = "Use `relay_dispatch_queue_remaining_capacity` instead"]
pub fn relay_dispatch_queue_size(para_id: Id) -> Vec<u8> {
let prefix = hex!["f5207f03cfdce586301014700e2c2593fad157e461d71fd4c1f936839a5f1f3e"];

Expand All @@ -224,6 +225,24 @@ pub mod well_known_keys {
})
}

/// Type safe version of `relay_dispatch_queue_size`.
#[deprecated = "Use `relay_dispatch_queue_remaining_capacity` instead"]
pub fn relay_dispatch_queue_size_typed(para: Id) -> WellKnownKey<(u32, u32)> {
#[allow(deprecated)]
relay_dispatch_queue_size(para).into()
}

/// The upward message dispatch queue remaining capacity for the given para id.
///
/// The storage entry stores a tuple of two values:
///
/// - `count: u32`, the number of additional messages which may be enqueued for the given para,
/// - `total_size: u32`, the total size of additional messages which may be enqueued for the
/// given para.
pub fn relay_dispatch_queue_remaining_capacity(para_id: Id) -> WellKnownKey<(u32, u32)> {
(b":relay_dispatch_queue_remaining_capacity", para_id).encode().into()
}

/// The HRMP channel for the given identifier.
///
/// The storage entry should be accessed as an `AbridgedHrmpChannel` encoded value.
Expand Down Expand Up @@ -1706,6 +1725,42 @@ impl PvfCheckStatement {
}
}

/// A well-known and typed storage key.
///
/// Allows for type-safe access to raw well-known storage keys.
pub struct WellKnownKey<T> {
/// The raw storage key.
pub key: Vec<u8>,
_p: sp_std::marker::PhantomData<T>,
}

impl<T> From<Vec<u8>> for WellKnownKey<T> {
fn from(key: Vec<u8>) -> Self {
Self { key, _p: Default::default() }
}
}

impl<T> AsRef<[u8]> for WellKnownKey<T> {
fn as_ref(&self) -> &[u8] {
self.key.as_ref()
}
}

impl<T: Decode> WellKnownKey<T> {
/// Gets the value or `None` if it does not exist or decoding failed.
pub fn get(&self) -> Option<T> {
sp_io::storage::get(&self.key)
.and_then(|raw| parity_scale_codec::DecodeAll::decode_all(&mut raw.as_ref()).ok())
}
}

impl<T: Encode> WellKnownKey<T> {
/// Sets the value.
pub fn set(&self, value: T) {
sp_io::storage::set(&self.key, &value.encode());
}
}

/// Type discriminator for PVF preparation timeouts
#[derive(Encode, Decode, TypeInfo, Clone, Copy, Debug, PartialEq, Eq)]
#[cfg_attr(feature = "std", derive(serde::Serialize, serde::Deserialize))]
Expand Down
39 changes: 36 additions & 3 deletions roadmap/implementers-guide/src/runtime/inclusion.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Inclusion Module

The inclusion module is responsible for inclusion and availability of scheduled parachains and parathreads.
The inclusion module is responsible for inclusion and availability of scheduled parachains and parathreads. It also manages the UMP dispatch queue of each parachain/thread.

## Storage

Expand Down Expand Up @@ -35,11 +35,27 @@ PendingAvailability: map ParaId => CandidatePendingAvailability;
PendingAvailabilityCommitments: map ParaId => CandidateCommitments;
```

## Config Dependencies

* `MessageQueue`:
The message queue provides general queueing and processing functionality. Currently it
replaces the old `UMP` dispatch queue. Other use-cases can be implemented as well by
adding new variants to `AggregateMessageOrigin`. Normally it should be set to an instance
of the `MessageQueue` pallet.

## Session Change

1. Clear out all candidates pending availability.
1. Clear out all validator bitfields.

Optional:
1. The UMP queue of all outgoing paras can be "swept". This would prevent the dispatch queue from automatically being serviced. It is a consideration for the chain and specific behaviour is not defined.

## Initialization

No initialization routine runs for this module. However, the initialization of the `MessageQueue` pallet will attempt to process any pending UMP messages.


## Routines

All failed checks should lead to an unrecoverable error making the block invalid.
Expand Down Expand Up @@ -88,7 +104,7 @@ All failed checks should lead to an unrecoverable error making the block invalid
1. Ensure that any code upgrade scheduled by the candidate does not happen within `config.validation_upgrade_cooldown` of `Paras::last_code_upgrade(para_id, true)`, if any, comparing against the value of `Paras::FutureCodeUpgrades` for the given para ID.
1. Check the collator's signature on the candidate data.
1. check the backing of the candidate using the signatures and the bitfields, comparing against the validators assigned to the groups, fetched with the `group_validators` lookup.
1. call `Ump::check_upward_messages(para, commitments.upward_messages)` to check that the upward messages are valid.
1. call `check_upward_messages(config, para, commitments.upward_messages)` to check that the upward messages are valid.
1. call `Dmp::check_processed_downward_messages(para, commitments.processed_downward_messages)` to check that the DMQ is properly drained.
1. call `Hrmp::check_hrmp_watermark(para, commitments.hrmp_watermark)` for each candidate to check rules of processing the HRMP watermark.
1. using `Hrmp::check_outbound_hrmp(sender, commitments.horizontal_messages)` ensure that the each candidate sent a valid set of horizontal messages
Expand All @@ -99,7 +115,7 @@ All failed checks should lead to an unrecoverable error making the block invalid
1. If the receipt contains a code upgrade, Call `Paras::schedule_code_upgrade(para_id, code, relay_parent_number, config)`.
> TODO: Note that this is safe as long as we never enact candidates where the relay parent is across a session boundary. In that case, which we should be careful to avoid with contextual execution, the configuration might have changed and the para may de-sync from the host's understanding of it.
1. Reward all backing validators of each candidate, contained within the `backers` field.
1. call `Ump::receive_upward_messages` for each backed candidate, using the [`UpwardMessage`s](../types/messages.md#upward-message) from the [`CandidateCommitments`](../types/candidate.md#candidate-commitments).
1. call `receive_upward_messages` for each backed candidate, using the [`UpwardMessage`s](../types/messages.md#upward-message) from the [`CandidateCommitments`](../types/candidate.md#candidate-commitments).
1. call `Dmp::prune_dmq` with the para id of the candidate and the candidate's `processed_downward_messages`.
1. call `Hrmp::prune_hrmp` with the para id of the candiate and the candidate's `hrmp_watermark`.
1. call `Hrmp::queue_outbound_hrmp` with the para id of the candidate and the list of horizontal messages taken from the commitment,
Expand All @@ -118,3 +134,20 @@ All failed checks should lead to an unrecoverable error making the block invalid
* `candidate_pending_availability(ParaId) -> Option<CommittedCandidateReceipt>`: returns the `CommittedCandidateReceipt` pending availability for the para provided, if any.
* `pending_availability(ParaId) -> Option<CandidatePendingAvailability>`: returns the metadata around the candidate pending availability for the para, if any.
* `collect_disputed(disputed: Vec<CandidateHash>) -> Vec<CoreIndex>`: Sweeps through all paras pending availability. If the candidate hash is one of the disputed candidates, then clean up the corresponding storage for that candidate and the commitments. Return a vector of cleaned-up core IDs.

These functions were formerly part of the UMP pallet:

* `check_upward_messages(P: ParaId, Vec<UpwardMessage>)`:
1. Checks that the parachain is not currently offboarding and error otherwise.
1. Checks that there are at most `config.max_upward_message_num_per_candidate` messages to be enqueued.
1. Checks that no message exceeds `config.max_upward_message_size`.
1. Checks that the total resulting queue size would not exceed `co`.
1. Verify that queuing up the messages could not result in exceeding the queue's footprint
according to the config items `config.max_upward_queue_count` and `config.max_upward_queue_size`. The queue's current footprint is provided in `well_known_keys`
in order to facilitate oraclisation on to the para.

Candidate Enactment:

* `receive_upward_messages(P: ParaId, Vec<UpwardMessage>)`:
1. Process each upward message `M` in order:
1. Place in the dispatch queue according to its para ID (or handle it immediately).
87 changes: 0 additions & 87 deletions roadmap/implementers-guide/src/runtime/ump.md

This file was deleted.

5 changes: 0 additions & 5 deletions roadmap/implementers-guide/src/types/runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,6 @@ struct HostConfiguration {
/// no further messages may be added to it. If it exceeds this then the queue may contain only
/// a single message.
pub max_upward_queue_size: u32,
/// The amount of weight we wish to devote to the processing the dispatchable upward messages
/// stage.
///
/// NOTE that this is a soft limit and could be exceeded.
pub ump_service_total_weight: Weight,
/// The maximum size of an upward message that can be sent by a candidate.
///
/// This parameter affects the upper bound of size of `CandidateCommitments`.
Expand Down
Loading

0 comments on commit 7c1dc94

Please sign in to comment.