Parameterize `AccountData` #409

ascjones · 2022-01-26T11:10:35Z

Fixes #391.

Previously the codegen hardcoded the AccountData parameter of RuntimeApi to DefaultAccountData, which in turn was hardcoded to using the DefaultConfig.

This meant that the user was not able to supply their custom Config implemetnation, nor a custom AccountData implementation.

This PR allows the user to supply their own custom Config impl, and optionally a custom AccountData impl.

Update

The macro now autogenerates the DefaultAccountData impl against the System::Account storage entry, and hardcodes the associated key (AccountId) and value (Nonce) types by looking them under the known names in the metadata.

I really wanted to allow the type system to do as much as possible with no macro magic, but couldn't get it to work due to this error. So now what it does is it e.g. looks up the type of System::Account::nonce and assigns it to type Nonce.

jsdw · 2022-01-26T11:22:10Z

examples/custom_config.rs

+impl Config for MyConfig {
+    type Index = <DefaultConfig as Config>::Index;
+    type BlockNumber = <DefaultConfig as Config>::BlockNumber;
+    type Hash = <DefaultConfig as Config>::Hash;
+    type Hashing = <DefaultConfig as Config>::Hashing;
+    type AccountId = <DefaultConfig as Config>::AccountId;
+    type Address = <DefaultConfig as Config>::Address;
+    type Header = <DefaultConfig as Config>::Header;
+    type Signature = <DefaultConfig as Config>::Signature;
+    type Extrinsic = <DefaultConfig as Config>::Extrinsic;
+}


Just for the sake of the example, could we actually override one or more of these types perhaps (maybe then with a comment saying what's going on), just to make it clear that that's sortof the point of this impl? (I realise that doing so may prevent it from actually working below; something that would want to be made clear also.)

jsdw

Wee suggestion re the example but looks great!

trusch · 2022-01-27T09:39:26Z

I tried using your example and ran into this:

error[E0277]: the trait bound `<T as Config>::Index: From<u64>` is not satisfied
  --> src/main.rs:26:1
   |
26 | #[subxt::subxt(runtime_metadata_path = "metadata.scale")]
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `From<u64>` is not implemented for `<T as Config>::Index`
   |
   = note: required because of the requirements on the impl of `Into<<T as Config>::Index>` for `u64`

jsdw · 2022-01-27T09:53:24Z

examples/custom_config.rs

+#[derive(Clone, Debug, Default, Eq, PartialEq)]
+pub struct MyConfig;
+impl Config for MyConfig {
+    type Index = u64; // this is different from the default `u32`


I like this; worth noting that the example won't run unless this is set back to the default though I think.

The example does run though, I guess because u64 : From<u32>

Ah; I assumed that @trusch's comment meant otherwise!

jsdw · 2022-01-27T09:54:11Z

I tried using your example and ran into this:

error[E0277]: the trait bound `<T as Config>::Index: From<u64>` is not satisfied
  --> src/main.rs:26:1
   |
26 | #[subxt::subxt(runtime_metadata_path = "metadata.scale")]
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `From<u64>` is not implemented for `<T as Config>::Index`
   |
   = note: required because of the requirements on the impl of `Into<<T as Config>::Index>` for `u64`

That is deliberate (but should be emphasised); I think it's worth showing how you'd actually customise the config, but of course doing so does mean that it won't actually run against a default node any more.

trusch · 2022-01-27T10:19:25Z

Sorry for not providing enough information to reproduce this: I used the metadata from the kilt network and the only change to the config was that I set the Index to u64 exactly as in your example. I attached the metadata I used since its only available through the websocket endpoints and therefore not downloadable using subxt-cli. I had to add the .md extension so github allowed me to upload it. The code that I try to compile is excactly the same as the code from your example.

Dependencies look look like this:

[dependencies]
subxt = { git = "https://github.com/paritytech/subxt.git", branch = "aj/custom-config" }
codec = { package = "parity-scale-codec", version = "2", default-features = false, features = ["derive", "full", "bit-vec"] }
sp-keyring = "4.0.0"
async-std = { version = "1.9.0", features = ["attributes", "tokio1"] }

ascjones · 2022-01-27T10:47:11Z

@trusch that file does not contain the metadata, just the subxt-cli usage docs 🙈

trusch · 2022-01-27T10:53:08Z

@ascjones My bad, I was a bit in a hurry and I guess I accidentally overwrote that file. Here is the correct one:
metadata.scale.md

ascjones · 2022-01-27T17:49:47Z

@trusch can you try now?

trusch · 2022-01-28T06:33:51Z

@ascjones It worked! The particular error I experienced is gone now 👍

I still can't get it to fully compile because one of the types generated from the metadata doesn't properly implement Decode:

error[E0277]: the trait bound `BoundedBTreeSet<DidEncryptionKey>: Decode` is not satisfied
  --> src/main.rs:26:1
   |
26 | #[subxt::subxt(runtime_metadata_path = "metadata.scale")]
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `Decode` is not implemented for `BoundedBTreeSet<DidEncryptionKey>`
   |
   = help: the following implementations were found:
             <BoundedBTreeSet<_0> as Decode>

But nevertheless, thanks for picking up and solving my initial issue so fast!

ascjones · 2022-01-28T09:10:08Z

the trait bound BoundedBTreeSet<DidEncryptionKey>: Decode is not satisfied

I'm guessing that is because DidEncryptionKey does not implement Ord. You can work around that by using a type substitute (define your own DidEncryptionKey with Ord implemented), or possibly add the Ord derive to all types: not sure if that will work though,

dvdplm

lgtm, left a few nits and doc suggestions.

codegen/src/api/mod.rs

dvdplm · 2022-01-31T09:41:09Z

codegen/src/api/mod.rs

+/// Most chains require a valid account nonce as part of the extrinsic, so the default behaviour of
+/// the client is to fetch the nonce for the current account (if it has not been specified).
+///
+/// The account nonce is commonly stored in the `System` pallet's `Account` storage item. This


I wonder if it's actually possible to build a chain that stores the nonce elsewhere?

Is "nonce" here the same as the "Account index"? I think both terms are fine, but we should perhaps clarify that it's the Config::Index associated type we're referring to here (if that's what it is).

I wonder if it's actually possible to build a chain that stores the nonce elsewhere?

It would involve a custom frame_system pallet. It's also possible that this will change across versions of frame_system, I know it has in the past

Is "nonce" here the same as the "Account index"?

Yes indeed it is used interchangeably. Personally I find index slightly misleading, and the field is called nonce, but I can change the assoc type name here for consistency?

dvdplm · 2022-01-31T09:44:19Z

codegen/src/api/mod.rs

+                    .iter()
+                    .find(|f| f.name() == Some(&"nonce".to_string()))?
+            } else {
+                abort_call_site!("Expected a `nonce` field in the account info struct")


I think both this assumption and the one below about it being a map should be documented: "The macro expects the AccountInfo type to be a map and have a nonce field." or something along those lines.

examples/custom_config.rs

dvdplm · 2022-01-31T10:32:24Z

examples/custom_config.rs

+    type BlockNumber = <DefaultConfig as Config>::BlockNumber;
+    type Hash = <DefaultConfig as Config>::Hash;
+    type Hashing = <DefaultConfig as Config>::Hashing;
+    type AccountId = <DefaultConfig as Config>::AccountId;


I wonder if this example would be more useful if we made a more plausible change to the config, something a project might actually want to do. Not for this PR, it'd be more like a tutorial I reckon.

Yes, it would be best accompanied with an actual runtime with the different type

Co-authored-by: David <[email protected]>

* Add custom config example * Make account data storage item configurable * Fix contracts * Regenerate polkadot codegen * Fmt * Specify different type for `MyConfig::Index` * Update comment in custom config example * Assign concrete types for default AccountData impl * Fmt * Fix contracts tests * Fmt * Add comments * Unlink doc comment trait (subxt not in scope) * Fix missing nonce field error message * Update codegen/src/api/mod.rs Co-authored-by: David <[email protected]> * Update examples/custom_config.rs Co-authored-by: David <[email protected]> * Rename Nonce assoc type to Index for consistency * Add module level docs about codegen assumptions Co-authored-by: David <[email protected]>

ascjones added 5 commits January 25, 2022 18:01

Add custom config example

f3130cc

Make account data storage item configurable

079efaa

Fix contracts

7f3a167

Regenerate polkadot codegen

be43687

Fmt

afb0beb

jsdw reviewed Jan 26, 2022

View reviewed changes

jsdw approved these changes Jan 26, 2022

View reviewed changes

Specify different type for MyConfig::Index

c466529

ascjones mentioned this pull request Jan 26, 2022

Subxt parse kilt metadata failed. response: expected u32, found u64 #369

Closed

jsdw reviewed Jan 27, 2022

View reviewed changes

Update comment in custom config example

9f810f0

ascjones added 4 commits January 27, 2022 17:37

Assign concrete types for default AccountData impl

b040cbb

Merge branch 'master' into aj/custom-config

5eb9695

Fmt

e13a989

Fix contracts tests

a07714c

Fmt

275e948

ascjones added 4 commits January 28, 2022 09:30

Merge branch 'master' into aj/custom-config

6cd1637

Add comments

5a8916c

Unlink doc comment trait (subxt not in scope)

797be56

Fix missing nonce field error message

2c3f388

ascjones mentioned this pull request Jan 28, 2022

[track] Release subxt to crates.io #345

Closed

18 tasks

dvdplm approved these changes Jan 31, 2022

View reviewed changes

ascjones and others added 4 commits January 31, 2022 11:48

Update codegen/src/api/mod.rs

0a138e1

Co-authored-by: David <[email protected]>

Update examples/custom_config.rs

28fbe60

Co-authored-by: David <[email protected]>

Rename Nonce assoc type to Index for consistency

c1a1610

Add module level docs about codegen assumptions

9d6a658

ascjones merged commit f5d8a84 into master Jan 31, 2022

ascjones deleted the aj/custom-config branch January 31, 2022 12:46

Xanewok mentioned this pull request Oct 26, 2022

Codegen assumes presence of frame_system::AccountData #348

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Parameterize `AccountData` #409

Parameterize `AccountData` #409

ascjones commented Jan 26, 2022 •

edited

Loading

jsdw Jan 26, 2022

jsdw left a comment

trusch commented Jan 27, 2022

jsdw Jan 27, 2022

ascjones Jan 27, 2022

jsdw Jan 27, 2022

jsdw commented Jan 27, 2022

trusch commented Jan 27, 2022 •

edited

Loading

ascjones commented Jan 27, 2022

trusch commented Jan 27, 2022

ascjones commented Jan 27, 2022

trusch commented Jan 28, 2022

ascjones commented Jan 28, 2022

dvdplm left a comment

dvdplm Jan 31, 2022

dvdplm Jan 31, 2022

ascjones Jan 31, 2022

dvdplm Jan 31, 2022

dvdplm Jan 31, 2022

ascjones Jan 31, 2022

Parameterize AccountData #409

Parameterize AccountData #409

Conversation

ascjones commented Jan 26, 2022 • edited Loading

Update

Choose a reason for hiding this comment

jsdw left a comment

Choose a reason for hiding this comment

trusch commented Jan 27, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsdw commented Jan 27, 2022

trusch commented Jan 27, 2022 • edited Loading

ascjones commented Jan 27, 2022

trusch commented Jan 27, 2022

ascjones commented Jan 27, 2022

trusch commented Jan 28, 2022

ascjones commented Jan 28, 2022

dvdplm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Parameterize `AccountData` #409

Parameterize `AccountData` #409

ascjones commented Jan 26, 2022 •

edited

Loading

trusch commented Jan 27, 2022 •

edited

Loading