Add support for generating metadata from runtime wasm files #1720
Conversation
Cargo.toml
Outdated
| @@ -115,6 +115,7 @@ which = "5.0.0" | |||
| strip-ansi-escapes = "0.2.0" | |||
| proptest = "1.5.0" | |||
| hex-literal = "0.4.1" | |||
| subwasmlib = { git = "https://github.com/chevdor/subwasm", rev = "v0.21.3" } | |||
There was a problem hiding this comment.
nit; There's no released crate on crates io?
macro/src/lib.rs
Outdated
| let subwasm = Subwasm::new(&source.try_into().map_err(to_error)?).map_err(to_error)?; | ||
| let mut output = vec![]; | ||
| subwasm | ||
| .write_metadata(OutputFormat::Scale, None, &mut output) |
There was a problem hiding this comment.
Ideally, it would be interesting if we could:
- decode the metadata in a way that we don't depend on substrate crates (
sp-*sc_executor) - extract from the wasm blob just the
Metadata_metadata()bytes
If that turns out to be too complicated, we can still look into obtaining that information via substrate-crates. Using subwasmlib to fetch the metadata brings quite a tree of dependencies.
For extracting the wasm blob, we'd need to rely on sp_maybe_compressed_blob:
https://github.com/chevdor/subwasm/blob/fae3e13bd1f8680bf32cbf2ed223223583f8d713/libs/wasm-loader/src/lib.rs#L152
For calling into the wasm blob and fetching the metadata, we'd rely onsc-executor:
https://github.com/chevdor/subwasm/blob/fae3e13bd1f8680bf32cbf2ed223223583f8d713/libs/wasm-testbed/src/lib.rs#L131-L147
There was a problem hiding this comment.
rewritten to use direct dependencies in last commit
There was a problem hiding this comment.
Nice! Looks good as a first step!
I would try to explore decoding the wasm blob internally, or use the substrate crates directly
edit; I think the end-goal here is to fetch the wasm blob directly from a repo (ie #1660 (comment)), which then justifies the subwasmlib dependency.
The end goal for me here is to point the subxt macro to some WASM runtime file (just as you can also point to the metadata.scale currently) and have it extract the metadata from that and use the metadata to generate an interface.
I agree; I think it'd be good to look into how easily we can avoid this subwasm crate and the various dependencies it brings in and to the decoding internally (or using the relevant |
I've redone things using relevant crates directly and PR unfortunately is still pulling quite a lot of deps. |
macro/Cargo.toml
Outdated
| sc-executor = { workspace = true } | ||
| sp-maybe-compressed-blob = { workspace = true } | ||
| sp-state-machine = { workspace = true } | ||
| sp-io = { workspace = true } | ||
| sc-executor-common = { workspace = true } |
There was a problem hiding this comment.
I wondfer how many dependencies we end up pulling in here; possibly worth putting behind a feature flag like "runtime_file_path" and then perhaps not enabling it by default (since I think it's probably a bit of an edge case that people would point straight to a runtime WASM blob).
There was a problem hiding this comment.
moved them to optional and hidden behind feature flag
| "Exclusively one of 'runtime_metadata_path', 'runtime_metadata_insecure_url' or `runtime_path` must be provided" | ||
| ); | ||
| }; | ||
| let root = std::env::var("CARGO_MANIFEST_DIR").unwrap_or_else(|_| ".".into()); |
There was a problem hiding this comment.
Think so, I've just taken the default value from 'runtime_metada_path' branch tbh
Cargo.lock
Outdated
There was a problem hiding this comment.
nice call, to add a feature-flag for these wasm deps in substrate
| sc-executor = "0.40.0" | ||
| sc-executor-common = "0.35.0" |
There was a problem hiding this comment.
Very much a Nit, but these are substrate crates too so could be below with the rest :)
Description
See #1660
Adds
wasm_file_pathkeyword support tosubxt::subxtmacro to allow extracting metadata from the runtime WASM.No support for compiling target crate to wasm and using it to generate the Metadata for now.
code taken from polkadot-sdk PR