Blocked with subxt extrinsics transactions failure ...

[rajesh-nodle]

Hello Subxt team,

We need your support badly, bit blocked in our parachain bringup, nodle auxillay service is part of nodle chain ecosystem.

Forked subxt and moved substrate dependency to polkadot-v0.9.12, Plz find link to the fork here https://github.com/NodleCode/subxt.

storage() access looks fine confirmed with example ``~/examples/fetch_all_accounts.rs`.

Having issue with extrinsics transaction tx(), with ~/examples/polkadot_balance_transfer.rs hit by below Transaction has a bad signature ...
This issue is, even observed with master https://github.com/paritytech/subxt branch as well.

  Compiling subxt v0.15.0 (/home/popoyi/ssdbox/cauvery/L5-blocks/ws-02-hot-wallet/subxt-polkadot-v0.9.12/subxt)
    Finished dev [unoptimized + debuginfo] target(s) in 12.12s
     Running `target/debug/examples/polkadot_balance_transfer`
[2021-12-05T12:29:02Z DEBUG subxt::rpc] request state_getMetadata: Some(ArrayRef([]))
[2021-12-05T12:29:02Z DEBUG subxt::rpc] request chain_getBlockHash: Some(ArrayRef([Number(0)]))
[2021-12-05T12:29:02Z DEBUG subxt::rpc] request state_getRuntimeVersion: Some(ArrayRef([Null]))
[2021-12-05T12:29:02Z DEBUG subxt::rpc] request system_properties: Some(ArrayRef([]))
[2021-12-05T12:29:02Z DEBUG subxt::rpc] request state_getStorage: Some(ArrayRef([String("0x26aa394eea5630e07c48ae0c9558cef7b99d880ec681799c0cf30e8886371da9de1e86a9a8c739864cf3cc5ec2bea59fd43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d"), Null]))
[2021-12-05T12:29:02Z DEBUG subxt::rpc] request author_submitExtrinsic: Some(ArrayRef([String("0x31028400d43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d01ba904c768bbbb268c899c88a92ed468fbe03cc2b80613fee4c033e3c0afb6c6295273390ab0cba44afc2ccfbb16982f0d52c37253ad3117209eaed04dd9c1b8a002800000200008eaf04151687736326c9fea17e25fc5287613693c912909cb226aa4794f26a48419c")]))
Error: Rpc(Request("{\"jsonrpc\":\"2.0\",\"error\":{\"code\":1010,\"message\":\"Invalid Transaction\",\"data\":\"Transaction has a bad signature\"},\"id\":5}"))
subxt on  polkadot-v0.9.12 [!?] via 🦀 v1.56.1 on ☁️  [email protected](us-central1) took 13s 

with nodle auxillary service transaction, hit with RPC error Could not decode Call, variant doesn't exist. Plz find reference to nodle chain metadata, generated uing codegen util https://gist.github.com/rajesh-nodle/414bfcd8bf6a2067460b7f9e0ba761dc, looks extrinsics & its arguments are OK.

[2021-12-05T11:49:38Z INFO  chain_executor::context] db_process_transaction>[669]=> Prof |  Extrinsic Preparation Time[331.17µs]!!!
[2021-12-05T11:49:38Z TRACE chain_executor::context] transmit_to_chain=>[185]=> Call[[
        Utility(
            as_derivative {
                index: 5,
                call: Balances(
                    transfer {
                        dest: MultiAddress::Id(
                            52be5714cdcd704163593fedbf08d96e262459310907de9bf76a5e7642383e3e (5DwCKtkR...),
                        ),
                        value: 1000000000000,
                    },
                ),
            },
        ),
        Utility(
            as_derivative {
                index: 10,
                call: Balances(
                    transfer {
                        dest: MultiAddress::Id(
                            d0c2566e7541c4f537a63b90fb2ec006c88c0cde81a3c4e0ed6ce5839b144709 (5GnRYi5j...),
                        ),
                        value: 1000000000000,
                    },
                ),
            },
        ),
    ]]!!!
[2021-12-05T11:49:38Z TRACE chain_executor::context] transmit_to_chain=>[191]=> result[Err(
        Rpc(
            Request(
                "{\"jsonrpc\":\"2.0\",\"error\":{\"code\":1002,\"message\":\"Verification Error: Runtime error: Execution failed: ApiError(FailedToConvertParameter { function: \\\"validate_transaction\\\", parameter: \\\"tx\\\", error: Error { cause: Some(Error { cause: None, desc: \\\"Could not decode `Call`, variant doesn't exist\\\" }), desc: \\\"Could not decode `Call::System.0`\\\" } })\",\"data\":\"RuntimeApi(\\\"Execution failed: ApiError(FailedToConvertParameter { function: \\\\\\\"validate_transaction\\\\\\\", parameter: \\\\\\\"tx\\\\\\\", error: Error { cause: Some(Error { cause: None, desc: \\\\\\\"Could not decode `Call`, variant doesn't exist\\\\\\\" }), desc: \\\\\\\"Could not decode `Call::System.0`\\\\\\\" } })\\\")\"},\"id\":7}",
            ),
        ),
    )]!!!
[2021-12-05T11:49:38Z ERROR chain_executor::context] transmit_to_chain=>[200]=> extrinsic failure Err Code[Rpc(
        Request(
            "{\"jsonrpc\":\"2.0\",\"error\":{\"code\":1002,\"message\":\"Verification Error: Runtime error: Execution failed: ApiError(FailedToConvertParameter { function: \\\"validate_transaction\\\", parameter: \\\"tx\\\", error: Error { cause: Some(Error { cause: None, desc: \\\"Could not decode `Call`, variant doesn't exist\\\" }), desc: \\\"Could not decode `Call::System.0`\\\" } })\",\"data\":\"RuntimeApi(\\\"Execution failed: ApiError(FailedToConvertParameter { function: \\\\\\\"validate_transaction\\\\\\\", parameter: \\\\\\\"tx\\\\\\\", error: Error { cause: Some(Error { cause: None, desc: \\\\\\\"Could not decode `Call`, variant doesn't exist\\\\\\\" }), desc: \\\\\\\"Could not decode `Call::System.0`\\\\\\\" } })\\\")\"},\"id\":7}",
        ),
    )]

nodle auxillay service subxt integration Snippet

[ascjones]

Can you try regenerating your static metadata for the client? When using the utility::batch dispatchable it requires the static Call instances to be constructed which need to match exactly those on the target node. See #356

[rajesh-nodle]

Thanks @ascjones for pointer ...

Regarding static metadata for the client ...

Did below step, please correct me if I am wrong ...

cargo run --release -p subxt-cli -- codegen \
  | rustfmt --edition=2018 --emit=stdout \
  > ~/L5-blocks/ws-02-hot-wallet/install-bin/nodle_metadata.rs

As part of codegen is there any possibility to specify generated_type_derives = "Clone, Debug" ?

[ascjones]

Ah I see you are not using the macro directly, why not?

Codegen should work but as you say has less options.

As part of codegen is there any possibility to specify generated_type_derives = "Clone, Debug" ?

That should be easy enough to add, but not high on the priority list at the moment. Please open an issue for it, or even better a PR :)

[rajesh-nodle]

Sure, right now made quick hack like below,
Will Soon get back with PR of cleaner solution :-)

Ah I see you are not using the macro directly, why not?

Looks like still I am missing some pieces.
Right now here is my understanding ...

• Dynamic way of creating metadata for the client using macro

• Static way of creating metadata for the client using codegen & import it

cargo run --release -p subxt-cli -- codegen \
  | rustfmt --edition=2018 --emit=stdout \
  > ~/L5-blocks/ws-02-hot-wallet/install-bin/nodle_runtime.rs

Tried both above possibilities, but still no luck, nodle chain throws same RPC error of Could not decode Call, variant doesn't exist.

Could you please confirm the right way of getting static metadata for the client.

[ascjones]

Well using the macro is the preferred way, and you can do that via

cargo run --release -p subxt-cli -- metadata -f bytes > nodle_metadata.scale

But if you say you have tried this already and it still fails, then the bug is somewhere else.

Do the all the calls work individually when not sent as a batch? e.g. calling tx().balances().transfer(..) instead of manually constructing the call.

[rajesh-nodle]

Sorry got blocked with something else ...

Regarding examples, have taken simple use-case from existing examples/polkadot_balance_transfer.rs

With nodle chain, hit with RPC error of "Transaction has a bad signature" ...

Tried same example with

Chain	Binary	Status
Substrate	https://releases.parity.io/substrate/x86_64-debian:stretch/latest/substrate/substrate	Passed
Polkadot	https://releases.parity.io/polkadot/x86_64-debian:stretch/v0.9.13/polkadot	Failed

Substrate Traces

  Finished dev [unoptimized + debuginfo] target(s) in 1m 12s
     Running `target/debug/examples/polkadot_balance_transfer`
[2021-12-10T05:35:35Z DEBUG subxt::rpc] request state_getMetadata: Some(ArrayRef([]))
[2021-12-10T05:35:35Z DEBUG subxt::rpc] request chain_getBlockHash: Some(ArrayRef([Number(0)]))
[2021-12-10T05:35:35Z DEBUG subxt::rpc] request state_getRuntimeVersion: Some(ArrayRef([Null]))
[2021-12-10T05:35:35Z DEBUG subxt::rpc] request system_properties: Some(ArrayRef([]))
[2021-12-10T05:35:35Z DEBUG subxt::rpc] request state_getStorage: Some(ArrayRef([String("0x26aa394eea5630e07c48ae0c9558cef7b99d880ec681799c0cf30e8886371da9de1e86a9a8c739864cf3cc5ec2bea59fd43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d"), Null]))
[2021-12-10T05:35:35Z DEBUG subxt::extrinsic] spec_version :: 268
[2021-12-10T05:35:35Z DEBUG subxt::extrinsic] tx_version :: 2
[2021-12-10T05:35:35Z DEBUG subxt::extrinsic] nonce :: 0
[2021-12-10T05:35:35Z DEBUG subxt::extrinsic] genesis_hash :: 0xa985ad443893d3921decd39e12d09efb33ec2a27e436b8af757cb9fb52c98a83
[2021-12-10T05:35:35Z DEBUG subxt::rpc] request author_submitExtrinsic: Some(ArrayRef([String("0x31028400d43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d01c0b6870bc80e22ca5164dc2362db54b4a9cdc19f2c53c8536c3590ef4938f6306f3b8ed4e5253a5d4b87593ca48f1c41451437fce9929a1ab651dc56b3ca2680000000000600008eaf04151687736326c9fea17e25fc5287613693c912909cb226aa4794f26a48419c")]))
Balance transfer extrinsic submitted: 0xacd8…0db0

Polkadot Traces

Compiling subxt v0.15.0 (/home/popoyi/ssdbox/cauvery/L5-blocks/ws-02-hot-wallet/shamb0-subxt/subxt)
    Finished dev [unoptimized + debuginfo] target(s) in 18.01s
     Running `target/debug/examples/polkadot_balance_transfer`
[2021-12-10T05:16:36Z DEBUG subxt::rpc] request state_getMetadata: Some(ArrayRef([]))
[2021-12-10T05:16:36Z DEBUG subxt::rpc] request chain_getBlockHash: Some(ArrayRef([Number(0)]))
[2021-12-10T05:16:36Z DEBUG subxt::rpc] request state_getRuntimeVersion: Some(ArrayRef([Null]))
[2021-12-10T05:16:36Z DEBUG subxt::rpc] request system_properties: Some(ArrayRef([]))
[2021-12-10T05:16:36Z DEBUG subxt::rpc] request state_getStorage: Some(ArrayRef([String("0x26aa394eea5630e07c48ae0c9558cef7b99d880ec681799c0cf30e8886371da9de1e86a9a8c739864cf3cc5ec2bea59fd43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d"), Null]))
[2021-12-10T05:16:36Z DEBUG subxt::extrinsic] spec_version :: 9130
[2021-12-10T05:16:36Z DEBUG subxt::extrinsic] tx_version :: 9
[2021-12-10T05:16:36Z DEBUG subxt::extrinsic] nonce :: 0
[2021-12-10T05:16:36Z DEBUG subxt::extrinsic] genesis_hash :: 0x45a8ae86a750ad6808a9f8809818207239d3ea50784b6645e6b9a1fce3752edf
[2021-12-10T05:16:36Z DEBUG subxt::rpc] request author_submitExtrinsic: Some(ArrayRef([String("0x31028400d43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d01666786cb976356a6c9f6aba85abc22fbbc72716a0200639654748ebfc5b6fe34115e14243a3f863605b5b5b2247085fc0e14db7ebf5396ec8ea3d95582b1e985000000000500008eaf04151687736326c9fea17e25fc5287613693c912909cb226aa4794f26a48419c")]))
Error: Rpc(Request("{\"jsonrpc\":\"2.0\",\"error\":{\"code\":1010,\"message\":\"Invalid Transaction\",\"data\":\"Transaction has a bad signature\"},\"id\":5}"))

Used subxt master with all recent updates

Is there any update from susbstrate is missing on Polkadot-v0.9.13 ? looks like for nodle we missing the same.

[ascjones]

Most likely a mismatch between the hardcoded DefaultExtra and the SignedExtra type from that version of polkadot. At the moment subxt is hardcoded to be compatible with the latest substrate node-runtime which may differ from polkadot. We are currently exploring solutions to this, see #331

[rajesh-nodle]

Perfect, its working.

Adopted SignedExtension to the need of nodle-chain (Hard-coded) :-), now its working ...

https://github.com/NodleCode/subxt/blob/polkadot-v0.9.12/src/extrinsic/extra.rs

Verified batch commands with unit test, and its working fine ...

https://github.com/NodleCode/subxt/blob/polkadot-v0.9.12/examples/utility_batch.rs

Thanks a lot for pointer & great support ...