Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Subxt gives error when used with risczero #1742

Closed
amiyatulu opened this issue Aug 31, 2024 · 15 comments
Closed

Subxt gives error when used with risczero #1742

amiyatulu opened this issue Aug 31, 2024 · 15 comments

Comments

@amiyatulu
Copy link

Subxt gives error when used with risczero.

guest_anonymous_account: The following warnings were emitted during compilation:
guest_anonymous_account: 
guest_anonymous_account: warning: [email protected]: riscv32-unknown-elf-gcc: error: unrecognized command-line option '-m64'
guest_anonymous_account: 
guest_anonymous_account: error: failed to run custom build command for `ring v0.17.8`
guest_anonymous_account: 
guest_anonymous_account: Caused by:
guest_anonymous_account:   process didn't exit successfully: `/home/amiya/Documents/workspace/zero-knowledge/anonymous_account/target/riscv-guest/release/build/ring-8e1a6a39262bd209/build-script-build` (exit status: 1)
guest_anonymous_account:   --- stdout
guest_anonymous_account:   cargo:rerun-if-env-changed=RING_PREGENERATE_ASM
guest_anonymous_account:   cargo:rustc-env=RING_CORE_PREFIX=ring_core_0_17_8_
guest_anonymous_account:   OPT_LEVEL = Some(0)
guest_anonymous_account:   TARGET = Some(x86_64-unknown-linux-gnu)
guest_anonymous_account:   OUT_DIR = Some(/home/amiya/Documents/workspace/zero-knowledge/anonymous_account/target/riscv-guest/release/build/ring-6aa5585402d7db96/out)
guest_anonymous_account:   HOST = Some(x86_64-unknown-linux-gnu)
guest_anonymous_account:   cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
guest_anonymous_account:   CC_x86_64-unknown-linux-gnu = None
guest_anonymous_account:   cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
guest_anonymous_account:   CC_x86_64_unknown_linux_gnu = None
guest_anonymous_account:   cargo:rerun-if-env-changed=HOST_CC
guest_anonymous_account:   HOST_CC = None
guest_anonymous_account:   cargo:rerun-if-env-changed=CC
guest_anonymous_account:   CC = Some(/home/amiya/.local/share/cargo-risczero/cpp/bin/riscv32-unknown-elf-gcc)
guest_anonymous_account:   RUSTC_WRAPPER = None
guest_anonymous_account:   cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
guest_anonymous_account:   cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
guest_anonymous_account:   CRATE_CC_NO_DEFAULTS = None
guest_anonymous_account:   DEBUG = Some(false)
guest_anonymous_account:   CARGO_CFG_TARGET_FEATURE = Some(fxsr,sse,sse2)
guest_anonymous_account:   cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
guest_anonymous_account:   CFLAGS_x86_64-unknown-linux-gnu = None
guest_anonymous_account:   cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
guest_anonymous_account:   CFLAGS_x86_64_unknown_linux_gnu = None
guest_anonymous_account:   cargo:rerun-if-env-changed=HOST_CFLAGS
guest_anonymous_account:   HOST_CFLAGS = None
guest_anonymous_account:   cargo:rerun-if-env-changed=CFLAGS
guest_anonymous_account:   CFLAGS = None
guest_anonymous_account:   cargo:warning=riscv32-unknown-elf-gcc: error: unrecognized command-line option '-m64'
guest_anonymous_account: 
guest_anonymous_account:   --- stderr
guest_anonymous_account: 
guest_anonymous_account: 
guest_anonymous_account:   error occurred: Command "/home/amiya/.local/share/cargo-risczero/cpp/bin/riscv32-unknown-elf-gcc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "include" "-I" "/home/amiya/Documents/workspace/zero-knowledge/anonymous_account/target/riscv-guest/release/build/ring-6aa5585402d7db96/out" "-Wall" "-Wextra" "-fvisibility=hidden" "-std=c1x" "-Wall" "-Wbad-function-cast" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wnested-externs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wstrict-prototypes" "-Wundef" "-Wuninitialized" "-g3" "-DNDEBUG" "-o" "/home/amiya/Documents/workspace/zero-knowledge/anonymous_account/target/riscv-guest/release/build/ring-6aa5585402d7db96/out/fad98b632b8ce3cc-curve25519.o" "-c" "crypto/curve25519/curve25519.c" with args riscv32-unknown-elf-gcc did not execute successfully (status code exit status: 1).
guest_anonymous_account: 
guest_anonymous_account: 
guest_anonymous_account: warning: build failed, waiting for other jobs to finish...
error: failed to run custom build command for `methods v0.1.0 (/home/amiya/Documents/workspace/zero-knowledge/anonymous_account/methods)`

Caused by:
  process didn't exit successfully: `/home/amiya/Documents/workspace/zero-knowledge/anonymous_account/target/debug/build/methods-d0d32bf95e276aa9/build-script-build` (exit status: 101)
  --- stdout
  Building guest package methods.guest_anonymous_account
  cargo:rerun-if-env-changed=RISC0_SKIP_BUILD
  Using rustc: /home/amiya/.rustup/toolchains/risc0/bin/rustc
  cargo:rerun-if-env-changed=RISC0_RUST_SRC
  Building guest package: cargo build --target riscv32im-risc0-zkvm-elf
  cargo:rerun-if-env-changed=RISC0_BUILD_DEBUG
@niklasad1
Copy link
Member

niklasad1 commented Aug 31, 2024

Hey @amiyatulu

subxt is using the jsonrpsee-ws-client by default it depends on ring which doesn't compile for risc-v and probably jsonrpsee doesn't as well. So you need to disable the jsonrpsee feature in subxt but it might possible to use jsonrpsee without tls.

What will work is provide some other rpc client that you know compiles risc zero and then implement the subxt::RpcClientT trait example here

We have not tested risc-v or know that much about the platform, so it might be a frustrating experiment but we do know that subxt-core compiles for no-std at least so not impossible :)

@amiyatulu
Copy link
Author

amiyatulu commented Aug 31, 2024

Here is issue on risc v support on ring.

briansmith/ring#1182

risc0 issue, may be one has to wait for the support:
risc0/risc0#829

When I use jsonrpsee, it successfully compiles in risc-zero but not with feature jsonrpsee-ws-client, that depends on ring.

@niklasad1
Copy link
Member

jsonrpsee without any features doesn't enable anything

Try building the following:

jsonrpsee-ws-client = { version = "0.22", default-features = false }

This will disable TLS

@amiyatulu
Copy link
Author

Tonic successfully compiled with risczero. But not sure how to implement subxt::RpcClientT trait with tonic.

Should I have to modify the struct

struct MyLoggingClient {
    log: Arc<Mutex<String>>,
}

Tonic rpc can be good alternative to jsonrpsee, an example file will be a good idea.

@niklasad1
Copy link
Member

jsonrpsee most likely works as well if you disable TLS and tonic hasn't enabled TLS in the example

@amiyatulu
Copy link
Author

amiyatulu commented Sep 23, 2024

Will there be security issues if I disable tls ?
Like manipulating data in between (data integrity) while producing zero knowledge proofs.

@niklasad1
Copy link
Member

Will there be security issues if I disable tls ?
Like manipulating data in between (data integrity) while producing zero knowledge proofs.

It's something you have to decide i.e, whether you can have it running on localhost or something that you trust but it's not recommended obviously (I don't know how to workaround that in risc-v)

Anyway I have created an example how to get you started with subxt without tls, https://github.com/niklasad1/subxt-riscv-example but it's nothing really that on our roadmap to support but it's quite cool...

@amiyatulu
Copy link
Author

Not localhost but using lightclient.

If I can fetch data from the light client without any security issues, then I don't have any problem.

However, to offload computations off-chain, RiscZero support is required. Using Subxt and risczero I can securely fetch data from the blockchain, make intermediate transactions, perform off-chain computations, and store the proof of computation on the blockchain.

For instance, to calculate the standard deviation of millions of values, I can fetch the data from the blockchain using RiscZero and Subxt, perform the calculation, and then submit the proof containing the output value, i.e., the standard deviation. One can be sure that output value is the actual standard deviation without manipulation. Importantly, the duration of the computation does not matter - even if it takes an hour, as only proof is still stored on the blockchain.

#1741

@amiyatulu
Copy link
Author

Also the dependancies don't work in risczero. It still gives ring error:

serde = "1.0.209"
serde_json = "1.0.127"
subxt = { version = "0.37.0", default-features = false, features = ["native"] }
tokio = { version = "1.40.0", features = ["rt-multi-thread", "macros", "time"] }
jsonrpsee-ws-client = { version = "0.24.5", default-features = false }
jsonrpsee-core = { version = "0.24.5", features = ["client"] }
futures = "0.3"

Error:

guest_anonymous_account: The following warnings were emitted during compilation:
guest_anonymous_account: 
guest_anonymous_account: warning: [email protected]: riscv32-unknown-elf-gcc: error: unrecognized command-line option '-m64'
guest_anonymous_account: 
guest_anonymous_account: error: failed to run custom build command for `ring v0.17.8`
guest_anonymous_account: 
guest_anonymous_account: Caused by:
guest_anonymous_account:   process didn't exit successfully: `/home/amiya/Documents/workspace/zero-knowledge/anonymous_account/target/riscv-guest/release/build/ring-03b100417c8058ad/build-script-build` (exit status: 1)
guest_anonymous_account:   --- stdout
guest_anonymous_account:   cargo:rerun-if-env-changed=RING_PREGENERATE_ASM
guest_anonymous_account:   cargo:rustc-env=RING_CORE_PREFIX=ring_core_0_17_8_
guest_anonymous_account:   OPT_LEVEL = Some(0)
guest_anonymous_account:   TARGET = Some(x86_64-unknown-linux-gnu)
guest_anonymous_account:   OUT_DIR = Some(/home/amiya/Documents/workspace/zero-knowledge/anonymous_account/target/riscv-guest/release/build/ring-8eb799c10db90681/out)
guest_anonymous_account:   HOST = Some(x86_64-unknown-linux-gnu)
guest_anonymous_account:   cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
guest_anonymous_account:   CC_x86_64-unknown-linux-gnu = None
guest_anonymous_account:   cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
guest_anonymous_account:   CC_x86_64_unknown_linux_gnu = None
guest_anonymous_account:   cargo:rerun-if-env-changed=HOST_CC
guest_anonymous_account:   HOST_CC = None
guest_anonymous_account:   cargo:rerun-if-env-changed=CC
guest_anonymous_account:   CC = Some(/home/amiya/.local/share/cargo-risczero/cpp/bin/riscv32-unknown-elf-gcc)
guest_anonymous_account:   RUSTC_WRAPPER = None

@niklasad1
Copy link
Member

Ok, subxt seems to leak that feature from the reconnecting-client. I pushed a fix but I was to lazy to install the toolchain.

I'll test it later today... or may you could test it again.

@amiyatulu
Copy link
Author

No, still the ring error.

@amiyatulu
Copy link
Author

jsonrpsee-ws-client is also not working, its gives sys error.

jsonrpsee-ws-client = { version = "0.24.5", default-features = false }

Error

guest_anonymous_account: error[E0583]: file not found for module `sys`
guest_anonymous_account:    --> /home/amiya/.cargo/registry/src/index.crates.io-6f17d22bba15001f/socket2-0.5.7/src/lib.rs:179:1
guest_anonymous_account:     |
guest_anonymous_account: 179 | mod sys;
guest_anonymous_account:     | ^^^^^^^^
guest_anonymous_account:     |
guest_anonymous_account:     = help: to create the module `sys`, create file "/home/amiya/.cargo/registry/src/index.crates.io-6f17d22bba15001f/socket2-0.5.7/src/sys.rs" or "/home/amiya/.cargo/registry/src/index.crates.io-6f17d22bba15001f/socket2-0.5.7/src/sys/mod.rs"
guest_anonymous_account:     = note: if there is a `mod sys` elsewhere in the crate already, import it with `use crate::...` instead
guest_anonymous_account: 
guest_anonymous_account: error: Socket2 doesn't support the compile target
guest_anonymous_account:    --> /home/amiya/.cargo/registry/src/index.crates.io-6f17d22bba15001f/socket2-0.5.7/src/lib.rs:182:1
guest_anonymous_account:     |
guest_anonymous_account: 182 | compile_error!("Socket2 doesn't support the compile target");
guest_anonymous_account:     | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
guest_anonymous_account: 
guest_anonymous_account: error[E0432]: unresolved imports `crate::sys::c_int`, `crate::sys::sa_family_t`, `crate::sys::sockaddr`, `crate::sys::sockaddr_in`, `crate::sys::sockaddr_in6`, `crate::sys::sockaddr_storage`, `crate::sys::socklen_t`, `crate::sys::AF_INET`, `crate::sys::AF_INET6`, `crate::sys::AF_UNIX`
guest_anonymous_account:   --> /home/amiya/.cargo/registry/src/index.crates.io-6f17d22bba15001f/socket2-0.5.7/src/sockaddr.rs:11:5
guest_anonymous_account:    |
guest_anonymous_account: 11 |     c_int, sa_family_t, sockaddr, sockaddr_in, sockaddr_in6, sockaddr_storage, socklen_t, AF_INET,
guest_anonymous_account:    |     ^^^^^  ^^^^^^^^^^^  ^^^^^^^^  ^^^^^^^^^^^  ^^^^^^^^^^^^  ^^^^^^^^^^^^^^^^  ^^^^^^^^^  ^^^^^^^ no `AF_INET` in `sys`
guest_anonymous_account:    |     |      |            |         |            |             |                 |
guest_anonymous_account:    |     |      |            |         |            |             |                 no `socklen_t` in `sys`
guest_anonymous_account:    |     |      |            |         |            |             no `sockaddr_storage` in `sys`
guest_anonymous_account:    |     |      |            |         |            no `sockaddr_in6` in `sys`
guest_anonymous_account:    |     |      |            |         no `sockaddr_in` in `sys`
guest_anonymous_account:    |     |      |            no `sockaddr` in `sys`
guest_anonymous_account:    |     |      no `sa_family_t` in `sys`
guest_anonymous_account:    |     no `c_int` in `sys`
guest_anonymous_account: 12 |     AF_INET6, AF_UNIX,
guest_anonymous_account:    |     ^^^^^^^^  ^^^^^^^ no `AF_UNIX` in `sys`
guest_anonymous_account:    |     |
guest_anonymous_account:    |     no `AF_INET6` in `sys`
guest_anonymous_account:    |
guest_anonymous_account:    = help: consider importing one of these items instead:
guest_anonymous_account:            core::ffi::c_int
guest_anonymous_account:            std::ffi::c_int
guest_anonymous_account:            std::os::raw::c_int
guest_anonymous_account:    = help: consider importing this module instead:
guest_anonymous_account:            crate::sockaddr

Only jsonrpsee-core and jsonrpsee (without any features) successfully compiles:

jsonrpsee = {version = "0.24.5"}
jsonrpsee-core = { version = "0.24.5", features = ["client"] }

@amiyatulu
Copy link
Author

amiyatulu commented Oct 4, 2024

Just discussing in risczero forum

I would imagine typically you might use subxt on the host to interact with the chain. Then, perform any verifications inside of the guest, which you could do without subxt dependencies.

As host subxt successfully compiles, but it doesn't compile in guest.

there is no network access or filesystem access within the zkVM and so it'd have to be an engineering question of how this could be done, something like zkTLS

@niklasad1
Copy link
Member

Ok, I have raised #1812 which is subxt issue that we should fix.
Sorry didn't realize that risczero didn't have an OS or socket support then it becomes hard to support

I would rather copy-paste this in to a discussion thread because I don't regard it as subxt issue because subxt has:

  • subxt-core (compiles for no-std)
  • offline-client
  • online-client (that requires std, socket and tls support)

Thus, the way forward for you is most likely to keep interacting with the chain via RPC as host and do the zk-stuff as guest. You may be able to use IPC or something else to interact between guest/host but I don't know that stuff...

Hopefully that will help to engineer something around it.

@amiyatulu
Copy link
Author

amiyatulu commented Oct 7, 2024

Sorry didn't realize that risczero didn't have an OS or socket support then it becomes hard to support

Yes, currently I am trying to just compare input data string and journal data of risczero in blockchain, to have offchain computing. One can also use hash of input data for comparision. It would cheaper to do, and solves most use cases.

subxt-signer is needed for guest in risczero and is already compatible with it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants