libain gRPC intgration

[Expand] RPC calls list

Working branch: epic/libain

Background

libain-rs takes advantage of Rust async/await to offer JSON RPC and gRPC servers. It also generates the necessary C++ glue code for seamlessly transitioning RPC functions to FFI functions which are then used by the library to serve RPC calls.

Advantages:

Having a typed language-agnostic spec (protobuf) which can then be used by other languages to generate their own clients and communicate through gRPC
Getting rid of libevent dependency in C++
Pave way for a new testing framework in Rust

Most of the work is automated in the Rust side (through codegen generated from build script), so you'll only have to modify the protobuf files and add/change C++ functions. This should automatically reflect in the served RPCs.

Exposed functions from Rust

init_runtime: To instantiate logger and runtime (event loop, scheduler and executor). The ownership is then given to C++ through an opaque type, which is then required for the rest of the functions.
start_servers: Spawns RPC servers in a separate thread bound to the provided addresses.
stop_servers: Consumes the runtime once and for all and shuts down the servers.

Adding new RPC call

Let's take getbestblockhash function as an example.

Firstly, we should write a fair enough protobuf definition for this function:

syntax = "proto3";
package rpc;

import "google/protobuf/empty.proto";
import "types/block.proto";

service Blockchain {
    // Returns the hash of the best (tip) block in the most-work fully-validated chain.
    rpc GetBestBlockHash(google.protobuf.Empty) returns (types.BlockResult);
}

This will reside in protobuf/rpc/blockchain.proto inside libain-rs

The corresponding type will be in protobuf/types/block.proto

syntax = "proto3";
package types;

message BlockResult {
    string hash = 1; // Hex-encoded data for block hash
}

In codegen.rs, we already have:

pub mod types {
    tonic::include_proto!("types");
}

pub mod rpc {
    tonic::include_proto!("rpc");
}

This will include the code generated from both rpc and types protobuf modules.

As we've added a new service, we have to mount it on both JSON RPC and gRPC servers. This can be done in lib.rs. All we have to do is import BlockchainService from codegen::rpc and invoke BlockchainService::service() and BlockchainService::module() functions for adding them to the corresponding servers.

During compilation, Rust will emit the C++ function signature inside libain.cpp and libain.hpp files in target/ directory. The generated Rust function will look like:

fn GetBestBlockHash(result: &mut BlockResult) -> Result<()>;

You can debug the generated code by navigating to target/debug/build/ain-grpc-{checksum}/out/*.rs and run cargo fmt -- target/debug/build/ain-grpc-*/out/*.rs to format the code for readability.

The corresponding C++ function will look like:

void GetBestBlockHash(BlockResult &result);

Note that the return value is passed as a mutable reference. This way, Rust will have exclusive ownership to the struct and we don't have to move/free anything in C++ side.

These changes should then be pushed to a branch in libain-rs repo.

Adding C++ function

libain-rs branch should now be reflected in depends/packages/libain.mk inside ain repo.

Based on the emitted function signature, the actual implementation in defid will now look like:

void GetBestBlockHash(BlockResult &result)
{
    LOCK(cs_main);
    result.hex_data = ::ChainActive().Tip()->GetBlockHash().GetHex();
}

Now, if we run ./make.sh build, we'll have the newly added function as part of defid, which we can test with the following command:

curl --data-binary '{"jsonrpc": "2.0", "id":"curltest", "method": "getbestblockhash", "params": [] }' \
    -H 'content-type: application/json' http://localhost:50050
{"jsonrpc":"2.0","result":"034ac8c88a1a9b846750768c1ad6f295bc4d0dc4b9b418aee5c0ebd609be8f90","id":"curltest"}

If libain-rs branch is changed along the way, then you'll have to force depends to download and compile it again. You can do so by running:

rm -rf depends/built/x86_64-pc-linux-gnu/libain; ./make.sh build

Initialization of Rust structs

Please make sure to not instantiate Rust structs as normal C++ structs like so:

Transaction txn;

This will contain garbage values and won't be suitable for our API. We already expose constructor functions which initialize these structs from Rust side and hand it over to C++. For the above example, you can call instead:

auto txn = MakeTransaction();

Customization

As of now, the build script supports the following (more can be added as we go):

Type attributes: Type-specific attributes can be set using TYPE_ATTRS constant. Rules can be specified to rename types and skip setting attributes for specific types (in case you want to implement manually).
Field attributes: With FIELD_ATTRS constant, fields can be manipulated similar to types. In case of fields, filters can be applied based on parent structs and field types.
Defaults: Some of the RPC calls may take inputs which have defaults. This can be specified in protobuf through comments. The format will be: [default: X] where X must be the corresponding default value. Notations must be specified as appropriate. Taking string as an example, the default should be specified like so: [default: "foobar"]
Skipping: The choice of server/client-side generation of code can also be specified on individual RPC calls which will skip generating the irrelevant code. If [client] is specified in the comment, only the RPC client code is generated and vice versa for [server], and if [ignore] is specified, it'll skip the call entirely.
URL: Sometimes RPC URLs can be different from the name we specify in protobuf, for which we can use [rpc: url] format in the comment.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly