[Feature]: Provable GraphQL Query Engine

[aaronc]

Summary

Develop a query engine in Rust that builds a GraphQL schema from ORM and collections schemas which can run client side (via WASM) and do optional state proofs.

Problem Definition

In various discussions around our current approach to queries the following pain points have come up:

• gRPC query servers currently need to be constructed manually even though we have some automated schema management solutions (ORM + collections)
• gRPC queries aren't provable
• our current approach makes nodes responsible for queries and some have raised concern around the performance of this plus lack of separation of concerns
• queries formulation depends on which indexes are defined on chain

Also #14965 attempts to define a gRPC query generator for ORM, but this still wouldn't be provable and it needs to deal with some complex address encodings to really be usable for clients.

Proposal

Build a GraphQL query engine in Rust that builds its schema for the ORM + collections schemas for a chain. This query engine would read ABCI state directly and eventually could support based on the app hash + light client + IAVL proofs.

Building a Rust engine will allow the same code to be used either in a browser via WASM or to be deployed as a service and run inside a node. By doing the querying directly against ABCI, it is possible to securely verify proofs client side.

GraphQL has the following benefits for this purpose:

• we can add functions for dealing with scalar types such as address bytes
• it is possible to define relations between different parts of the schema

We could also consider something based on gRPC or REST, but I think GraphQL would generally provide a better dev UX. But that could be up for discussion and it could be possible to also support other query interfaces.

In order to define an API that does not depend on a specific set of indexes to be present on-chain, a query planner layer should be used as an intermediary on top of the KV store. This could allow other storage backends with custom indexes to be used with the same query API. https://github.com/apache/arrow-datafusion or https://github.com/gluesql/gluesql could maybe be used as out of the box query planners or a custom one could be built. Having an SQL query planner layer would have the added benefit of an SQL query API for state.

NOTE: we likely don't have bandwidth for this near term - this issue is mainly to document the idea for the future and to align ORM and collections schema design.

[robert-zaremba]

BTW, there is also tiny-go which works well with browsers: https://tinygo.org/docs/guides/webassembly/

[aaronc]

Tiny Go looks interesting but its docs on wasm integration are quite slim compared to Rust, plus GC overhead... so I suspect performance and bundle size would suffer

[tac0turtle]

im hesitant to do this as grpc has the potential to cover all we need. If people would want graphql i think we should recommend to do that as a secondary process, not something built into the node.

[aaronc]

im hesitant to do this as grpc has the potential to cover all we need. If people would want graphql i think we should recommend to do that as a secondary process, not something built into the node.

This proposal is specifically about building a secondary process/client-side library instead of something in a node

[elix1er]

LFG