Off-chain, local store for ORM module state (managed by the state machine)

[robert-zaremba]

Is your feature request related to a problem? Please describe.

The ORM package (/orm) is used to provide other modules with index related records (eg a sequence of integers for new record ids). Building indexing features are not needed for consensus. We don't need to store the latest ID / sequence number in a state storage.

Storing things in the state machine state has a performance and storage overhead: it has to be organized in the state consensus data structure (merkle tree) which is much slower. Also adding more stuff there impacts the performance of the chain.

Describe the solution you'd like

We can use an off-chain, in process helper DB (goleveldb, berkeleyDB....) to store support data (eg the latest sequence number).

This won't decrease the security of the blockchain, nor impact the consensus: the transaction generated records, with proper IDs, will continue to be stored in the state tree. So the proper values will still be verified by the consensus.

Additional context

Updating the orm to use a helper DB (eg goleveldb) is rather straightforward. We can use the current tm-db interface or picking one KV store and use it.

The problem is with with node sync: both by importing a chain dump, or using fast sync. Currently, only chain data is exported. So we would need to add orm off-chain state to genesis and to the fast sync.

[aaronc]

Not a bad idea, but obviously lots of details to be worked out. How much does this overlap with the caching work Ian has been working on? Would we use a kv-store or why not just sqlite? Would these queries be available within the consensus layer or not? (In ADR 033, queries become available to other modules in the consensus layer)

[robert-zaremba]

This could be complementary to Ian work. The difference is in: who is in charge of a supplementary KV store:

• here I propose to use a store which is managed by a module. It's not a part of the blockchain committed state, but it's used for consensus (inside the state machine). Example: we don't need to commit to the latest sequence number. It's enough that the network will agree what's the next record ID number when the record is inserted to the blockchain state. The only extra requirement here is that we can sync that supplementary DB when doing a blockchain sync.
• ADR 038: State Listening is rather for off-chain use - it allows dapps to get more domain oriented data without querying the Blockchain (eg trade timeseries, ...).

Would we use a kv-store or why not just sqlite?

That would depend on the module. I think for orm it would make sense to use sqlite for things like MultiKeyIndex.

[tac0turtle]

Talked briefly with Robert about this, the concern I brought up is around state sync. If you plan on allowing state sync you will need to chuck this data as well, otherwise the app is non-deterministic. This can be done but has some trade offs.

How would this differ from using an oracle module for when the data is needed?

[robert-zaremba]

Yes, this will need integration with state sync.

How would this differ from using an oracle module for when the data is needed?

Oracle is not managed by a state machine. This proposal is a mechanism for a support DB controlled by the state machine.

[ryanchristo]

Any further discussion around this should move to the sdk. It looks like related discussions are happening in the following:

• Table Store (aka ORM) Package cosmos/cosmos-sdk#9156
• Store Decoding cosmos/cosmos-sdk#9158
• Synchronize state to an external database cosmos/cosmos-sdk#7099