Decouple the compile-time and runtime artifacts of offchain state

[45930]

Summary

The way that offchain state works today is a developer will create their custom offchain state class and object in on call:

const offchainState = OffchainState(
  {
    someMap: OffchainState.Map(PublicKey, UInt64),
    someValue: OffchainState.Field(UInt64),
  },
  { logTotalCapacity: 10, maxActionsPerProof: 5 }
);

This object is then passed into the definition of the smart contract, e.g. in the settle method:

@method
async settle(proof: StateProof) {
  await offchainState.settle(proof);
}

Because the specific object called offchainState is referenced in the smart contract, the object and the class become linked. Every instance of the smart contract class will reference the same object offchainState.

Where has this caused issues?

There is a community builder working on a project that has run into this issue: https://discord.com/channels/484437221055922177/1287933364007079967

They have duplicates of the same smart contract with which they want to use offchain state.

I also ran into this problem when trying to write isolated unit tests for an app that uses offchain state.

[45930]

Proposed API

I think a more ergonomic API would use the OffchainState function to create a new class, rather than an instance.

class MyOffchainState extends OffchainState(
  {
    someMap: OffchainState.Map(PublicKey, UInt64),
    someValue: OffchainState.Field(UInt64),
  },
  { logTotalCapacity: 10, maxActionsPerProof: 5 }
) {}

class MySmartContract extends SmartContract {
  @state(MyOffchainState.Commitments) offchainState = MyOffchainState.initialCommitments();
  
  @method
  async settle(proof: MyOffchainState.Proof) {
    proof.verify();
    const commitments = this.offchainState.getAndRequireEquals();
    proof.publicInputs.commitments.assertEqual(commitments);
    this.offchainState.set(proof.publicOutput.commitments);
  }
}

By using a class in the compilation of the offchain state zk program and the smart contract, the developer could have many instances of MyOffchainState handled in one js thread that each go to different smart contract instances.

[mitschabaude]

I agree on the direction! But in your proposed API, where/how is this.offchainState defined?

[45930]

@mitschabaude

@state(MyOffchainState.Commitments) offchainState = MyOffchainState.initialCommitments();

It is only the commitment that are referenced in the smart contract. Then the settle method accepts a rollup proof that must be the right kind of rollup proof. I'm not sure if my proposal is enough to ensure that.

[mitschabaude]

Ah I see, I missed that!

But then, what about the API for getting and setting offchain state fields?

[mitschabaude]

Somehow, for doing offchain state operations, the smart contract needs access to the Merkle map that the offchain state is kept in.

Currently, there is just a single Merkle map per offchain state "class", but I think we want to avoid that - because the Merkle map is created from actions fetched from a particular account, so there should never be two different accounts used with the same Merkle map.

A contract instance is basically a representation of zkapp account, so it makes sense to require a 1:1 correspondence between contract instances and offchain Merkle maps.
So I think, when instantiating a contract, we should store an "offchain state instance" on it, which holds the Merkle map, similar to how it's done with reducer

[mitschabaude]

How about this @45930?

// not a class, because users don't need any features of classes like adding methods
const offchainState = OffchainState(
  {
    someMap: OffchainState.Map(PublicKey, UInt64),
    someValue: OffchainState.Field(UInt64),
  },
  { logTotalCapacity: 10, maxActionsPerProof: 5 }
);

class StateProof extends offchainState.Proof {}

class MySmartContract extends SmartContract {
  @state(MyOffchainState.Commitments) offchainCommitments = MyOffchainState.initialCommitments();

  // instantiate for this contract
  // replaces previous `setContractClass()` / `setContractInstance()`
  offchainState = offchainState(MySmartContract);
  
  @method.returns(UInt64)
  async getBalance(address: PublicKey) {
    return (await this.offchainState.fields.accounts.get(address)).orElse(0n);
  }

  @method
  async settle(proof: StateProof) {
    await this.offchainState.settle(proof);
  }
}

[45930]

Ah right right right...

So a non-provable property like

class MySmartContract extends SmartContract {
  @state(MyOffchainState.Commitments) offchainState = MyOffchainState.initialCommitments();

  _offchainState = MyOffchainState.default(); // offchainState is a reserved keyword to implement an offchain state contract, but we can update the naming as well
  

But do we even need that? If we point to an existing deployed smart contract that we haven't caught up with yet, then we can sync the actions:

o1js/src/lib/mina/actions/offchain-state.ts

Line 219 in 3ef4f6f

let { merkleMap, valueMap } = await fetchMerkleMap(

So we don't necessarily need to track a local version. We could instead emit actions that are especially defined by the MyOffchainState class, then any instance of MyOffchainState could read the latest data from archive, rather than reference its own local storage.

[mitschabaude]

So we don't necessarily need to track a local version.

I think it's important to keep the design capable of basic efficiency, i.e. not fetch all actions every time we want access to a state field. So I do think we want a local store.

[45930]

not fetch all actions every time

This is fair enough, but also breaks down if more than one party operates the contract right? We can't guarantee that all actions will be routed through our operator, and not someone else's.

In any case, I think we can keep the local storage, but make it a little easier to reset or edit the offchainState.internal state.

[mitschabaude]

In any case, I think we can keep the local storage, but make it a little easier to reset or edit the offchainState.internal state.

Yeah and there should be a method to re-sync the local store to any new actions from the network

[45930]

How about this

Yeah that's clean!