Hash inputs in JS #310

mitschabaude · 2022-07-30T08:42:38Z

snarkyjs side of JS-compatible hash inputs MinaProtocol/mina#11590
implements computing hash inputs on the JS side:
- Before, we had to convert snarkyjs parties to Party.Checked.t on the OCaml side, where it would use Party.Checked.digest to compute its hash. Under the hood, the most of the logic in Party.Checked.digest is about converting the party to a list of field elements, the "hash input", which is then hashed.
- In this PR, we implement Types.Party.toInput() completely on the JS side, removing a crucial dependency of snarkyjs on mina_base. Party hashing in JS now only depends on OCaml via Poseidon.update which provides the "raw", non-application-specific hashing logic
- toInput is implemented as usual, with a recursive function leveraging the auto-generated party structure
- the PR contains a refactor of parties-helpers.ts which achieves sharing code between most of the functions which walk the auto-generated structure
consistency between the OCaml and JS sides of computing party hashes is enforced in various ways:
1. a new unit test that runs in Mina CI: https://github.com/MinaProtocol/mina/pull/11590/files#diff-9bd2909dd52a761e3aab0e4ad4ed357803f352286e1bdadcab95732b774b97a6R1
2. We still use the old OCaml logic for computing party hashes outside the circuit. This can be removed at some point, but it provides a very tight consistency check for now: https://github.com/o1-labs/snarkyjs/pull/310/files#diff-7df11bef616dc6cacab4cb8ee003eeab5fb37eaa6c39678415965ed52c8a57d3R815
in support for consistent hash inputs, fixes how token symbols are handled in snarkyjs, adds a new TokenSymbol circuit type
unrelatedly, this has some minor changes related to fixing timestamp preconditions (i.e., this is also the snarkyjs companion for Fix timestamp precondition MinaProtocol/mina#11577)
closes Fix setting timestamp precondition #180

mitschabaude

some comments to guide review:

mitschabaude · 2022-07-30T11:22:36Z

src/examples/to-hash-input.ts

@@ -0,0 +1,179 @@
+import {


this is a sort of unit test, which is also run in Mina CI. I left it here just because for the Mina version, I had to remove the TS types, and it's easier to read & modify with the types in place

mitschabaude · 2022-07-30T11:24:15Z

src/lib/fetch.ts

@@ -139,7 +139,7 @@ function toPermission(p: AuthRequired): Permission {
 type FetchedAccount = {
  publicKey: string;
  nonce: string;
-  tokenId: string;
+  token: string;


this file has some unrelated fixes which turned up when debugging snarkyjs transactions against a local Mina node, while debugging timestamp preconditions

Good catch on figuring out these fixes!

mitschabaude · 2022-07-30T11:24:50Z

src/lib/hash.ts

+
+type Input = { fields?: Field[]; packed?: [Field, number][] };
+
+type TokenSymbol = { symbol: string; field: Field };


new TokenSymbol type!

mitschabaude · 2022-07-30T11:25:38Z

src/lib/hash.ts

+ * Convert the {fields, packed} hash input representation to a list of field elements
+ * Random_oracle_input.Chunked.pack_to_fields
+ */
+function packToFields({ fields = [], packed = [] }: Input) {


this is the last step that's necessary to convert the result of toInput to something which can be hashed

mitschabaude · 2022-07-30T11:26:26Z

src/lib/party.ts

-      let fields = Types.Party.toFields(this);
-      return Ledger.hashPartyFromFields(fields);
+      let input = Types.Party.toInput(this);
+      return hashWithPrefix(prefixes.body, packToFields(input));


this is the main achievement of this PR!

Awesome work!

mitschabaude · 2022-07-30T11:27:03Z

src/lib/precondition.ts

@@ -57,8 +57,6 @@ let unimplementedPreconditions: LongKey[] = [
  'account.isNew',
  // this is partially unimplemented because the field is not returned by the local blockchain
  'account.delegate',
-  // this is unimplemented because setting this precondition made the integration test fail
-  'network.timestamp',


fixed the timestamp precondition, so removed it from the list of unimplemented preconditions!

mitschabaude · 2022-07-30T11:27:11Z

src/lib/state.ts

@@ -257,7 +257,7 @@ To write a correct circuit, you must avoid any dependency on the concrete value
        throw Error(
          'fetch can only be called when the State is assigned to a SmartContract @state.'
        );
-      if (Mina.currentTransaction !== undefined)
+      if (Mina.currentTransaction.has())


unrelated fix

mitschabaude · 2022-07-30T11:27:34Z

src/lib/zkapp.ts

-      let party = Party.createSigned(feePayerKey);
+      let feePayerAddress = feePayerKey.toPublicKey();
+      let party = Party.defaultParty(feePayerAddress);
+      party.body.useFullCommitment = Bool(true);


unrelated fix, necessary for intg test

mitschabaude · 2022-07-30T11:28:28Z

src/snarky/parties-helpers.ts

@@ -52,183 +56,115 @@ function asFieldsAndAux<T, TJson>(typeData: Layout, customTypes: CustomTypes) {
  };
 }

-function toJson(typeData: Layout, value: any, customTypes: CustomTypes): any {


lots of refactoring here -- see the new mapReduce function at the bottom of the file, which abstracts what most of the functions here were doing!

bkase

Looks good! Just one naming nit

bkase · 2022-08-01T14:19:59Z

src/snarky/parties-helpers.ts

+  );
+}
+
+type MapReduceSpec<T, R> = {


I think this is semantically more of a fold ala Fields.fold in the ocaml code that’s generated from deriving fields. I think something like FoldSpec and genericFold or layoutFold?

ymekuria · 2022-08-01T16:50:18Z

src/examples/to-hash-input.ts

+let delegate = PrivateKey.random().toPublicKey();
+update.delegate.value = delegate;
+
+party.tokenSymbol.set('BLABLA');


Hahaha! This token symbol is funny.

ymekuria · 2022-08-01T16:57:24Z

src/examples/to-hash-input.ts

+  jsLayout.Party.entries.body.entries.preconditions.entries.network
+);
+let network = party.body.preconditions.network;
+party.network.stakingEpochData.ledger.hash.assertEquals(Field.random());


What does invoking assertEquals on a random Field element do here?

It changes a precondition on the Party. I'm trying to add as much changes to the party here as possible, to not miss a part of it where the hashing logic doesn't agree between JS and OCaml (if I were just using the default party that I start out with, the hash input would be mostly zeros -- so two of those zeros could be switched and we wouldn't notice)

Ok that makes sense.

mitschabaude added 18 commits July 27, 2022 22:26

generalize some logic dealing with generated types

344dc08

timestamp test

33c8d74

fix some fetch issues

e7f383a

remove block time hack

29d9291

generalize toJson

328d5f9

try fix for invalid nonce

fef1164

toInput

f2a5bd7

move timestamp precondition to implemented

1308526

implement pack_to_fields

dfe4311

export type converters for testing

ac8d0aa

expose ocaml-side to_input converters for testing

0b54f02

start testing toInput against ocaml

5451f3f

implement token symbol

9489280

finish hash input tests

c64443e

add more tests & fix sign to_input

7294dfb

fix packToFields

704e139

cleanup

bea057b

make in-circuit party hashing use JS toInput

01febdb

mitschabaude commented Jul 30, 2022

View reviewed changes

mitschabaude mentioned this pull request Jul 30, 2022

Transpose party leaf types #311

Merged

Base automatically changed from feature/composability-rebased to main August 1, 2022 09:12

Merge branch 'main' into feature/compatible-hashing

b1b3913

bkase approved these changes Aug 1, 2022

View reviewed changes

MartinMinkov approved these changes Aug 1, 2022

View reviewed changes

ymekuria approved these changes Aug 1, 2022

View reviewed changes

mitschabaude added 3 commits August 1, 2022 19:17

fix case of zero args

ed9398f

rename mapReduce -> layoutFold

141be00

update bindings

15718f9

mitschabaude merged commit d180ddb into main Aug 1, 2022

mitschabaude deleted the feature/compatible-hashing branch August 1, 2022 21:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hash inputs in JS #310

Hash inputs in JS #310

mitschabaude commented Jul 30, 2022 •

edited

Loading

mitschabaude left a comment

mitschabaude Jul 30, 2022

mitschabaude Jul 30, 2022

MartinMinkov Aug 1, 2022

mitschabaude Jul 30, 2022

mitschabaude Jul 30, 2022

mitschabaude Jul 30, 2022

MartinMinkov Aug 1, 2022

mitschabaude Jul 30, 2022

mitschabaude Jul 30, 2022

mitschabaude Jul 30, 2022

mitschabaude Jul 30, 2022

bkase left a comment

bkase Aug 1, 2022

ymekuria Aug 1, 2022

ymekuria Aug 1, 2022

mitschabaude Aug 1, 2022 •

edited

Loading

ymekuria Aug 1, 2022


		type Input = { fields?: Field[]; packed?: [Field, number][] };

		type TokenSymbol = { symbol: string; field: Field };

Hash inputs in JS #310

Hash inputs in JS #310

Conversation

mitschabaude commented Jul 30, 2022 • edited Loading

mitschabaude left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bkase left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mitschabaude Aug 1, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mitschabaude commented Jul 30, 2022 •

edited

Loading

mitschabaude Aug 1, 2022 •

edited

Loading