Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: emit nullifiers in set.get_notes #4940

Closed
wants to merge 17 commits into from
Closed

Conversation

nventuro
Copy link
Contributor

@nventuro nventuro commented Mar 4, 2024

Closes #4346.

This changes PrivateSet (previously called Set) so that it emits nullifiers in get_notes. This is to ensure the notes are active: the old API made it possible to 'get' notes that had been nulllified, which is highly non-obvious to someone who is not an expert at Aztec.

Almost all of the callsites emitted nullifiers immediately after using the notes, so these were relatively easy changes. They seemed to rely on a call to remove failing unless some note had been actually retrieved, so I added some is_some assertions. We may want to expand the note getter API to also provide a minimum number of notes that must be returned: I created #4988 for this. Some callsites also made me think that we want to rename this to pop_notes (#4989).

Some of the weird callsites are the inclusion proof tests/examples, which don't really need get_notes since they are already proving inclusion themselves - I adapted them to use view_notes instead. This meant however that we can't rely on the options (e.g. selects) being enforced, so I had to manually add assertions - this was also not obvious, and the type of issue that noir-lang/noir#4442 would help prevent. My overall impression is that the current API is not good enough for the historical inclusion proof use case: we may want a version of get_notes that performs content assertions but does not check existance, or an altogether different inclusion proof API in which we e.g. say 'prove that a note that passes these selects, filters, etc. exists'.

The final strange callsite is the escrow contract, which uses the set like a Singleton (now PrivateMutable). I addressed this in #4942.

Finally, PrivateSet no longer allows querying for nullified notes (because it attempts to nullify them). We're not using this anywhere, but it also hints at the API not being stellar for the historical inclusion proof usecase.


I also removed some doc delimiters that we're not currently using.

nventuro added a commit that referenced this pull request Mar 5, 2024
Working on #4940 I
found that this contract uses `PrivateSet` (aka `Set`), when
`PrivateImmutable` would actually be a much better fit. The escrow was
reading set notes without nullifying them because it doesn't ever remove
notes from the set (and hence all notes that were ever created are
always valid). This can be very confusing for a beginner, and was a bad
example to showcase.

I also removed the docs decorators since we don't ever reference them.
@nventuro nventuro changed the title Emit nullifiers in set.get_notes feat!: emit nullifiers in set.get_notes Mar 5, 2024
@nventuro
Copy link
Contributor Author

nventuro commented Mar 6, 2024

Currently blocked by noir-lang/noir#4497.

@LHerskind LHerskind mentioned this pull request Mar 8, 2024
@nventuro nventuro marked this pull request as ready for review May 7, 2024 20:14
@nventuro
Copy link
Contributor Author

nventuro commented May 9, 2024

Blocked by noir-lang/noir#5008.

@nventuro
Copy link
Contributor Author

I thought noir-lang/noir#5008 was a blocker, but it was just a crash instead of the compiler reporting that we're doing something invalid: we can't have mutable references go from constrained into unconstrained. This is a big problem because our state variables hold Context objects, which in turn hold Option<mut &PrivateContext> (and public and avm). This means that any state variable function (such as view_notes) which takes self cannot be called from a constrained Noir function.

The way I addressed that here in 4b723d2 was by simply calling the oracle myself instead of using the aztec-nr code, but this is of course not a viable long-term solution. The contract where I had to do this is not a good example of real usage as it mostly serves as an example for how to construct a note inclusion proof. and does not necesarily represent how a real application might use this API. Regardless, it does slow a flaw in the design.

A possible path forward might be #5886, which would let us have impls of the state variables that hold PrivateContext instead of mut &PrivateContext, and therefore let us properly mark where we need write access and where we don't. This could also be combined with view functions (#6338) to create state-variable objects that have been fed an immutable context, and that way detect at compile-time that users are attempting to call functions that need write access (instead of failing the is_static check in runtime).

@AztecBot
Copy link
Collaborator

AztecBot commented May 10, 2024

Docs Preview

Hey there! 👋 You can check your preview at https://6650cdba34c06844c2779d03--aztec-docs-dev.netlify.app

Copy link
Contributor

@LHerskind LHerskind left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good to me, unclear why it is failing ci though 😬

@@ -44,35 +44,41 @@ impl<Note> PrivateSet<Note> {
// DEPRECATED
fn assert_contains_and_remove(_self: Self, _note: &mut Note, _nonce: Field) {
assert(
false, "`assert_contains_and_remove` has been deprecated. Please call PXE.addNote() to add a note to the database. Then use PrivateSet.get_notes() and PrivateSet.remove() in your contract to verify and remove a note."
false, "`assert_contains_and_remove` has been deprecated. Please call PXE.addNote() to add a note to the database. Then use PrivateSet.get_notes() in your contract to verify and remove a note."
);
}

// DEPRECATED
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we not get rid of this one if deprecated, why keep it? 👀 better fit for separate issue, just saw it here.

noir-projects/aztec-nr/aztec/src/state_vars/private_set.nr Outdated Show resolved Hide resolved
@AztecBot
Copy link
Collaborator

AztecBot commented May 16, 2024

Benchmark results

Metrics with a significant change:

  • protocol_circuit_proving_time_in_ms (private-kernel-tail): 35,687 (-20%)
  • protocol_circuit_proving_time_in_ms (public-kernel-tail): 152,877 (-16%)
Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Proof generation

Each column represents the number of threads used in proof generation.

Metric 1 threads 4 threads 16 threads 32 threads 64 threads
proof_construction_time_sha256 5,745 1,573 (+1%) 712 767 (+1%) 774

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric 8 txs 32 txs 64 txs
l1_rollup_calldata_size_in_bytes 1,412 1,412 1,412
l1_rollup_calldata_gas 9,464 9,476 9,464
l1_rollup_execution_gas 616,105 616,117 616,105
l2_block_processing_time_in_ms 1,288 4,792 (-1%) 9,533 (-1%)
l2_block_building_time_in_ms 47,504 (+2%) 188,640 (+2%) 376,959 (+2%)
l2_block_rollup_simulation_time_in_ms 47,334 (+2%) 188,010 (+2%) 375,714 (+2%)
l2_block_public_tx_process_time_in_ms 25,298 (+2%) 106,009 (+2%) 216,832 (+3%)

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric 3 blocks 5 blocks
node_history_sync_time_in_ms 9,462 (-1%) 14,486
node_database_size_in_bytes 14,491,728 21,323,856
pxe_database_size_in_bytes 18,071 29,868

Circuits stats

Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit simulation_time_in_ms witness_generation_time_in_ms proving_time_in_ms input_size_in_bytes output_size_in_bytes proof_size_in_bytes num_public_inputs size_in_gates
private-kernel-init 166 (+1%) 3,715 (+5%) 22,165 (-14%) 20,630 64,614 89,536 2,731 1,048,576
private-kernel-inner 631 (+1%) 4,285 (-7%) 48,730 (-2%) 92,318 64,614 89,536 2,731 2,097,152
private-kernel-tail 553 (+1%) 2,927 (-14%) ⚠️ 35,687 (-20%) 90,237 77,498 10,656 266 2,097,152
base-parity 6.45 1,277 (+10%) 2,929 (+3%) 128 64.0 2,208 2.00 131,072
root-parity 49.6 (+1%) 62.3 (-10%) 44,694 (-11%) 27,084 64.0 2,720 18.0 2,097,152
base-rollup 778 (-1%) 2,321 (-3%) 74,520 (-15%) 119,610 756 3,648 47.0 4,194,304
root-rollup 114 (+2%) 68.6 (-12%) 19,304 (-13%) 25,297 620 3,456 41.0 1,048,576
public-kernel-app-logic 530 (+1%) 2,845 47,400 (-6%) 104,941 86,302 114,784 3,520 2,097,152
public-kernel-tail 1,204 (+1%) 23,804 (+1%) ⚠️ 152,877 (-16%) 395,386 7,522 10,656 266 8,388,608
private-kernel-reset-small 601 (+1%) 2,318 (+15%) 43,038 (-15%) 120,733 64,614 89,536 2,731 2,097,152
merge-rollup 28.7 N/A N/A 16,534 756 N/A N/A N/A
public-kernel-setup 633 N/A N/A 104,941 86,302 N/A N/A N/A
public-kernel-teardown 541 (+1%) N/A N/A 104,941 86,302 N/A N/A N/A
private-kernel-tail-to-public N/A 8,590 (+7%) 82,710 (-15%) N/A N/A 114,784 3,520 4,194,304

Stats on running time collected for app circuits

Function input_size_in_bytes output_size_in_bytes witness_generation_time_in_ms proof_size_in_bytes proving_time_in_ms size_in_gates num_public_inputs
ContractClassRegisterer:register 1,344 9,944 465 (-1%) N/A N/A N/A N/A
ContractInstanceDeployer:deploy 1,408 9,944 42.4 (+3%) N/A N/A N/A N/A
MultiCallEntrypoint:entrypoint 1,920 9,944 1,463 (+2%) N/A N/A N/A N/A
SchnorrAccount:constructor 1,312 9,944 1,025 (+2%) N/A N/A N/A N/A
SchnorrAccount:entrypoint 2,304 9,944 2,106 (-1%) 16,768 46,553 (-11%) 2,097,152 457
Token:privately_mint_private_note 1,280 9,944 1,169 (+4%) N/A N/A N/A N/A
Token:transfer 1,376 9,944 3,770 (-8%) 16,768 53,143 (+4%) 2,097,152 457
Benchmarking:create_note 1,312 9,944 978 (-1%) N/A N/A N/A N/A
FPC:fee_entrypoint_public 1,344 9,944 232 (+1%) N/A N/A N/A N/A
SchnorrAccount:spend_private_authwit 1,280 9,944 79.1 (+2%) N/A N/A N/A N/A
Token:unshield 1,376 9,944 2,917 (-13%) N/A N/A N/A N/A
FPC:fee_entrypoint_private 1,376 9,944 3,690 (-12%) N/A N/A N/A N/A

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric 1 leaves 16 leaves 64 leaves 128 leaves 512 leaves 1024 leaves 2048 leaves 4096 leaves 32 leaves
batch_insert_into_append_only_tree_16_depth_ms 10.5 17.0 N/A N/A N/A N/A N/A N/A N/A
batch_insert_into_append_only_tree_16_depth_hash_count 16.7 31.8 N/A N/A N/A N/A N/A N/A N/A
batch_insert_into_append_only_tree_16_depth_hash_ms 0.608 0.521 N/A N/A N/A N/A N/A N/A N/A
batch_insert_into_append_only_tree_32_depth_ms N/A N/A 48.7 76.5 247 (-1%) 477 928 (-1%) 1,838 (-1%) N/A
batch_insert_into_append_only_tree_32_depth_hash_count N/A N/A 95.9 159 543 1,055 2,079 4,127 N/A
batch_insert_into_append_only_tree_32_depth_hash_ms N/A N/A 0.498 0.472 0.448 (-1%) 0.445 (+1%) 0.440 (-1%) 0.439 (-1%) N/A
batch_insert_into_indexed_tree_20_depth_ms N/A N/A 58.5 112 (-1%) 356 (-1%) 700 1,385 (-1%) 2,758 (-1%) N/A
batch_insert_into_indexed_tree_20_depth_hash_count N/A N/A 106 208 692 1,363 2,707 5,395 N/A
batch_insert_into_indexed_tree_20_depth_hash_ms N/A N/A 0.506 0.503 (-1%) 0.483 0.481 0.478 (-1%) 0.479 (-1%) N/A
batch_insert_into_indexed_tree_40_depth_ms N/A N/A N/A N/A N/A N/A N/A N/A 62.5
batch_insert_into_indexed_tree_40_depth_hash_count N/A N/A N/A N/A N/A N/A N/A N/A 107
batch_insert_into_indexed_tree_40_depth_hash_ms N/A N/A N/A N/A N/A N/A N/A N/A 0.554

Miscellaneous

Transaction sizes based on how many contract classes are registered in the tx.

Metric 0 registered classes 1 registered classes
tx_size_in_bytes 83,794 665,117

Transaction size based on fee payment method

| Metric | |
| - | |

@nventuro nventuro enabled auto-merge (squash) May 23, 2024 18:07
@nventuro
Copy link
Contributor Author

This is now quite outdated: we use BoundedVec instead of arrays (#7050) and can return zero notes (#7621). We may still wish to include a version of pop_notes, but I think for now it's best to close this.

@nventuro nventuro closed this Jul 25, 2024
auto-merge was automatically disabled July 25, 2024 21:43

Pull request was closed

@nventuro nventuro deleted the nv/aztecnr-get_note branch July 25, 2024 21:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

refactor(Notes): Split get_note into get_note and get_note_unsafe
4 participants