Skip to content

Commit

Permalink
feat: squash transient note logs (#6268)
Browse files Browse the repository at this point in the history
Closes #1641

## This PR:

### Logs as `SideEffect`s -> `LogHash` or `NoteLogHash`
Adds new structs to track logs in the kernel circuits, so we can track
individual log lengths (and assign gas accordingly) and separate logs
linked to notes from generic logs.
The length changes contribute to #4712 but this PR does not close it -
we need some more work in ts to ensure logs from the non-revertible side
actually persist to the tx.

### Adds `note_encrypted_logs`
To aid the kernels/pxe/note dbs in managing note encrypted logs, most of
the changed files are adding a new array of note logs to anything
already containing `.encrypted_logs_<hashes/hash>`.
Since all our current contracts/tests only emit encrypted logs for
notes, the values expected in `.encrypted_logs` have basically moved
over to `.note_encrypted_logs`. A later PR will add code for generic
encrypted logs (see below).
Includes adding logs to the note cache in `client_execution_context` and
removing `LogsCache`, as the note cache handles note specific logs.

### Squashes transient note logs
Removes any logs linked to note hashes nullified in the same tx. This
involves adding note logs to transient hints for the private kernel tail
and chopping them inside the circuit.

### TODOs/Notes

- This PR ended up pretty large, so while we can silo log hashes now, it
would be cleaner to do in a follow up PR.
- There are some small changes to public/AVM contexts which are mostly
to replace `SideEffect` with `LogHash` and a couple of things addressed
in comments below.
- Currently, the only encrypted logs that are emitted are linked to
notes. To allow for generic values emitted as encrypted logs, I'll need
to add a new oracle call and methods, which makes sense to do once we
encrypt inside the circuit (#1139).
- Total logs length (e.g. `encrypted_log_preimages_length` is no longer
required now we track indvidual lengths (it's also not trusted since it
comes from contexts), so it can be removed from the kernels.
- We originally tried to track log lengths the same way as `ts`, which
adds 4 bytes to each nested 'group' of logs:
`L2BlockL2Logs.txLogs = TxL2Logs -> .functionLogs = FunctionL2Logs ->
.logs = <Un>EncryptedL2Log[]`
but this never actually matched up because we accumulate and sort logs
regardless of what call they came from. E.g. 3 groups of
`FunctionL2Logs` would each have +4 length when added to a tx, but we
take the contents, sort them, then add to a single `FunctionL2Logs.logs`
array before assigning to a tx, so end up with a single +4 length. This
length value is only used on L1 to destructure the blob of bytes
representing logs. Any incorrect lengths would lead to a logs hash
mismatch, so we don't need to match them in the circuit anyway. For this
reason, I've removed a few unnecessary +4s from the circuits.
  • Loading branch information
MirandaWood authored May 16, 2024
1 parent 74e98d4 commit 4574877
Show file tree
Hide file tree
Showing 127 changed files with 1,952 additions and 745 deletions.
26 changes: 15 additions & 11 deletions barretenberg/cpp/src/barretenberg/vm/avm_trace/aztec_constants.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ const size_t MAX_NOTE_HASH_READ_REQUESTS_PER_CALL = 32;
const size_t MAX_NULLIFIER_READ_REQUESTS_PER_CALL = 2;
const size_t MAX_NULLIFIER_NON_EXISTENT_READ_REQUESTS_PER_CALL = 2;
const size_t MAX_NULLIFIER_KEY_VALIDATION_REQUESTS_PER_CALL = 1;
const size_t MAX_NOTE_ENCRYPTED_LOGS_PER_CALL = 16;
const size_t MAX_ENCRYPTED_LOGS_PER_CALL = 4;
const size_t MAX_UNENCRYPTED_LOGS_PER_CALL = 4;
const size_t MAX_NEW_NOTE_HASHES_PER_TX = 64;
Expand All @@ -27,6 +28,7 @@ const size_t MAX_NOTE_HASH_READ_REQUESTS_PER_TX = 128;
const size_t MAX_NULLIFIER_READ_REQUESTS_PER_TX = 8;
const size_t MAX_NULLIFIER_NON_EXISTENT_READ_REQUESTS_PER_TX = 8;
const size_t MAX_NULLIFIER_KEY_VALIDATION_REQUESTS_PER_TX = 4;
const size_t MAX_NOTE_ENCRYPTED_LOGS_PER_TX = 64;
const size_t MAX_ENCRYPTED_LOGS_PER_TX = 8;
const size_t MAX_UNENCRYPTED_LOGS_PER_TX = 8;
const size_t NUM_ENCRYPTED_LOGS_HASHES_PER_TX = 1;
Expand Down Expand Up @@ -94,6 +96,8 @@ const size_t NULLIFIER_KEY_VALIDATION_REQUEST_LENGTH = 3;
const size_t SCOPED_NULLIFIER_KEY_VALIDATION_REQUEST_LENGTH = NULLIFIER_KEY_VALIDATION_REQUEST_LENGTH + 1;
const size_t PARTIAL_STATE_REFERENCE_LENGTH = 6;
const size_t READ_REQUEST_LENGTH = 2;
const size_t LOG_HASH_LENGTH = 3;
const size_t NOTE_LOG_HASH_LENGTH = 4;
const size_t NOTE_HASH_LENGTH = 2;
const size_t SCOPED_NOTE_HASH_LENGTH = NOTE_HASH_LENGTH + 2;
const size_t NULLIFIER_LENGTH = 3;
Expand All @@ -112,17 +116,17 @@ const size_t PRIVATE_CIRCUIT_PUBLIC_INPUTS_LENGTH =
(NOTE_HASH_LENGTH * MAX_NEW_NOTE_HASHES_PER_CALL) + (NULLIFIER_LENGTH * MAX_NEW_NULLIFIERS_PER_CALL) +
MAX_PRIVATE_CALL_STACK_LENGTH_PER_CALL + MAX_PUBLIC_CALL_STACK_LENGTH_PER_CALL + 1 +
(L2_TO_L1_MESSAGE_LENGTH * MAX_NEW_L2_TO_L1_MSGS_PER_CALL) + 2 +
(SIDE_EFFECT_LENGTH * MAX_ENCRYPTED_LOGS_PER_CALL) + (SIDE_EFFECT_LENGTH * MAX_UNENCRYPTED_LOGS_PER_CALL) + 2 +
HEADER_LENGTH + TX_CONTEXT_LENGTH;
(NOTE_LOG_HASH_LENGTH * MAX_NOTE_ENCRYPTED_LOGS_PER_CALL) + (LOG_HASH_LENGTH * MAX_ENCRYPTED_LOGS_PER_CALL) +
(LOG_HASH_LENGTH * MAX_UNENCRYPTED_LOGS_PER_CALL) + 2 + HEADER_LENGTH + TX_CONTEXT_LENGTH;
const size_t PUBLIC_CIRCUIT_PUBLIC_INPUTS_LENGTH =
CALL_CONTEXT_LENGTH + 2 + (READ_REQUEST_LENGTH * MAX_NULLIFIER_READ_REQUESTS_PER_CALL) +
(READ_REQUEST_LENGTH * MAX_NULLIFIER_NON_EXISTENT_READ_REQUESTS_PER_CALL) +
(CONTRACT_STORAGE_UPDATE_REQUEST_LENGTH * MAX_PUBLIC_DATA_UPDATE_REQUESTS_PER_CALL) +
(CONTRACT_STORAGE_READ_LENGTH * MAX_PUBLIC_DATA_READS_PER_CALL) + MAX_PUBLIC_CALL_STACK_LENGTH_PER_CALL +
(NOTE_HASH_LENGTH * MAX_NEW_NOTE_HASHES_PER_CALL) + (NULLIFIER_LENGTH * MAX_NEW_NULLIFIERS_PER_CALL) +
(L2_TO_L1_MESSAGE_LENGTH * MAX_NEW_L2_TO_L1_MSGS_PER_CALL) + 2 +
(SIDE_EFFECT_LENGTH * MAX_UNENCRYPTED_LOGS_PER_CALL) + 1 + HEADER_LENGTH + GLOBAL_VARIABLES_LENGTH +
AZTEC_ADDRESS_LENGTH + /* revert_code */ 1 + 2 * GAS_LENGTH + /* transaction_fee */ 1;
(L2_TO_L1_MESSAGE_LENGTH * MAX_NEW_L2_TO_L1_MSGS_PER_CALL) + 2 + (LOG_HASH_LENGTH * MAX_UNENCRYPTED_LOGS_PER_CALL) +
1 + HEADER_LENGTH + GLOBAL_VARIABLES_LENGTH + AZTEC_ADDRESS_LENGTH + /* revert_code */ 1 + 2 * GAS_LENGTH +
/* transaction_fee */ 1;
const size_t PRIVATE_CALL_STACK_ITEM_LENGTH =
AZTEC_ADDRESS_LENGTH + FUNCTION_DATA_LENGTH + PRIVATE_CIRCUIT_PUBLIC_INPUTS_LENGTH;
const size_t PUBLIC_CONTEXT_INPUTS_LENGTH =
Expand All @@ -137,24 +141,24 @@ const size_t VALIDATION_REQUESTS_LENGTH =
(PUBLIC_DATA_READ_LENGTH * MAX_PUBLIC_DATA_READS_PER_TX);
const size_t PUBLIC_DATA_UPDATE_REQUEST_LENGTH = 2;
const size_t COMBINED_ACCUMULATED_DATA_LENGTH =
MAX_NEW_NOTE_HASHES_PER_TX + MAX_NEW_NULLIFIERS_PER_TX + MAX_NEW_L2_TO_L1_MSGS_PER_TX + 4 +
MAX_NEW_NOTE_HASHES_PER_TX + MAX_NEW_NULLIFIERS_PER_TX + MAX_NEW_L2_TO_L1_MSGS_PER_TX + 5 +
(MAX_PUBLIC_DATA_UPDATE_REQUESTS_PER_TX * PUBLIC_DATA_UPDATE_REQUEST_LENGTH) + GAS_LENGTH;
const size_t COMBINED_CONSTANT_DATA_LENGTH = HEADER_LENGTH + TX_CONTEXT_LENGTH + GLOBAL_VARIABLES_LENGTH;
const size_t CALLER_CONTEXT_LENGTH = 2 * AZTEC_ADDRESS_LENGTH;
const size_t CALL_REQUEST_LENGTH = 1 + AZTEC_ADDRESS_LENGTH + CALLER_CONTEXT_LENGTH + 2;
const size_t PRIVATE_ACCUMULATED_DATA_LENGTH =
(SCOPED_NOTE_HASH_LENGTH * MAX_NEW_NOTE_HASHES_PER_TX) + (SCOPED_NULLIFIER_LENGTH * MAX_NEW_NULLIFIERS_PER_TX) +
(MAX_NEW_L2_TO_L1_MSGS_PER_TX * SCOPED_L2_TO_L1_MESSAGE_LENGTH) + (SIDE_EFFECT_LENGTH * MAX_ENCRYPTED_LOGS_PER_TX) +
(SIDE_EFFECT_LENGTH * MAX_UNENCRYPTED_LOGS_PER_TX) + 2 +
(CALL_REQUEST_LENGTH * MAX_PRIVATE_CALL_STACK_LENGTH_PER_TX) +
(MAX_NEW_L2_TO_L1_MSGS_PER_TX * SCOPED_L2_TO_L1_MESSAGE_LENGTH) +
(NOTE_LOG_HASH_LENGTH * MAX_NOTE_ENCRYPTED_LOGS_PER_TX) + (LOG_HASH_LENGTH * MAX_ENCRYPTED_LOGS_PER_TX) +
(LOG_HASH_LENGTH * MAX_UNENCRYPTED_LOGS_PER_TX) + 2 + (CALL_REQUEST_LENGTH * MAX_PRIVATE_CALL_STACK_LENGTH_PER_TX) +
(CALL_REQUEST_LENGTH * MAX_PUBLIC_CALL_STACK_LENGTH_PER_TX);
const size_t PRIVATE_KERNEL_CIRCUIT_PUBLIC_INPUTS_LENGTH =
1 + VALIDATION_REQUESTS_LENGTH + PRIVATE_ACCUMULATED_DATA_LENGTH + COMBINED_CONSTANT_DATA_LENGTH +
CALL_REQUEST_LENGTH + AZTEC_ADDRESS_LENGTH;
const size_t PUBLIC_ACCUMULATED_DATA_LENGTH =
(MAX_NEW_NOTE_HASHES_PER_TX * NOTE_HASH_LENGTH) + (MAX_NEW_NULLIFIERS_PER_TX * NULLIFIER_LENGTH) +
(MAX_NEW_L2_TO_L1_MSGS_PER_TX * 1) + (MAX_ENCRYPTED_LOGS_PER_TX * SIDE_EFFECT_LENGTH) +
(MAX_UNENCRYPTED_LOGS_PER_TX * SIDE_EFFECT_LENGTH) + 2 +
(MAX_NEW_L2_TO_L1_MSGS_PER_TX * 1) + (NOTE_LOG_HASH_LENGTH * MAX_NOTE_ENCRYPTED_LOGS_PER_TX) +
(MAX_ENCRYPTED_LOGS_PER_TX * LOG_HASH_LENGTH) + (MAX_UNENCRYPTED_LOGS_PER_TX * LOG_HASH_LENGTH) + 2 +
(MAX_PUBLIC_DATA_UPDATE_REQUESTS_PER_TX * PUBLIC_DATA_UPDATE_REQUEST_LENGTH) +
(MAX_PUBLIC_CALL_STACK_LENGTH_PER_TX * CALL_REQUEST_LENGTH) + GAS_LENGTH;
const size_t PUBLIC_KERNEL_CIRCUIT_PUBLIC_INPUTS_LENGTH =
Expand Down
28 changes: 16 additions & 12 deletions l1-contracts/src/core/libraries/ConstantsGen.sol
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ library Constants {
uint256 internal constant MAX_NULLIFIER_READ_REQUESTS_PER_CALL = 2;
uint256 internal constant MAX_NULLIFIER_NON_EXISTENT_READ_REQUESTS_PER_CALL = 2;
uint256 internal constant MAX_NULLIFIER_KEY_VALIDATION_REQUESTS_PER_CALL = 16;
uint256 internal constant MAX_NOTE_ENCRYPTED_LOGS_PER_CALL = 16;
uint256 internal constant MAX_ENCRYPTED_LOGS_PER_CALL = 4;
uint256 internal constant MAX_UNENCRYPTED_LOGS_PER_CALL = 4;
uint256 internal constant MAX_NEW_NOTE_HASHES_PER_TX = 64;
Expand All @@ -39,6 +40,7 @@ library Constants {
uint256 internal constant MAX_NULLIFIER_READ_REQUESTS_PER_TX = 8;
uint256 internal constant MAX_NULLIFIER_NON_EXISTENT_READ_REQUESTS_PER_TX = 8;
uint256 internal constant MAX_NULLIFIER_KEY_VALIDATION_REQUESTS_PER_TX = 64;
uint256 internal constant MAX_NOTE_ENCRYPTED_LOGS_PER_TX = 64;
uint256 internal constant MAX_ENCRYPTED_LOGS_PER_TX = 8;
uint256 internal constant MAX_UNENCRYPTED_LOGS_PER_TX = 8;
uint256 internal constant NUM_ENCRYPTED_LOGS_HASHES_PER_TX = 1;
Expand Down Expand Up @@ -123,6 +125,8 @@ library Constants {
NULLIFIER_KEY_VALIDATION_REQUEST_LENGTH + 1;
uint256 internal constant PARTIAL_STATE_REFERENCE_LENGTH = 6;
uint256 internal constant READ_REQUEST_LENGTH = 2;
uint256 internal constant LOG_HASH_LENGTH = 3;
uint256 internal constant NOTE_LOG_HASH_LENGTH = 4;
uint256 internal constant NOTE_HASH_LENGTH = 2;
uint256 internal constant SCOPED_NOTE_HASH_LENGTH = NOTE_HASH_LENGTH + 2;
uint256 internal constant NULLIFIER_LENGTH = 3;
Expand All @@ -143,8 +147,9 @@ library Constants {
+ (NULLIFIER_LENGTH * MAX_NEW_NULLIFIERS_PER_CALL) + MAX_PRIVATE_CALL_STACK_LENGTH_PER_CALL
+ MAX_PUBLIC_CALL_STACK_LENGTH_PER_CALL + 1
+ (L2_TO_L1_MESSAGE_LENGTH * MAX_NEW_L2_TO_L1_MSGS_PER_CALL) + 2
+ (SIDE_EFFECT_LENGTH * MAX_ENCRYPTED_LOGS_PER_CALL)
+ (SIDE_EFFECT_LENGTH * MAX_UNENCRYPTED_LOGS_PER_CALL) + 2 + HEADER_LENGTH + TX_CONTEXT_LENGTH;
+ (NOTE_LOG_HASH_LENGTH * MAX_NOTE_ENCRYPTED_LOGS_PER_CALL)
+ (LOG_HASH_LENGTH * MAX_ENCRYPTED_LOGS_PER_CALL)
+ (LOG_HASH_LENGTH * MAX_UNENCRYPTED_LOGS_PER_CALL) + 2 + HEADER_LENGTH + TX_CONTEXT_LENGTH;
uint256 internal constant PUBLIC_CIRCUIT_PUBLIC_INPUTS_LENGTH = CALL_CONTEXT_LENGTH + 2
+ (READ_REQUEST_LENGTH * MAX_NULLIFIER_READ_REQUESTS_PER_CALL)
+ (READ_REQUEST_LENGTH * MAX_NULLIFIER_NON_EXISTENT_READ_REQUESTS_PER_CALL)
Expand All @@ -153,9 +158,8 @@ library Constants {
+ MAX_PUBLIC_CALL_STACK_LENGTH_PER_CALL + (NOTE_HASH_LENGTH * MAX_NEW_NOTE_HASHES_PER_CALL)
+ (NULLIFIER_LENGTH * MAX_NEW_NULLIFIERS_PER_CALL)
+ (L2_TO_L1_MESSAGE_LENGTH * MAX_NEW_L2_TO_L1_MSGS_PER_CALL) + 2
+ (SIDE_EFFECT_LENGTH * MAX_UNENCRYPTED_LOGS_PER_CALL) + 1 + HEADER_LENGTH
+ GLOBAL_VARIABLES_LENGTH + AZTEC_ADDRESS_LENGTH /* revert_code */ + 1 + 2 * GAS_LENGTH /* transaction_fee */
+ 1;
+ (LOG_HASH_LENGTH * MAX_UNENCRYPTED_LOGS_PER_CALL) + 1 + HEADER_LENGTH + GLOBAL_VARIABLES_LENGTH
+ AZTEC_ADDRESS_LENGTH /* revert_code */ + 1 + 2 * GAS_LENGTH /* transaction_fee */ + 1;
uint256 internal constant PRIVATE_CALL_STACK_ITEM_LENGTH =
AZTEC_ADDRESS_LENGTH + FUNCTION_DATA_LENGTH + PRIVATE_CIRCUIT_PUBLIC_INPUTS_LENGTH;
uint256 internal constant PUBLIC_CONTEXT_INPUTS_LENGTH =
Expand All @@ -170,7 +174,7 @@ library Constants {
+ (PUBLIC_DATA_READ_LENGTH * MAX_PUBLIC_DATA_READS_PER_TX);
uint256 internal constant PUBLIC_DATA_UPDATE_REQUEST_LENGTH = 2;
uint256 internal constant COMBINED_ACCUMULATED_DATA_LENGTH = MAX_NEW_NOTE_HASHES_PER_TX
+ MAX_NEW_NULLIFIERS_PER_TX + MAX_NEW_L2_TO_L1_MSGS_PER_TX + 4
+ MAX_NEW_NULLIFIERS_PER_TX + MAX_NEW_L2_TO_L1_MSGS_PER_TX + 5
+ (MAX_PUBLIC_DATA_UPDATE_REQUESTS_PER_TX * PUBLIC_DATA_UPDATE_REQUEST_LENGTH) + GAS_LENGTH;
uint256 internal constant COMBINED_CONSTANT_DATA_LENGTH =
HEADER_LENGTH + TX_CONTEXT_LENGTH + GLOBAL_VARIABLES_LENGTH;
Expand All @@ -181,19 +185,19 @@ library Constants {
SCOPED_NOTE_HASH_LENGTH * MAX_NEW_NOTE_HASHES_PER_TX
) + (SCOPED_NULLIFIER_LENGTH * MAX_NEW_NULLIFIERS_PER_TX)
+ (MAX_NEW_L2_TO_L1_MSGS_PER_TX * SCOPED_L2_TO_L1_MESSAGE_LENGTH)
+ (SIDE_EFFECT_LENGTH * MAX_ENCRYPTED_LOGS_PER_TX)
+ (SIDE_EFFECT_LENGTH * MAX_UNENCRYPTED_LOGS_PER_TX) + 2
+ (CALL_REQUEST_LENGTH * MAX_PRIVATE_CALL_STACK_LENGTH_PER_TX)
+ (NOTE_LOG_HASH_LENGTH * MAX_NOTE_ENCRYPTED_LOGS_PER_TX)
+ (LOG_HASH_LENGTH * MAX_ENCRYPTED_LOGS_PER_TX) + (LOG_HASH_LENGTH * MAX_UNENCRYPTED_LOGS_PER_TX)
+ 2 + (CALL_REQUEST_LENGTH * MAX_PRIVATE_CALL_STACK_LENGTH_PER_TX)
+ (CALL_REQUEST_LENGTH * MAX_PUBLIC_CALL_STACK_LENGTH_PER_TX);
uint256 internal constant PRIVATE_KERNEL_CIRCUIT_PUBLIC_INPUTS_LENGTH = 1
+ VALIDATION_REQUESTS_LENGTH + PRIVATE_ACCUMULATED_DATA_LENGTH + COMBINED_CONSTANT_DATA_LENGTH
+ CALL_REQUEST_LENGTH + AZTEC_ADDRESS_LENGTH;
uint256 internal constant PUBLIC_ACCUMULATED_DATA_LENGTH = (
MAX_NEW_NOTE_HASHES_PER_TX * NOTE_HASH_LENGTH
) + (MAX_NEW_NULLIFIERS_PER_TX * NULLIFIER_LENGTH) + (MAX_NEW_L2_TO_L1_MSGS_PER_TX * 1)
+ (MAX_ENCRYPTED_LOGS_PER_TX * SIDE_EFFECT_LENGTH)
+ (MAX_UNENCRYPTED_LOGS_PER_TX * SIDE_EFFECT_LENGTH) + 2
+ (MAX_PUBLIC_DATA_UPDATE_REQUESTS_PER_TX * PUBLIC_DATA_UPDATE_REQUEST_LENGTH)
+ (NOTE_LOG_HASH_LENGTH * MAX_NOTE_ENCRYPTED_LOGS_PER_TX)
+ (MAX_ENCRYPTED_LOGS_PER_TX * LOG_HASH_LENGTH) + (MAX_UNENCRYPTED_LOGS_PER_TX * LOG_HASH_LENGTH)
+ 2 + (MAX_PUBLIC_DATA_UPDATE_REQUESTS_PER_TX * PUBLIC_DATA_UPDATE_REQUEST_LENGTH)
+ (MAX_PUBLIC_CALL_STACK_LENGTH_PER_TX * CALL_REQUEST_LENGTH) + GAS_LENGTH;
uint256 internal constant PUBLIC_KERNEL_CIRCUIT_PUBLIC_INPUTS_LENGTH = VALIDATION_REQUESTS_LENGTH
+ PUBLIC_ACCUMULATED_DATA_LENGTH + PUBLIC_ACCUMULATED_DATA_LENGTH + COMBINED_CONSTANT_DATA_LENGTH
Expand Down
Loading

0 comments on commit 4574877

Please sign in to comment.