feat: Allow nested arrays and vectors in Brillig foreign calls #4404
Conversation
|
I think acir.hpp in barretenberg needs to be updated with the changes in acir.cpp. Keep in mind that you'll need to replace the |
noir/acvm-repo/brillig_vm/src/lib.rs
Outdated
| ForeignCallParam::Single(value) => { | ||
| self.memory.write(*value_index, *value); | ||
| ValueOrArray::MemoryAddress(value_index) => { | ||
| assert!(*value_type == HeapValueType::Simple); |
There was a problem hiding this comment.
Rather than having these asserts on value_type couldn't we match against (destination, value_type) and then have one failure case for the match statement about having a mismatched type.
There was a problem hiding this comment.
Ah, that's a good idea! Thanks!
noir/acvm-repo/brillig_vm/src/lib.rs
Outdated
| match input { | ||
| ValueOrArray::MemoryAddress(value_index) => self.memory.read(value_index).into(), | ||
| ValueOrArray::MemoryAddress(value_index) => { | ||
| assert!(*value_type == HeapValueType::Simple); |
There was a problem hiding this comment.
Same comment about the asserts on value_type as in process_opcode
|
Some minor comments, overall looks ready to merge after the serialization is updated. |
Addressing the code review comments, match on value and type simultaneously instead of just the value and then asserting on the type.
| std::vector<Circuit::Opcode> opcodes; | ||
| Circuit::ExpressionWidth expression_width; | ||
| std::vector<Circuit::Witness> private_parameters; | ||
| Circuit::PublicInputs public_parameters; | ||
| Circuit::PublicInputs return_values; | ||
| std::vector<std::tuple<Circuit::OpcodeLocation, std::string>> assert_messages; |
There was a problem hiding this comment.
I think this changes need to be removed! Also it's missing passing the formatter on the whole file
There was a problem hiding this comment.
To clarify, it should look like the previous diff without the Circuit:: prefixes.
There was a problem hiding this comment.
Hmm... I thought this would have the same contents as the generated acir.cpp with the previously mentioned replacement only. Do you have the procedure to update this file documented somewhere?
There was a problem hiding this comment.
@ggiraldez unfortunately the documentation on this process is lacking. The replacement mentioned here is the only other place. I'm going to quickly do the serialization for you and also make an issue to document this process.
There was a problem hiding this comment.
I think I was able to fix this one, and found the format.sh script in the directory. I'll push a commit with the changes, but I'm unsure it's working properly. Locally, I managed to get the build dependencies installed for Barrentenberg, but it's still complaining about a different thing somewhere else in the codebase. I'm sure I'm missing some language dependency or something.
There was a problem hiding this comment.
Ok the committed changes look correct. Let's see what CI says. I will document the process for any future updates
Head branch was pushed to by a user without write access
This PR mirrors the changes of #4404 to make them easier to merge --------- Co-authored-by: Gustavo Giráldez <[email protected]>
|
Merged here #4478 |
Thank you! |
This PR mirrors the changes of #4404 to make them easier to merge --------- Co-authored-by: Gustavo Giráldez <[email protected]>
This PR mirrors the changes of AztecProtocol/aztec-packages#4404 to make them easier to merge --------- Co-authored-by: Gustavo Giráldez <[email protected]>
…Protocol#4478) This PR mirrors the changes of AztecProtocol#4404 to make them easier to merge --------- Co-authored-by: Gustavo Giráldez <[email protected]>
Description
This is a mirror of noir-lang/noir#4053. From the discussion there, it was suggested that we should open the PR on this repository instead, due to the changes in the serialization format.
Problem*
Resolves noir-lang/noir#4052
The Brillig VM has no knowledge of the data types of array/slices elements, but it also references arrays and vectors by a pointer to a small structure in memory. This results in incorrect data passed to foreign functions when the values consist of arrays (or slices) nested inside other arrays (or slices).
Summary*
This PR adds element type descriptors to
HeapArrayandHeapVectorBrillig structures to allow the code that gathers the foreign call parameters to correctly interpret the data to be read from the VM memory.Additional Context
The counterpart of is returning nested arrays/slices from foreign calls, which this PR does not implement. The existing foreign functions (and the ones required by the debugger oracle) do not require this, so it's not currently a problem.
Furthermore, implementing that feature would require some design decisions regarding how are the arrays/slices allocated in memory and by who.
To return arrays the compiler now generates code to pre-allocate the heap array, adjusting the heap pointer register and setting a register for the VM to write the results into memory. For nested arrays, it would need to pre-allocate the outer as well as the inner arrays and write the pointers to the inner arrays into memory, all before making the foreign call.
When adding slices (heap vectors) to the mix, it gets more complicated because they are dynamically sized, so it wouldn't be possible for the compiler to pre-allocate the inner arrays/slices. Also, for slices the compiler will allocate a register for the VM to write the size of the returned result, which it uses to generate code to adjust the heap pointer register after the call is complete.
It's unclear whose responsibility would be to allocate heap space when mixing arrays and slices, and the compiled code should handle it. A simpler approach may be for the compiler to generate the foreign call passing the heap pointer register and delegating all the allocations and heap pointer adjustments to the VM implementation. That would simplify generated Brillig code and move the burden on to the VM implementation.
Documentation*
Check one:
PR Checklist*
cargo fmton default settings.