Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate prover and verifier keys in CompressedSNARK #145

Merged
merged 8 commits into from
Mar 3, 2023
Merged

Conversation

srinathsetty
Copy link
Collaborator

No description provided.

@srinathsetty srinathsetty merged commit 1e49952 into main Mar 3, 2023
@srinathsetty srinathsetty deleted the key branch March 3, 2023 02:36
huitseeker added a commit to huitseeker/Nova that referenced this pull request Dec 9, 2023
* Schoolbook adapted KZG PoC

Remove toxic waste from setup parameters

Refactorings

ZeroMorph skeleton

* feat: provide a non-hiding variant of KZG

Drafts a non-hiding variant of the KZG PCS on univariate Dense polynomial representations

Details:
- Changed access levels and ordering for various modules and functions like `sumcheck`, `cpu_best_multiexp`, and `mod kzg;`.
- Upgraded `halo2curves` version and added new dependencies like `pairing`, `anyhow`, `rand`, and `group`.
- Leveraged the UniPoly in sumcheck.rs
- Created `non_hiding_kzg.rs` file, introducing new structures and functionalities like `UVUniversalKZGParam`, `UVKZGProverKey`, `UVKZGPoly`, and `UVKZGPCS` along with their implementation and tests.
- Modified the import of `Engine` in `kzg.rs` from `halo2curves::pairing::Engine` to `pairing::Engine` (reflecting the new version of halo2curves).

feat: add batch commit / open / prove

* Small subtasks: params, phi, un function, EE

Minor adjustments

* chore: setup zeromorph

refactor: Remove kzg and zeromorph provider modules

* feat: add openings for Zeromorph

- Added new data structures and their related functions including the `ZMProverKey`, `ZMVerifierKey`, `ZMCommitment`, `ZMEvaluation`, `ZMProof` and `ZMPCS` in `non_hiding_zeromorph.rs`
- Implemented new functionalities in `non_hiding_zeromorph.rs` and `spartan/polynomial.rs` to provide better handling of polynomials, including fetching commitments for a polynomial, computing the quotient polynomials of an existing polynomial.
- Enhanced the `UniPoly` struct in `spartan/sumcheck.rs` with `Index` and `IndexMut` for better coefficient access, and `AddAssign` and `MulAssign` for scalar and self types.
- Removed and replaced certain elements in `UVUniversalKZGParam` struct in `non_hiding_kzg.rs` for improved flexibility, and included import and implementation of `TranscriptReprTrait`.

draft ZM verify

set up test structure

defer test_quo

* Include commits into ZMProof and adjust RO

* feat: Refactor polynomial evaluation methods

- Temporarily disabled some assertions in both `non_hiding_zeromorph.rs` and `polynomial.rs` for debugging purposes.
- Introduced `evaluate_opt`, `fix_variables`, and `fix_one_variable_helper` functions in `polynomial.rs` to support multilinear polynomial evaluation and partial evaluation.

fix: add domain separators

- Integrated additional functionality into the ZMPCS implementation in `non_hiding_zeromorph.rs` by adding a protocol name function.
- Improved security in `non_hiding_zeromorph.rs` by integrating domain separators into transcript's `open` and `verify` functions.

refactor: Refactor code and test performance for NHZM

- Implement `AsRef` trait for `UniPoly` structure in `src/spartan/sumcheck.rs` to facilitate direct access to its coefficients.
- Refactor code in `src/provider/non_hiding_zeromorph.rs` to directly use `into_iter` in map function when creating `quotients_polys`, avoiding a large clone
- Enhance test performance in `src/provider/non_hiding_zeromorph.rs` by pre-generating a `universal_setup` for tests, introducing a `max_vars` variable and RNG in the `commit_open_verify_with` test function, and adjusting the range of num_vars used for testing.

fix: adjust APIs & Serialize impls for the Nova traits

* feat: set up serde & abomonation bounds

* add generic PCS errors

* pp

wip

* fix: adjust type parameters for nontrivial_with_zm test

fix: ignore panicking test

fix: add doctest for evaluate_opt

fix: remove obsolete comments

chore: move UniPoly methods where they should be

test: make clear current zeromorph operates in monomial basis

- Added an additional test `test_dense_evaliations()` to provide more comprehensive testing for the `evaluate()` and `evaluate_opt()` functions in `MultilinearPolynomial`.

refactor TranscriptReprTrait impl for compat with Commitments

* feat: Implement KZG commitment trait and serialization features

- Added serialization, deserialization, and Abomonation traits to UVUniversalKZGParam struct in `non_hiding_kzg.rs` file along with implementing comparison and length evaluation traits.
- Created a new file `kzg_commitment.rs` which implements KZG Commitment Engine and setup, commit functions.
- Integrated `kzg_commitment` in module `provider` and set up conversions between Commitment and UVKZGCommitment.
- Enhanced assertion in `minroot_serde.rs` file from clone comparison to dereferenced comparison in MinRoot delay case.

* Use the ZMPCS Evaluation Engine and the KZG Commitment Engine in tests.

feat: Improve `prove` and `verify` methods in `ZMPCS` struct

- Updated `prove` and `verify` methods in `ZMPCS<E>` struct within `non_hiding_zeromorph.rs` with actual logic for commitment, evaluation, and verification.
- Adjusted the construction of `ZMCommitment` and `ZMEvaluation` within `prove` and `verify` methods.
- Commented on portions of the code in `non_hiding_zeromorph.rs` that need further modification and refinement.
- Modified test value for `test_pp_digest_with` in the `test_supernova_pp_digest` test case to match the new expected output.

* fix evaluation reversal bug

fix: remove superfluous eval functions

fix: remove endianness shenanigans

test: add evaluation unit test

* fix: parallellize pp generation

* feat: Enhance KZG commitment SRS generation efficiency using parallel computation

- Introduced a new module `util` within the `provider` module, implementing fixed-base MSM,
- New trait constraint has been imposed for `E::Fr: PrimeFieldBits` within the `non_hiding_zeromorph.rs` file, and usages have been adjusted in the `test` module.
- Adding the `PrimeFieldBits` import from the `ff` crate and importing `provider::util` from the local crate.

refactor: Refactor utility functions for elliptic curve operations

- Renamed and moved `util.rs` to `fb_msm.rs` under the `provider` directory.
- documented the FB MSM

* Apply comments from code review

Co-authored-by: Adrian Hamelink <[email protected]>

* Improve documentation and testing in non_hiding_zeromorph.rs

- Enhanced the clarity and accuracy of code comments and documentation in `non_hiding_zeromorph.rs`, specifically within the `ZMPCS` struct's methods.
- Enriched the `quotients` function's documentation, detailing its mathematical underpinnings
- Implemented a new rigorous test, `test_quotients`, to ensure the `quotients` function's output satisfies the polynomial identity.

* test: refactor random ML poly generation

* Finish documenting / testing Zeromorph (microsoft#142)

* fix: remove a TODO using RefCast

* fix: remove unsightly clone

* doc: comment intermediate ZM steps

* refactor: Refactor `open` function in zeromorph provider

Extracted the computation of `q_hat` in the `open` function into a new function `batched_lifted_degree_quotient` for more modular code.

* test: test batched_lifted_degree_quotient

Addition of a new test `test_batched_lifted_degree_quotient` to validate batched degree quotient lifting for polynomials.

* refactor: Refine computations in non_hiding_zeromorph.rs

- Overhauled the 'eval_and_quotient_scalars' function with a revised return type with pair of vectors rather than scalar and a vector,
- correspondingly split scalars `degree_check_q_scalars` and `zmpoly_q_scalars` within the `open` method of `non_hiding_zeromorph.rs`.
- Adjusted the 'verify' method to accommodate the modified 'eval_and_quotient_scalars' function output changes.

* test: test partially evaluated quotient \zeta_x

- Introduced a new test case `test_partially_evaluated_quotient_zeta` to validate `zeta_x` construction.
- correct an off-by-one error in the y_k terms

* test: test partially evaluated quotient

- Created a new function `phi` for evaluating using an inefficient formula in `non_hiding_zeromorph.rs`
- Implemented a new test function `test_partially_evaluated_quotient_z` for validation of `Z_x` computation method in `non_hiding_zeromorph.rs`

* fix: refine q_hat comment

* fix: organize code more sparsely after rebase

* refactor: Improve code readability and error handling in zeromorph

- Updated and enhanced clarity and consistency of mathematical notation in comments across `non_hiding_kzg.rs` and `non_hiding_zeromorph.rs` files.
- Implemented error handling in the `ZMPCS::verify` function within the `non_hiding_zeromorph.rs` file.

* fix: compute quotient commitments in parallel

* refactor: clean up

---------

Co-authored-by: emmorais <[email protected]>
Co-authored-by: Matej Penciak <[email protected]>
Co-authored-by: Adrian Hamelink <[email protected]>
huitseeker added a commit to argumentcomputer/Nova that referenced this pull request Jan 25, 2024
* Schoolbook adapted KZG PoC

Remove toxic waste from setup parameters

Refactorings

ZeroMorph skeleton

* feat: provide a non-hiding variant of KZG

Drafts a non-hiding variant of the KZG PCS on univariate Dense polynomial representations

Details:
- Changed access levels and ordering for various modules and functions like `sumcheck`, `cpu_best_multiexp`, and `mod kzg;`.
- Upgraded `halo2curves` version and added new dependencies like `pairing`, `anyhow`, `rand`, and `group`.
- Leveraged the UniPoly in sumcheck.rs
- Created `non_hiding_kzg.rs` file, introducing new structures and functionalities like `UVUniversalKZGParam`, `UVKZGProverKey`, `UVKZGPoly`, and `UVKZGPCS` along with their implementation and tests.
- Modified the import of `Engine` in `kzg.rs` from `halo2curves::pairing::Engine` to `pairing::Engine` (reflecting the new version of halo2curves).

feat: add batch commit / open / prove

* Small subtasks: params, phi, un function, EE

Minor adjustments

* chore: setup zeromorph

refactor: Remove kzg and zeromorph provider modules

* feat: add openings for Zeromorph

- Added new data structures and their related functions including the `ZMProverKey`, `ZMVerifierKey`, `ZMCommitment`, `ZMEvaluation`, `ZMProof` and `ZMPCS` in `non_hiding_zeromorph.rs`
- Implemented new functionalities in `non_hiding_zeromorph.rs` and `spartan/polynomial.rs` to provide better handling of polynomials, including fetching commitments for a polynomial, computing the quotient polynomials of an existing polynomial.
- Enhanced the `UniPoly` struct in `spartan/sumcheck.rs` with `Index` and `IndexMut` for better coefficient access, and `AddAssign` and `MulAssign` for scalar and self types.
- Removed and replaced certain elements in `UVUniversalKZGParam` struct in `non_hiding_kzg.rs` for improved flexibility, and included import and implementation of `TranscriptReprTrait`.

draft ZM verify

set up test structure

defer test_quo

* Include commits into ZMProof and adjust RO

* feat: Refactor polynomial evaluation methods

- Temporarily disabled some assertions in both `non_hiding_zeromorph.rs` and `polynomial.rs` for debugging purposes.
- Introduced `evaluate_opt`, `fix_variables`, and `fix_one_variable_helper` functions in `polynomial.rs` to support multilinear polynomial evaluation and partial evaluation.

fix: add domain separators

- Integrated additional functionality into the ZMPCS implementation in `non_hiding_zeromorph.rs` by adding a protocol name function.
- Improved security in `non_hiding_zeromorph.rs` by integrating domain separators into transcript's `open` and `verify` functions.

refactor: Refactor code and test performance for NHZM

- Implement `AsRef` trait for `UniPoly` structure in `src/spartan/sumcheck.rs` to facilitate direct access to its coefficients.
- Refactor code in `src/provider/non_hiding_zeromorph.rs` to directly use `into_iter` in map function when creating `quotients_polys`, avoiding a large clone
- Enhance test performance in `src/provider/non_hiding_zeromorph.rs` by pre-generating a `universal_setup` for tests, introducing a `max_vars` variable and RNG in the `commit_open_verify_with` test function, and adjusting the range of num_vars used for testing.

fix: adjust APIs & Serialize impls for the Nova traits

* feat: set up serde & abomonation bounds

* add generic PCS errors

* pp

wip

* fix: adjust type parameters for nontrivial_with_zm test

fix: ignore panicking test

fix: add doctest for evaluate_opt

fix: remove obsolete comments

chore: move UniPoly methods where they should be

test: make clear current zeromorph operates in monomial basis

- Added an additional test `test_dense_evaliations()` to provide more comprehensive testing for the `evaluate()` and `evaluate_opt()` functions in `MultilinearPolynomial`.

refactor TranscriptReprTrait impl for compat with Commitments

* feat: Implement KZG commitment trait and serialization features

- Added serialization, deserialization, and Abomonation traits to UVUniversalKZGParam struct in `non_hiding_kzg.rs` file along with implementing comparison and length evaluation traits.
- Created a new file `kzg_commitment.rs` which implements KZG Commitment Engine and setup, commit functions.
- Integrated `kzg_commitment` in module `provider` and set up conversions between Commitment and UVKZGCommitment.
- Enhanced assertion in `minroot_serde.rs` file from clone comparison to dereferenced comparison in MinRoot delay case.

* Use the ZMPCS Evaluation Engine and the KZG Commitment Engine in tests.

feat: Improve `prove` and `verify` methods in `ZMPCS` struct

- Updated `prove` and `verify` methods in `ZMPCS<E>` struct within `non_hiding_zeromorph.rs` with actual logic for commitment, evaluation, and verification.
- Adjusted the construction of `ZMCommitment` and `ZMEvaluation` within `prove` and `verify` methods.
- Commented on portions of the code in `non_hiding_zeromorph.rs` that need further modification and refinement.
- Modified test value for `test_pp_digest_with` in the `test_supernova_pp_digest` test case to match the new expected output.

* fix evaluation reversal bug

fix: remove superfluous eval functions

fix: remove endianness shenanigans

test: add evaluation unit test

* fix: parallellize pp generation

* feat: Enhance KZG commitment SRS generation efficiency using parallel computation

- Introduced a new module `util` within the `provider` module, implementing fixed-base MSM,
- New trait constraint has been imposed for `E::Fr: PrimeFieldBits` within the `non_hiding_zeromorph.rs` file, and usages have been adjusted in the `test` module.
- Adding the `PrimeFieldBits` import from the `ff` crate and importing `provider::util` from the local crate.

refactor: Refactor utility functions for elliptic curve operations

- Renamed and moved `util.rs` to `fb_msm.rs` under the `provider` directory.
- documented the FB MSM

* Apply comments from code review

Co-authored-by: Adrian Hamelink <[email protected]>

* Improve documentation and testing in non_hiding_zeromorph.rs

- Enhanced the clarity and accuracy of code comments and documentation in `non_hiding_zeromorph.rs`, specifically within the `ZMPCS` struct's methods.
- Enriched the `quotients` function's documentation, detailing its mathematical underpinnings
- Implemented a new rigorous test, `test_quotients`, to ensure the `quotients` function's output satisfies the polynomial identity.

* test: refactor random ML poly generation

* Finish documenting / testing Zeromorph (microsoft#142)

* fix: remove a TODO using RefCast

* fix: remove unsightly clone

* doc: comment intermediate ZM steps

* refactor: Refactor `open` function in zeromorph provider

Extracted the computation of `q_hat` in the `open` function into a new function `batched_lifted_degree_quotient` for more modular code.

* test: test batched_lifted_degree_quotient

Addition of a new test `test_batched_lifted_degree_quotient` to validate batched degree quotient lifting for polynomials.

* refactor: Refine computations in non_hiding_zeromorph.rs

- Overhauled the 'eval_and_quotient_scalars' function with a revised return type with pair of vectors rather than scalar and a vector,
- correspondingly split scalars `degree_check_q_scalars` and `zmpoly_q_scalars` within the `open` method of `non_hiding_zeromorph.rs`.
- Adjusted the 'verify' method to accommodate the modified 'eval_and_quotient_scalars' function output changes.

* test: test partially evaluated quotient \zeta_x

- Introduced a new test case `test_partially_evaluated_quotient_zeta` to validate `zeta_x` construction.
- correct an off-by-one error in the y_k terms

* test: test partially evaluated quotient

- Created a new function `phi` for evaluating using an inefficient formula in `non_hiding_zeromorph.rs`
- Implemented a new test function `test_partially_evaluated_quotient_z` for validation of `Z_x` computation method in `non_hiding_zeromorph.rs`

* fix: refine q_hat comment

* fix: organize code more sparsely after rebase

* refactor: Improve code readability and error handling in zeromorph

- Updated and enhanced clarity and consistency of mathematical notation in comments across `non_hiding_kzg.rs` and `non_hiding_zeromorph.rs` files.
- Implemented error handling in the `ZMPCS::verify` function within the `non_hiding_zeromorph.rs` file.

* fix: compute quotient commitments in parallel

* refactor: clean up

---------

Co-authored-by: emmorais <[email protected]>
Co-authored-by: Matej Penciak <[email protected]>
Co-authored-by: Adrian Hamelink <[email protected]>
huitseeker added a commit to huitseeker/Nova that referenced this pull request Jan 26, 2024
* Schoolbook adapted KZG PoC

Remove toxic waste from setup parameters

Refactorings

ZeroMorph skeleton

* feat: provide a non-hiding variant of KZG

Drafts a non-hiding variant of the KZG PCS on univariate Dense polynomial representations

Details:
- Changed access levels and ordering for various modules and functions like `sumcheck`, `cpu_best_multiexp`, and `mod kzg;`.
- Upgraded `halo2curves` version and added new dependencies like `pairing`, `anyhow`, `rand`, and `group`.
- Leveraged the UniPoly in sumcheck.rs
- Created `non_hiding_kzg.rs` file, introducing new structures and functionalities like `UVUniversalKZGParam`, `UVKZGProverKey`, `UVKZGPoly`, and `UVKZGPCS` along with their implementation and tests.
- Modified the import of `Engine` in `kzg.rs` from `halo2curves::pairing::Engine` to `pairing::Engine` (reflecting the new version of halo2curves).

feat: add batch commit / open / prove

* Small subtasks: params, phi, un function, EE

Minor adjustments

* chore: setup zeromorph

refactor: Remove kzg and zeromorph provider modules

* feat: add openings for Zeromorph

- Added new data structures and their related functions including the `ZMProverKey`, `ZMVerifierKey`, `ZMCommitment`, `ZMEvaluation`, `ZMProof` and `ZMPCS` in `non_hiding_zeromorph.rs`
- Implemented new functionalities in `non_hiding_zeromorph.rs` and `spartan/polynomial.rs` to provide better handling of polynomials, including fetching commitments for a polynomial, computing the quotient polynomials of an existing polynomial.
- Enhanced the `UniPoly` struct in `spartan/sumcheck.rs` with `Index` and `IndexMut` for better coefficient access, and `AddAssign` and `MulAssign` for scalar and self types.
- Removed and replaced certain elements in `UVUniversalKZGParam` struct in `non_hiding_kzg.rs` for improved flexibility, and included import and implementation of `TranscriptReprTrait`.

draft ZM verify

set up test structure

defer test_quo

* Include commits into ZMProof and adjust RO

* feat: Refactor polynomial evaluation methods

- Temporarily disabled some assertions in both `non_hiding_zeromorph.rs` and `polynomial.rs` for debugging purposes.
- Introduced `evaluate_opt`, `fix_variables`, and `fix_one_variable_helper` functions in `polynomial.rs` to support multilinear polynomial evaluation and partial evaluation.

fix: add domain separators

- Integrated additional functionality into the ZMPCS implementation in `non_hiding_zeromorph.rs` by adding a protocol name function.
- Improved security in `non_hiding_zeromorph.rs` by integrating domain separators into transcript's `open` and `verify` functions.

refactor: Refactor code and test performance for NHZM

- Implement `AsRef` trait for `UniPoly` structure in `src/spartan/sumcheck.rs` to facilitate direct access to its coefficients.
- Refactor code in `src/provider/non_hiding_zeromorph.rs` to directly use `into_iter` in map function when creating `quotients_polys`, avoiding a large clone
- Enhance test performance in `src/provider/non_hiding_zeromorph.rs` by pre-generating a `universal_setup` for tests, introducing a `max_vars` variable and RNG in the `commit_open_verify_with` test function, and adjusting the range of num_vars used for testing.

fix: adjust APIs & Serialize impls for the Nova traits

* feat: set up serde & abomonation bounds

* add generic PCS errors

* pp

wip

* fix: adjust type parameters for nontrivial_with_zm test

fix: ignore panicking test

fix: add doctest for evaluate_opt

fix: remove obsolete comments

chore: move UniPoly methods where they should be

test: make clear current zeromorph operates in monomial basis

- Added an additional test `test_dense_evaliations()` to provide more comprehensive testing for the `evaluate()` and `evaluate_opt()` functions in `MultilinearPolynomial`.

refactor TranscriptReprTrait impl for compat with Commitments

* feat: Implement KZG commitment trait and serialization features

- Added serialization, deserialization, and Abomonation traits to UVUniversalKZGParam struct in `non_hiding_kzg.rs` file along with implementing comparison and length evaluation traits.
- Created a new file `kzg_commitment.rs` which implements KZG Commitment Engine and setup, commit functions.
- Integrated `kzg_commitment` in module `provider` and set up conversions between Commitment and UVKZGCommitment.
- Enhanced assertion in `minroot_serde.rs` file from clone comparison to dereferenced comparison in MinRoot delay case.

* Use the ZMPCS Evaluation Engine and the KZG Commitment Engine in tests.

feat: Improve `prove` and `verify` methods in `ZMPCS` struct

- Updated `prove` and `verify` methods in `ZMPCS<E>` struct within `non_hiding_zeromorph.rs` with actual logic for commitment, evaluation, and verification.
- Adjusted the construction of `ZMCommitment` and `ZMEvaluation` within `prove` and `verify` methods.
- Commented on portions of the code in `non_hiding_zeromorph.rs` that need further modification and refinement.
- Modified test value for `test_pp_digest_with` in the `test_supernova_pp_digest` test case to match the new expected output.

* fix evaluation reversal bug

fix: remove superfluous eval functions

fix: remove endianness shenanigans

test: add evaluation unit test

* fix: parallellize pp generation

* feat: Enhance KZG commitment SRS generation efficiency using parallel computation

- Introduced a new module `util` within the `provider` module, implementing fixed-base MSM,
- New trait constraint has been imposed for `E::Fr: PrimeFieldBits` within the `non_hiding_zeromorph.rs` file, and usages have been adjusted in the `test` module.
- Adding the `PrimeFieldBits` import from the `ff` crate and importing `provider::util` from the local crate.

refactor: Refactor utility functions for elliptic curve operations

- Renamed and moved `util.rs` to `fb_msm.rs` under the `provider` directory.
- documented the FB MSM

* Apply comments from code review

Co-authored-by: Adrian Hamelink <[email protected]>

* Improve documentation and testing in non_hiding_zeromorph.rs

- Enhanced the clarity and accuracy of code comments and documentation in `non_hiding_zeromorph.rs`, specifically within the `ZMPCS` struct's methods.
- Enriched the `quotients` function's documentation, detailing its mathematical underpinnings
- Implemented a new rigorous test, `test_quotients`, to ensure the `quotients` function's output satisfies the polynomial identity.

* test: refactor random ML poly generation

* Finish documenting / testing Zeromorph (microsoft#142)

* fix: remove a TODO using RefCast

* fix: remove unsightly clone

* doc: comment intermediate ZM steps

* refactor: Refactor `open` function in zeromorph provider

Extracted the computation of `q_hat` in the `open` function into a new function `batched_lifted_degree_quotient` for more modular code.

* test: test batched_lifted_degree_quotient

Addition of a new test `test_batched_lifted_degree_quotient` to validate batched degree quotient lifting for polynomials.

* refactor: Refine computations in non_hiding_zeromorph.rs

- Overhauled the 'eval_and_quotient_scalars' function with a revised return type with pair of vectors rather than scalar and a vector,
- correspondingly split scalars `degree_check_q_scalars` and `zmpoly_q_scalars` within the `open` method of `non_hiding_zeromorph.rs`.
- Adjusted the 'verify' method to accommodate the modified 'eval_and_quotient_scalars' function output changes.

* test: test partially evaluated quotient \zeta_x

- Introduced a new test case `test_partially_evaluated_quotient_zeta` to validate `zeta_x` construction.
- correct an off-by-one error in the y_k terms

* test: test partially evaluated quotient

- Created a new function `phi` for evaluating using an inefficient formula in `non_hiding_zeromorph.rs`
- Implemented a new test function `test_partially_evaluated_quotient_z` for validation of `Z_x` computation method in `non_hiding_zeromorph.rs`

* fix: refine q_hat comment

* fix: organize code more sparsely after rebase

* refactor: Improve code readability and error handling in zeromorph

- Updated and enhanced clarity and consistency of mathematical notation in comments across `non_hiding_kzg.rs` and `non_hiding_zeromorph.rs` files.
- Implemented error handling in the `ZMPCS::verify` function within the `non_hiding_zeromorph.rs` file.

* fix: compute quotient commitments in parallel

* refactor: clean up

---------

Co-authored-by: emmorais <[email protected]>
Co-authored-by: Matej Penciak <[email protected]>
Co-authored-by: Adrian Hamelink <[email protected]>
huitseeker added a commit to argumentcomputer/Nova that referenced this pull request Jan 26, 2024
* Schoolbook adapted KZG PoC

Remove toxic waste from setup parameters

Refactorings

ZeroMorph skeleton

* feat: provide a non-hiding variant of KZG

Drafts a non-hiding variant of the KZG PCS on univariate Dense polynomial representations

Details:
- Changed access levels and ordering for various modules and functions like `sumcheck`, `cpu_best_multiexp`, and `mod kzg;`.
- Upgraded `halo2curves` version and added new dependencies like `pairing`, `anyhow`, `rand`, and `group`.
- Leveraged the UniPoly in sumcheck.rs
- Created `non_hiding_kzg.rs` file, introducing new structures and functionalities like `UVUniversalKZGParam`, `UVKZGProverKey`, `UVKZGPoly`, and `UVKZGPCS` along with their implementation and tests.
- Modified the import of `Engine` in `kzg.rs` from `halo2curves::pairing::Engine` to `pairing::Engine` (reflecting the new version of halo2curves).

feat: add batch commit / open / prove

* Small subtasks: params, phi, un function, EE

Minor adjustments

* chore: setup zeromorph

refactor: Remove kzg and zeromorph provider modules

* feat: add openings for Zeromorph

- Added new data structures and their related functions including the `ZMProverKey`, `ZMVerifierKey`, `ZMCommitment`, `ZMEvaluation`, `ZMProof` and `ZMPCS` in `non_hiding_zeromorph.rs`
- Implemented new functionalities in `non_hiding_zeromorph.rs` and `spartan/polynomial.rs` to provide better handling of polynomials, including fetching commitments for a polynomial, computing the quotient polynomials of an existing polynomial.
- Enhanced the `UniPoly` struct in `spartan/sumcheck.rs` with `Index` and `IndexMut` for better coefficient access, and `AddAssign` and `MulAssign` for scalar and self types.
- Removed and replaced certain elements in `UVUniversalKZGParam` struct in `non_hiding_kzg.rs` for improved flexibility, and included import and implementation of `TranscriptReprTrait`.

draft ZM verify

set up test structure

defer test_quo

* Include commits into ZMProof and adjust RO

* feat: Refactor polynomial evaluation methods

- Temporarily disabled some assertions in both `non_hiding_zeromorph.rs` and `polynomial.rs` for debugging purposes.
- Introduced `evaluate_opt`, `fix_variables`, and `fix_one_variable_helper` functions in `polynomial.rs` to support multilinear polynomial evaluation and partial evaluation.

fix: add domain separators

- Integrated additional functionality into the ZMPCS implementation in `non_hiding_zeromorph.rs` by adding a protocol name function.
- Improved security in `non_hiding_zeromorph.rs` by integrating domain separators into transcript's `open` and `verify` functions.

refactor: Refactor code and test performance for NHZM

- Implement `AsRef` trait for `UniPoly` structure in `src/spartan/sumcheck.rs` to facilitate direct access to its coefficients.
- Refactor code in `src/provider/non_hiding_zeromorph.rs` to directly use `into_iter` in map function when creating `quotients_polys`, avoiding a large clone
- Enhance test performance in `src/provider/non_hiding_zeromorph.rs` by pre-generating a `universal_setup` for tests, introducing a `max_vars` variable and RNG in the `commit_open_verify_with` test function, and adjusting the range of num_vars used for testing.

fix: adjust APIs & Serialize impls for the Nova traits

* feat: set up serde & abomonation bounds

* add generic PCS errors

* pp

wip

* fix: adjust type parameters for nontrivial_with_zm test

fix: ignore panicking test

fix: add doctest for evaluate_opt

fix: remove obsolete comments

chore: move UniPoly methods where they should be

test: make clear current zeromorph operates in monomial basis

- Added an additional test `test_dense_evaliations()` to provide more comprehensive testing for the `evaluate()` and `evaluate_opt()` functions in `MultilinearPolynomial`.

refactor TranscriptReprTrait impl for compat with Commitments

* feat: Implement KZG commitment trait and serialization features

- Added serialization, deserialization, and Abomonation traits to UVUniversalKZGParam struct in `non_hiding_kzg.rs` file along with implementing comparison and length evaluation traits.
- Created a new file `kzg_commitment.rs` which implements KZG Commitment Engine and setup, commit functions.
- Integrated `kzg_commitment` in module `provider` and set up conversions between Commitment and UVKZGCommitment.
- Enhanced assertion in `minroot_serde.rs` file from clone comparison to dereferenced comparison in MinRoot delay case.

* Use the ZMPCS Evaluation Engine and the KZG Commitment Engine in tests.

feat: Improve `prove` and `verify` methods in `ZMPCS` struct

- Updated `prove` and `verify` methods in `ZMPCS<E>` struct within `non_hiding_zeromorph.rs` with actual logic for commitment, evaluation, and verification.
- Adjusted the construction of `ZMCommitment` and `ZMEvaluation` within `prove` and `verify` methods.
- Commented on portions of the code in `non_hiding_zeromorph.rs` that need further modification and refinement.
- Modified test value for `test_pp_digest_with` in the `test_supernova_pp_digest` test case to match the new expected output.

* fix evaluation reversal bug

fix: remove superfluous eval functions

fix: remove endianness shenanigans

test: add evaluation unit test

* fix: parallellize pp generation

* feat: Enhance KZG commitment SRS generation efficiency using parallel computation

- Introduced a new module `util` within the `provider` module, implementing fixed-base MSM,
- New trait constraint has been imposed for `E::Fr: PrimeFieldBits` within the `non_hiding_zeromorph.rs` file, and usages have been adjusted in the `test` module.
- Adding the `PrimeFieldBits` import from the `ff` crate and importing `provider::util` from the local crate.

refactor: Refactor utility functions for elliptic curve operations

- Renamed and moved `util.rs` to `fb_msm.rs` under the `provider` directory.
- documented the FB MSM

* Apply comments from code review

Co-authored-by: Adrian Hamelink <[email protected]>

* Improve documentation and testing in non_hiding_zeromorph.rs

- Enhanced the clarity and accuracy of code comments and documentation in `non_hiding_zeromorph.rs`, specifically within the `ZMPCS` struct's methods.
- Enriched the `quotients` function's documentation, detailing its mathematical underpinnings
- Implemented a new rigorous test, `test_quotients`, to ensure the `quotients` function's output satisfies the polynomial identity.

* test: refactor random ML poly generation

* Finish documenting / testing Zeromorph (microsoft#142)

* fix: remove a TODO using RefCast

* fix: remove unsightly clone

* doc: comment intermediate ZM steps

* refactor: Refactor `open` function in zeromorph provider

Extracted the computation of `q_hat` in the `open` function into a new function `batched_lifted_degree_quotient` for more modular code.

* test: test batched_lifted_degree_quotient

Addition of a new test `test_batched_lifted_degree_quotient` to validate batched degree quotient lifting for polynomials.

* refactor: Refine computations in non_hiding_zeromorph.rs

- Overhauled the 'eval_and_quotient_scalars' function with a revised return type with pair of vectors rather than scalar and a vector,
- correspondingly split scalars `degree_check_q_scalars` and `zmpoly_q_scalars` within the `open` method of `non_hiding_zeromorph.rs`.
- Adjusted the 'verify' method to accommodate the modified 'eval_and_quotient_scalars' function output changes.

* test: test partially evaluated quotient \zeta_x

- Introduced a new test case `test_partially_evaluated_quotient_zeta` to validate `zeta_x` construction.
- correct an off-by-one error in the y_k terms

* test: test partially evaluated quotient

- Created a new function `phi` for evaluating using an inefficient formula in `non_hiding_zeromorph.rs`
- Implemented a new test function `test_partially_evaluated_quotient_z` for validation of `Z_x` computation method in `non_hiding_zeromorph.rs`

* fix: refine q_hat comment

* fix: organize code more sparsely after rebase

* refactor: Improve code readability and error handling in zeromorph

- Updated and enhanced clarity and consistency of mathematical notation in comments across `non_hiding_kzg.rs` and `non_hiding_zeromorph.rs` files.
- Implemented error handling in the `ZMPCS::verify` function within the `non_hiding_zeromorph.rs` file.

* fix: compute quotient commitments in parallel

* refactor: clean up

---------

Co-authored-by: emmorais <[email protected]>
Co-authored-by: Matej Penciak <[email protected]>
Co-authored-by: Adrian Hamelink <[email protected]>
huitseeker added a commit to argumentcomputer/Nova that referenced this pull request Jan 26, 2024
* Schoolbook adapted KZG PoC

Remove toxic waste from setup parameters

Refactorings

ZeroMorph skeleton

* feat: provide a non-hiding variant of KZG

Drafts a non-hiding variant of the KZG PCS on univariate Dense polynomial representations

Details:
- Changed access levels and ordering for various modules and functions like `sumcheck`, `cpu_best_multiexp`, and `mod kzg;`.
- Upgraded `halo2curves` version and added new dependencies like `pairing`, `anyhow`, `rand`, and `group`.
- Leveraged the UniPoly in sumcheck.rs
- Created `non_hiding_kzg.rs` file, introducing new structures and functionalities like `UVUniversalKZGParam`, `UVKZGProverKey`, `UVKZGPoly`, and `UVKZGPCS` along with their implementation and tests.
- Modified the import of `Engine` in `kzg.rs` from `halo2curves::pairing::Engine` to `pairing::Engine` (reflecting the new version of halo2curves).

feat: add batch commit / open / prove

* Small subtasks: params, phi, un function, EE

Minor adjustments

* chore: setup zeromorph

refactor: Remove kzg and zeromorph provider modules

* feat: add openings for Zeromorph

- Added new data structures and their related functions including the `ZMProverKey`, `ZMVerifierKey`, `ZMCommitment`, `ZMEvaluation`, `ZMProof` and `ZMPCS` in `non_hiding_zeromorph.rs`
- Implemented new functionalities in `non_hiding_zeromorph.rs` and `spartan/polynomial.rs` to provide better handling of polynomials, including fetching commitments for a polynomial, computing the quotient polynomials of an existing polynomial.
- Enhanced the `UniPoly` struct in `spartan/sumcheck.rs` with `Index` and `IndexMut` for better coefficient access, and `AddAssign` and `MulAssign` for scalar and self types.
- Removed and replaced certain elements in `UVUniversalKZGParam` struct in `non_hiding_kzg.rs` for improved flexibility, and included import and implementation of `TranscriptReprTrait`.

draft ZM verify

set up test structure

defer test_quo

* Include commits into ZMProof and adjust RO

* feat: Refactor polynomial evaluation methods

- Temporarily disabled some assertions in both `non_hiding_zeromorph.rs` and `polynomial.rs` for debugging purposes.
- Introduced `evaluate_opt`, `fix_variables`, and `fix_one_variable_helper` functions in `polynomial.rs` to support multilinear polynomial evaluation and partial evaluation.

fix: add domain separators

- Integrated additional functionality into the ZMPCS implementation in `non_hiding_zeromorph.rs` by adding a protocol name function.
- Improved security in `non_hiding_zeromorph.rs` by integrating domain separators into transcript's `open` and `verify` functions.

refactor: Refactor code and test performance for NHZM

- Implement `AsRef` trait for `UniPoly` structure in `src/spartan/sumcheck.rs` to facilitate direct access to its coefficients.
- Refactor code in `src/provider/non_hiding_zeromorph.rs` to directly use `into_iter` in map function when creating `quotients_polys`, avoiding a large clone
- Enhance test performance in `src/provider/non_hiding_zeromorph.rs` by pre-generating a `universal_setup` for tests, introducing a `max_vars` variable and RNG in the `commit_open_verify_with` test function, and adjusting the range of num_vars used for testing.

fix: adjust APIs & Serialize impls for the Nova traits

* feat: set up serde & abomonation bounds

* add generic PCS errors

* pp

wip

* fix: adjust type parameters for nontrivial_with_zm test

fix: ignore panicking test

fix: add doctest for evaluate_opt

fix: remove obsolete comments

chore: move UniPoly methods where they should be

test: make clear current zeromorph operates in monomial basis

- Added an additional test `test_dense_evaliations()` to provide more comprehensive testing for the `evaluate()` and `evaluate_opt()` functions in `MultilinearPolynomial`.

refactor TranscriptReprTrait impl for compat with Commitments

* feat: Implement KZG commitment trait and serialization features

- Added serialization, deserialization, and Abomonation traits to UVUniversalKZGParam struct in `non_hiding_kzg.rs` file along with implementing comparison and length evaluation traits.
- Created a new file `kzg_commitment.rs` which implements KZG Commitment Engine and setup, commit functions.
- Integrated `kzg_commitment` in module `provider` and set up conversions between Commitment and UVKZGCommitment.
- Enhanced assertion in `minroot_serde.rs` file from clone comparison to dereferenced comparison in MinRoot delay case.

* Use the ZMPCS Evaluation Engine and the KZG Commitment Engine in tests.

feat: Improve `prove` and `verify` methods in `ZMPCS` struct

- Updated `prove` and `verify` methods in `ZMPCS<E>` struct within `non_hiding_zeromorph.rs` with actual logic for commitment, evaluation, and verification.
- Adjusted the construction of `ZMCommitment` and `ZMEvaluation` within `prove` and `verify` methods.
- Commented on portions of the code in `non_hiding_zeromorph.rs` that need further modification and refinement.
- Modified test value for `test_pp_digest_with` in the `test_supernova_pp_digest` test case to match the new expected output.

* fix evaluation reversal bug

fix: remove superfluous eval functions

fix: remove endianness shenanigans

test: add evaluation unit test

* fix: parallellize pp generation

* feat: Enhance KZG commitment SRS generation efficiency using parallel computation

- Introduced a new module `util` within the `provider` module, implementing fixed-base MSM,
- New trait constraint has been imposed for `E::Fr: PrimeFieldBits` within the `non_hiding_zeromorph.rs` file, and usages have been adjusted in the `test` module.
- Adding the `PrimeFieldBits` import from the `ff` crate and importing `provider::util` from the local crate.

refactor: Refactor utility functions for elliptic curve operations

- Renamed and moved `util.rs` to `fb_msm.rs` under the `provider` directory.
- documented the FB MSM

* Apply comments from code review

Co-authored-by: Adrian Hamelink <[email protected]>

* Improve documentation and testing in non_hiding_zeromorph.rs

- Enhanced the clarity and accuracy of code comments and documentation in `non_hiding_zeromorph.rs`, specifically within the `ZMPCS` struct's methods.
- Enriched the `quotients` function's documentation, detailing its mathematical underpinnings
- Implemented a new rigorous test, `test_quotients`, to ensure the `quotients` function's output satisfies the polynomial identity.

* test: refactor random ML poly generation

* Finish documenting / testing Zeromorph (microsoft#142)

* fix: remove a TODO using RefCast

* fix: remove unsightly clone

* doc: comment intermediate ZM steps

* refactor: Refactor `open` function in zeromorph provider

Extracted the computation of `q_hat` in the `open` function into a new function `batched_lifted_degree_quotient` for more modular code.

* test: test batched_lifted_degree_quotient

Addition of a new test `test_batched_lifted_degree_quotient` to validate batched degree quotient lifting for polynomials.

* refactor: Refine computations in non_hiding_zeromorph.rs

- Overhauled the 'eval_and_quotient_scalars' function with a revised return type with pair of vectors rather than scalar and a vector,
- correspondingly split scalars `degree_check_q_scalars` and `zmpoly_q_scalars` within the `open` method of `non_hiding_zeromorph.rs`.
- Adjusted the 'verify' method to accommodate the modified 'eval_and_quotient_scalars' function output changes.

* test: test partially evaluated quotient \zeta_x

- Introduced a new test case `test_partially_evaluated_quotient_zeta` to validate `zeta_x` construction.
- correct an off-by-one error in the y_k terms

* test: test partially evaluated quotient

- Created a new function `phi` for evaluating using an inefficient formula in `non_hiding_zeromorph.rs`
- Implemented a new test function `test_partially_evaluated_quotient_z` for validation of `Z_x` computation method in `non_hiding_zeromorph.rs`

* fix: refine q_hat comment

* fix: organize code more sparsely after rebase

* refactor: Improve code readability and error handling in zeromorph

- Updated and enhanced clarity and consistency of mathematical notation in comments across `non_hiding_kzg.rs` and `non_hiding_zeromorph.rs` files.
- Implemented error handling in the `ZMPCS::verify` function within the `non_hiding_zeromorph.rs` file.

* fix: compute quotient commitments in parallel

* refactor: clean up

---------

Co-authored-by: emmorais <[email protected]>
Co-authored-by: Matej Penciak <[email protected]>
Co-authored-by: Adrian Hamelink <[email protected]>
huitseeker added a commit to argumentcomputer/Nova that referenced this pull request Feb 21, 2024
* Schoolbook adapted KZG PoC

Remove toxic waste from setup parameters

Refactorings

ZeroMorph skeleton

* feat: provide a non-hiding variant of KZG

Drafts a non-hiding variant of the KZG PCS on univariate Dense polynomial representations

Details:
- Changed access levels and ordering for various modules and functions like `sumcheck`, `cpu_best_multiexp`, and `mod kzg;`.
- Upgraded `halo2curves` version and added new dependencies like `pairing`, `anyhow`, `rand`, and `group`.
- Leveraged the UniPoly in sumcheck.rs
- Created `non_hiding_kzg.rs` file, introducing new structures and functionalities like `UVUniversalKZGParam`, `UVKZGProverKey`, `UVKZGPoly`, and `UVKZGPCS` along with their implementation and tests.
- Modified the import of `Engine` in `kzg.rs` from `halo2curves::pairing::Engine` to `pairing::Engine` (reflecting the new version of halo2curves).

feat: add batch commit / open / prove

* Small subtasks: params, phi, un function, EE

Minor adjustments

* chore: setup zeromorph

refactor: Remove kzg and zeromorph provider modules

* feat: add openings for Zeromorph

- Added new data structures and their related functions including the `ZMProverKey`, `ZMVerifierKey`, `ZMCommitment`, `ZMEvaluation`, `ZMProof` and `ZMPCS` in `non_hiding_zeromorph.rs`
- Implemented new functionalities in `non_hiding_zeromorph.rs` and `spartan/polynomial.rs` to provide better handling of polynomials, including fetching commitments for a polynomial, computing the quotient polynomials of an existing polynomial.
- Enhanced the `UniPoly` struct in `spartan/sumcheck.rs` with `Index` and `IndexMut` for better coefficient access, and `AddAssign` and `MulAssign` for scalar and self types.
- Removed and replaced certain elements in `UVUniversalKZGParam` struct in `non_hiding_kzg.rs` for improved flexibility, and included import and implementation of `TranscriptReprTrait`.

draft ZM verify

set up test structure

defer test_quo

* Include commits into ZMProof and adjust RO

* feat: Refactor polynomial evaluation methods

- Temporarily disabled some assertions in both `non_hiding_zeromorph.rs` and `polynomial.rs` for debugging purposes.
- Introduced `evaluate_opt`, `fix_variables`, and `fix_one_variable_helper` functions in `polynomial.rs` to support multilinear polynomial evaluation and partial evaluation.

fix: add domain separators

- Integrated additional functionality into the ZMPCS implementation in `non_hiding_zeromorph.rs` by adding a protocol name function.
- Improved security in `non_hiding_zeromorph.rs` by integrating domain separators into transcript's `open` and `verify` functions.

refactor: Refactor code and test performance for NHZM

- Implement `AsRef` trait for `UniPoly` structure in `src/spartan/sumcheck.rs` to facilitate direct access to its coefficients.
- Refactor code in `src/provider/non_hiding_zeromorph.rs` to directly use `into_iter` in map function when creating `quotients_polys`, avoiding a large clone
- Enhance test performance in `src/provider/non_hiding_zeromorph.rs` by pre-generating a `universal_setup` for tests, introducing a `max_vars` variable and RNG in the `commit_open_verify_with` test function, and adjusting the range of num_vars used for testing.

fix: adjust APIs & Serialize impls for the Nova traits

* feat: set up serde & abomonation bounds

* add generic PCS errors

* pp

wip

* fix: adjust type parameters for nontrivial_with_zm test

fix: ignore panicking test

fix: add doctest for evaluate_opt

fix: remove obsolete comments

chore: move UniPoly methods where they should be

test: make clear current zeromorph operates in monomial basis

- Added an additional test `test_dense_evaliations()` to provide more comprehensive testing for the `evaluate()` and `evaluate_opt()` functions in `MultilinearPolynomial`.

refactor TranscriptReprTrait impl for compat with Commitments

* feat: Implement KZG commitment trait and serialization features

- Added serialization, deserialization, and Abomonation traits to UVUniversalKZGParam struct in `non_hiding_kzg.rs` file along with implementing comparison and length evaluation traits.
- Created a new file `kzg_commitment.rs` which implements KZG Commitment Engine and setup, commit functions.
- Integrated `kzg_commitment` in module `provider` and set up conversions between Commitment and UVKZGCommitment.
- Enhanced assertion in `minroot_serde.rs` file from clone comparison to dereferenced comparison in MinRoot delay case.

* Use the ZMPCS Evaluation Engine and the KZG Commitment Engine in tests.

feat: Improve `prove` and `verify` methods in `ZMPCS` struct

- Updated `prove` and `verify` methods in `ZMPCS<E>` struct within `non_hiding_zeromorph.rs` with actual logic for commitment, evaluation, and verification.
- Adjusted the construction of `ZMCommitment` and `ZMEvaluation` within `prove` and `verify` methods.
- Commented on portions of the code in `non_hiding_zeromorph.rs` that need further modification and refinement.
- Modified test value for `test_pp_digest_with` in the `test_supernova_pp_digest` test case to match the new expected output.

* fix evaluation reversal bug

fix: remove superfluous eval functions

fix: remove endianness shenanigans

test: add evaluation unit test

* fix: parallellize pp generation

* feat: Enhance KZG commitment SRS generation efficiency using parallel computation

- Introduced a new module `util` within the `provider` module, implementing fixed-base MSM,
- New trait constraint has been imposed for `E::Fr: PrimeFieldBits` within the `non_hiding_zeromorph.rs` file, and usages have been adjusted in the `test` module.
- Adding the `PrimeFieldBits` import from the `ff` crate and importing `provider::util` from the local crate.

refactor: Refactor utility functions for elliptic curve operations

- Renamed and moved `util.rs` to `fb_msm.rs` under the `provider` directory.
- documented the FB MSM

* Apply comments from code review

Co-authored-by: Adrian Hamelink <[email protected]>

* Improve documentation and testing in non_hiding_zeromorph.rs

- Enhanced the clarity and accuracy of code comments and documentation in `non_hiding_zeromorph.rs`, specifically within the `ZMPCS` struct's methods.
- Enriched the `quotients` function's documentation, detailing its mathematical underpinnings
- Implemented a new rigorous test, `test_quotients`, to ensure the `quotients` function's output satisfies the polynomial identity.

* test: refactor random ML poly generation

* Finish documenting / testing Zeromorph (microsoft#142)

* fix: remove a TODO using RefCast

* fix: remove unsightly clone

* doc: comment intermediate ZM steps

* refactor: Refactor `open` function in zeromorph provider

Extracted the computation of `q_hat` in the `open` function into a new function `batched_lifted_degree_quotient` for more modular code.

* test: test batched_lifted_degree_quotient

Addition of a new test `test_batched_lifted_degree_quotient` to validate batched degree quotient lifting for polynomials.

* refactor: Refine computations in non_hiding_zeromorph.rs

- Overhauled the 'eval_and_quotient_scalars' function with a revised return type with pair of vectors rather than scalar and a vector,
- correspondingly split scalars `degree_check_q_scalars` and `zmpoly_q_scalars` within the `open` method of `non_hiding_zeromorph.rs`.
- Adjusted the 'verify' method to accommodate the modified 'eval_and_quotient_scalars' function output changes.

* test: test partially evaluated quotient \zeta_x

- Introduced a new test case `test_partially_evaluated_quotient_zeta` to validate `zeta_x` construction.
- correct an off-by-one error in the y_k terms

* test: test partially evaluated quotient

- Created a new function `phi` for evaluating using an inefficient formula in `non_hiding_zeromorph.rs`
- Implemented a new test function `test_partially_evaluated_quotient_z` for validation of `Z_x` computation method in `non_hiding_zeromorph.rs`

* fix: refine q_hat comment

* fix: organize code more sparsely after rebase

* refactor: Improve code readability and error handling in zeromorph

- Updated and enhanced clarity and consistency of mathematical notation in comments across `non_hiding_kzg.rs` and `non_hiding_zeromorph.rs` files.
- Implemented error handling in the `ZMPCS::verify` function within the `non_hiding_zeromorph.rs` file.

* fix: compute quotient commitments in parallel

* refactor: clean up

---------

Co-authored-by: emmorais <[email protected]>
Co-authored-by: Matej Penciak <[email protected]>
Co-authored-by: Adrian Hamelink <[email protected]>
huitseeker added a commit to argumentcomputer/Nova that referenced this pull request May 2, 2024
* Schoolbook adapted KZG PoC

Remove toxic waste from setup parameters

Refactorings

ZeroMorph skeleton

* feat: provide a non-hiding variant of KZG

Drafts a non-hiding variant of the KZG PCS on univariate Dense polynomial representations

Details:
- Changed access levels and ordering for various modules and functions like `sumcheck`, `cpu_best_multiexp`, and `mod kzg;`.
- Upgraded `halo2curves` version and added new dependencies like `pairing`, `anyhow`, `rand`, and `group`.
- Leveraged the UniPoly in sumcheck.rs
- Created `non_hiding_kzg.rs` file, introducing new structures and functionalities like `UVUniversalKZGParam`, `UVKZGProverKey`, `UVKZGPoly`, and `UVKZGPCS` along with their implementation and tests.
- Modified the import of `Engine` in `kzg.rs` from `halo2curves::pairing::Engine` to `pairing::Engine` (reflecting the new version of halo2curves).

feat: add batch commit / open / prove

* Small subtasks: params, phi, un function, EE

Minor adjustments

* chore: setup zeromorph

refactor: Remove kzg and zeromorph provider modules

* feat: add openings for Zeromorph

- Added new data structures and their related functions including the `ZMProverKey`, `ZMVerifierKey`, `ZMCommitment`, `ZMEvaluation`, `ZMProof` and `ZMPCS` in `non_hiding_zeromorph.rs`
- Implemented new functionalities in `non_hiding_zeromorph.rs` and `spartan/polynomial.rs` to provide better handling of polynomials, including fetching commitments for a polynomial, computing the quotient polynomials of an existing polynomial.
- Enhanced the `UniPoly` struct in `spartan/sumcheck.rs` with `Index` and `IndexMut` for better coefficient access, and `AddAssign` and `MulAssign` for scalar and self types.
- Removed and replaced certain elements in `UVUniversalKZGParam` struct in `non_hiding_kzg.rs` for improved flexibility, and included import and implementation of `TranscriptReprTrait`.

draft ZM verify

set up test structure

defer test_quo

* Include commits into ZMProof and adjust RO

* feat: Refactor polynomial evaluation methods

- Temporarily disabled some assertions in both `non_hiding_zeromorph.rs` and `polynomial.rs` for debugging purposes.
- Introduced `evaluate_opt`, `fix_variables`, and `fix_one_variable_helper` functions in `polynomial.rs` to support multilinear polynomial evaluation and partial evaluation.

fix: add domain separators

- Integrated additional functionality into the ZMPCS implementation in `non_hiding_zeromorph.rs` by adding a protocol name function.
- Improved security in `non_hiding_zeromorph.rs` by integrating domain separators into transcript's `open` and `verify` functions.

refactor: Refactor code and test performance for NHZM

- Implement `AsRef` trait for `UniPoly` structure in `src/spartan/sumcheck.rs` to facilitate direct access to its coefficients.
- Refactor code in `src/provider/non_hiding_zeromorph.rs` to directly use `into_iter` in map function when creating `quotients_polys`, avoiding a large clone
- Enhance test performance in `src/provider/non_hiding_zeromorph.rs` by pre-generating a `universal_setup` for tests, introducing a `max_vars` variable and RNG in the `commit_open_verify_with` test function, and adjusting the range of num_vars used for testing.

fix: adjust APIs & Serialize impls for the Nova traits

* feat: set up serde & abomonation bounds

* add generic PCS errors

* pp

wip

* fix: adjust type parameters for nontrivial_with_zm test

fix: ignore panicking test

fix: add doctest for evaluate_opt

fix: remove obsolete comments

chore: move UniPoly methods where they should be

test: make clear current zeromorph operates in monomial basis

- Added an additional test `test_dense_evaliations()` to provide more comprehensive testing for the `evaluate()` and `evaluate_opt()` functions in `MultilinearPolynomial`.

refactor TranscriptReprTrait impl for compat with Commitments

* feat: Implement KZG commitment trait and serialization features

- Added serialization, deserialization, and Abomonation traits to UVUniversalKZGParam struct in `non_hiding_kzg.rs` file along with implementing comparison and length evaluation traits.
- Created a new file `kzg_commitment.rs` which implements KZG Commitment Engine and setup, commit functions.
- Integrated `kzg_commitment` in module `provider` and set up conversions between Commitment and UVKZGCommitment.
- Enhanced assertion in `minroot_serde.rs` file from clone comparison to dereferenced comparison in MinRoot delay case.

* Use the ZMPCS Evaluation Engine and the KZG Commitment Engine in tests.

feat: Improve `prove` and `verify` methods in `ZMPCS` struct

- Updated `prove` and `verify` methods in `ZMPCS<E>` struct within `non_hiding_zeromorph.rs` with actual logic for commitment, evaluation, and verification.
- Adjusted the construction of `ZMCommitment` and `ZMEvaluation` within `prove` and `verify` methods.
- Commented on portions of the code in `non_hiding_zeromorph.rs` that need further modification and refinement.
- Modified test value for `test_pp_digest_with` in the `test_supernova_pp_digest` test case to match the new expected output.

* fix evaluation reversal bug

fix: remove superfluous eval functions

fix: remove endianness shenanigans

test: add evaluation unit test

* fix: parallellize pp generation

* feat: Enhance KZG commitment SRS generation efficiency using parallel computation

- Introduced a new module `util` within the `provider` module, implementing fixed-base MSM,
- New trait constraint has been imposed for `E::Fr: PrimeFieldBits` within the `non_hiding_zeromorph.rs` file, and usages have been adjusted in the `test` module.
- Adding the `PrimeFieldBits` import from the `ff` crate and importing `provider::util` from the local crate.

refactor: Refactor utility functions for elliptic curve operations

- Renamed and moved `util.rs` to `fb_msm.rs` under the `provider` directory.
- documented the FB MSM

* Apply comments from code review

Co-authored-by: Adrian Hamelink <[email protected]>

* Improve documentation and testing in non_hiding_zeromorph.rs

- Enhanced the clarity and accuracy of code comments and documentation in `non_hiding_zeromorph.rs`, specifically within the `ZMPCS` struct's methods.
- Enriched the `quotients` function's documentation, detailing its mathematical underpinnings
- Implemented a new rigorous test, `test_quotients`, to ensure the `quotients` function's output satisfies the polynomial identity.

* test: refactor random ML poly generation

* Finish documenting / testing Zeromorph (microsoft#142)

* fix: remove a TODO using RefCast

* fix: remove unsightly clone

* doc: comment intermediate ZM steps

* refactor: Refactor `open` function in zeromorph provider

Extracted the computation of `q_hat` in the `open` function into a new function `batched_lifted_degree_quotient` for more modular code.

* test: test batched_lifted_degree_quotient

Addition of a new test `test_batched_lifted_degree_quotient` to validate batched degree quotient lifting for polynomials.

* refactor: Refine computations in non_hiding_zeromorph.rs

- Overhauled the 'eval_and_quotient_scalars' function with a revised return type with pair of vectors rather than scalar and a vector,
- correspondingly split scalars `degree_check_q_scalars` and `zmpoly_q_scalars` within the `open` method of `non_hiding_zeromorph.rs`.
- Adjusted the 'verify' method to accommodate the modified 'eval_and_quotient_scalars' function output changes.

* test: test partially evaluated quotient \zeta_x

- Introduced a new test case `test_partially_evaluated_quotient_zeta` to validate `zeta_x` construction.
- correct an off-by-one error in the y_k terms

* test: test partially evaluated quotient

- Created a new function `phi` for evaluating using an inefficient formula in `non_hiding_zeromorph.rs`
- Implemented a new test function `test_partially_evaluated_quotient_z` for validation of `Z_x` computation method in `non_hiding_zeromorph.rs`

* fix: refine q_hat comment

* fix: organize code more sparsely after rebase

* refactor: Improve code readability and error handling in zeromorph

- Updated and enhanced clarity and consistency of mathematical notation in comments across `non_hiding_kzg.rs` and `non_hiding_zeromorph.rs` files.
- Implemented error handling in the `ZMPCS::verify` function within the `non_hiding_zeromorph.rs` file.

* fix: compute quotient commitments in parallel

* refactor: clean up

---------

Co-authored-by: emmorais <[email protected]>
Co-authored-by: Matej Penciak <[email protected]>
Co-authored-by: Adrian Hamelink <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant