Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement PCS IPA and IPA accumulation in NativeLoader #14

Merged
merged 1 commit into from
Nov 9, 2022

Conversation

han0110
Copy link
Collaborator

@han0110 han0110 commented Nov 9, 2022

IPA accumulation in Halo2Loader is still WIP.

@han0110 han0110 merged commit 5c31088 into privacy-scaling-explorations:main Nov 9, 2022
@han0110 han0110 deleted the feature/pcs-ipa branch November 9, 2022 03:58
jonathanpwang referenced this pull request in axiom-crypto/snark-verifier Dec 15, 2022
* chore: add display feature to Cargo.toml

* fix: change test data directory to ./data and give error message

* feat: add succinct_verify_or_dummy to allow flag to turn off verification in aggregation snark

* Rollback to pse halo2 and halo2wrong for first release (#5)

* feat: move `Accumulator` to `accumulator.rs`

* feat: update due to halo2

* feat: upgrade to use branch `feature/generic-instructions` of `halo2wrong`

* refactor: rollback to `{halo2,halo2_wrong}` without challenge API and cleanup dependencies

* chore: rename statement to instance and auxliary to witness

* chore: use `finalize` instead of `code`

* feat: add `Code::deployment` and `EvmLoader::deployment_code`; add example `evm-verifier-codegen`

* fix: typo

* feat: reduce generated evm verifier size; rename to `evm-verifier` and add another example `evm-verifier-with-accumulator`

* fix: due to `halo2wrong`

* feat: reorganize mods and traits

* fix: allow empty `values` in `sum_*` and move them under `ScalarLoader`

* ci: use `--all-features` for `cargo test`

* fix: use same strategy for aggregation testing

* fix: simplify trait `PlonkVerifier` again

* fix: move system specified transcript under mod `system`

* feat: add `quotient_poly` info in `Protocol`

* feat: implement linearization for circom integration

* feat: re-export loader's dependency for consumer

* refactor: for circom's integration

* tmp: pin `revm` to rev

* fix: remove parentheses

* fix: upgrade for multi-phase halo2

* feat: improve error reporting

* chore: rename crate to snake case

* feat: add `Domain` as an input of `PolynomialCommitmentScheme::read_proof`

* refactor: for further integration

* feat: generalize to suppoer also ipa and add builder fns to `system::halo2::Config`

* feat: add `KzgDecider` for simple evm verifier

* refactor: split `AccumulationScheme` and `AccumulatorEncoding`

* refactor: split `PolynomialCommitmentScheme` and `MultiOpenScheme`

* fix: not need sealed actually

* fix: `chunk_size` should be `LIMBS` when recovering accumulator

* feat: add `Expression::DistributePowers` to avoid stack overflow

* fix: update and pin foundry

* fix: move testing circuits under `system/halo2`

* fix: allow accumulate single accumulator

* feat: remove all patch and make less depending `halo2wrong`

* chore: update dependencies with latest halo2_proofs

* add `serialize` feature to turn on/off vkey/pkey write

* feat: change `name` to function instead of const in `TargetCircuit`

* fix: `load_verify_circuit_degree` now takes in environmental variable
`VERIFY_CONFIG`

* Generalized `Halo2Loader` (#12)

* feat: generalize `Protocol` for further usage

* feat: add `EccInstruction::{fixed_base_msm,variable_base_msm,sum_with_const}`

* chore: move `rand_chacha` as dev dependency

* feat: implement `IntegerInstructions` and `EccInstructions` traits for
`halo2-lib`

* feat: implement `EncodeNative` and working example
`evm-verifier-with-accumulator` using `halo2-lib`
* Does not yet use custom `fixed_base_msm`

* feat: add example `recursion`

* chore: update for compatibility with halo2-lib

* poseidon: switch to rate = 2, `t = 3` for faster proving time

* update: remove unnecessary trait `where` clause

* feat: add example `recursion`

* General refactor for further integration (#13)

* feat: remove dev-dependency `foundry` and vendor necessary part of it

* refactor: simplify traits and remove unused stuff

* refactor: much less clone

* feat: generalized `AccumulatorEncoding` for `EccInstructions`

* feat: implement ipa pcs and accumulation (#14)

* feat: add example `recursion`

* wip: update to halo2-lib v0.2.0

* wip: recursion example currently works if circuit does not use simple
selectors

* Simple selector compression is not correctly handled by `CsProxy` at
  the moment

* fix: got recursion example working with halo2-lib

* where halo2-lib uses simple selectors
* only works if all selector columns are in active use and can't be
  optimized away
* tricks `CsProxy` by turning all selectors on in row `0`

* feat: add `sdk` module for snark aggregation toolkit

* chore: move `aggregation` under `halo2` inside `sdk`

* Update `EvmLoader` to generate Yul code instead of bytecode (#15)

* Update `EvmLoader` to generate Yul instead of bytecode

* feat: simplify

* feat: Add missing end_gas_metering impl

Co-authored-by: Han <[email protected]>

Co-authored-by: Han <[email protected]>

* feat: add `evm` module to `sdk` with evm proof and verifier functions

* add bench for zkevm EVM circuit, currently uses my local repo because
  official repo does not compile... use scroll-dev-1115 circuit
implementation where power of randomness is generated internally and not
from public inputs (far too many public inputs for aggregation)

* chore: update Cargo.toml

* reorg: rename repo to snark-verifier

* split into two crates: snark-verifier and snark-verifier-sdk
* previous module sdk moved to snark-verifier-sdk

Co-authored-by: Han <[email protected]>
Co-authored-by: DoHoon Kim <[email protected]>
zemse pushed a commit to zemse/snark-verifier that referenced this pull request Jan 18, 2024
zemse pushed a commit to zemse/snark-verifier that referenced this pull request Jan 18, 2024
* Rollback to pse halo2 and halo2wrong for first release (privacy-scaling-explorations#5)

* feat: move `Accumulator` to `accumulator.rs`

* feat: update due to halo2

* feat: upgrade to use branch `feature/generic-instructions` of `halo2wrong`

* refactor: rollback to `{halo2,halo2_wrong}` without challenge API and cleanup dependencies

* chore: rename statement to instance and auxliary to witness

* chore: use `finalize` instead of `code`

* feat: add `Code::deployment` and `EvmLoader::deployment_code`; add example `evm-verifier-codegen`

* fix: typo

* feat: reduce generated evm verifier size; rename to `evm-verifier` and add another example `evm-verifier-with-accumulator`

* fix: due to `halo2wrong`

* feat: reorganize mods and traits

* fix: allow empty `values` in `sum_*` and move them under `ScalarLoader`

* ci: use `--all-features` for `cargo test`

* fix: use same strategy for aggregation testing

* fix: simplify trait `PlonkVerifier` again

* fix: move system specified transcript under mod `system`

* feat: add `quotient_poly` info in `Protocol`

* feat: implement linearization for circom integration

* feat: re-export loader's dependency for consumer

* refactor: for circom's integration

* tmp: pin `revm` to rev

* fix: remove parentheses

* fix: upgrade for multi-phase halo2

* feat: improve error reporting

* chore: rename crate to snake case

* feat: add `Domain` as an input of `PolynomialCommitmentScheme::read_proof`

* refactor: for further integration

* feat: generalize to suppoer also ipa and add builder fns to `system::halo2::Config`

* feat: add `KzgDecider` for simple evm verifier

* refactor: split `AccumulationScheme` and `AccumulatorEncoding`

* refactor: split `PolynomialCommitmentScheme` and `MultiOpenScheme`

* fix: not need sealed actually

* fix: `chunk_size` should be `LIMBS` when recovering accumulator

* feat: add `Expression::DistributePowers` to avoid stack overflow

* fix: update and pin foundry

* fix: move testing circuits under `system/halo2`

* fix: allow accumulate single accumulator

* feat: remove all patch and make less depending `halo2wrong`

* Generalized `Halo2Loader` (privacy-scaling-explorations#12)

* feat: generalize `Protocol` for further usage

* feat: add `EccInstruction::{fixed_base_msm,variable_base_msm,sum_with_const}`

* chore: move `rand_chacha` as dev dependency

* General refactor for further integration (privacy-scaling-explorations#13)

* feat: remove dev-dependency `foundry` and vendor necessary part of it

* refactor: simplify traits and remove unused stuff

* refactor: much less clone

* feat: generalized `AccumulatorEncoding` for `EccInstructions`

* feat: implement ipa pcs and accumulation (privacy-scaling-explorations#14)

* ci: add `svm-rs` and install `[email protected]` in job `test` (privacy-scaling-explorations#16)

* Update `EvmLoader` to generate Yul code instead of bytecode (privacy-scaling-explorations#15)

* Update `EvmLoader` to generate Yul instead of bytecode

* feat: simplify

* feat: Add missing end_gas_metering impl

Co-authored-by: Han <[email protected]>

Co-authored-by: Han <[email protected]>

* fix: pin all `revm` dependencies (privacy-scaling-explorations#18)

* fix: looser trait bound on impl `CostEstimation` for `Plonk` (privacy-scaling-explorations#20)

* Restructure for more kind of verifier (privacy-scaling-explorations#21)

* feat: restructure to monorepo and expand the project scope to be generic (s)nark verifier

* feat: reorganize mods and traits for further new features

* refactor: simplify trait bounds

* chore: use hyphen case for crate name (`snark_verifier` -> `snark-verifier`)

* docs: add `#![deny(missing_docs)]` and simple documents

* refactor: remove redudant check `validate_ec_point` (still doesn not support identity)

* feat: expand more things and fix typos

Co-authored-by: Chih Cheng Liang <[email protected]>
Co-authored-by: Carlos Pérez <[email protected]>

* fix: rustdoc warnings

* chore: update dependencies (privacy-scaling-explorations#24)

* chore: update `halo2` and `halo2wrong` version (privacy-scaling-explorations#25)

* fix: enable `util::hash::poseidon` only when `feature = loader_halo2` (privacy-scaling-explorations#27)

* feat: working update to halo2-lib v0.3.0

* feat: update zkevm bench

* feat: update recursion example

* feat: switch poseidon native implementation to Scroll's audited version

* fix: invert determinant only once in Cramer's rule

* chore: fix doc

* chore

* chore: forgot to update halo2-base dependency in snark-verifier-sdk

* Minor update (privacy-scaling-explorations#8)

* feat(sdk): remove duplicate code in
`RangeWithInstanceCircuitBuilder::synthesize`

* feat(sdk): Proof caching when using feature 'halo2-pse'

* chore: sync with halo2-lib

* chore: switch to halo2-lib release-0.3.0 branch

* Moved `RangeWithInstanceCircuitBuilder` to halo2-lib (privacy-scaling-explorations#9)

* chore: sync with halo2-lib

* fix: clippy

* chore: fix halo2-base branch in sdk

* feat: update to halo2-lib new types (privacy-scaling-explorations#10)

* feat: add `assert` for non-empty accumulators in `decide_all` (privacy-scaling-explorations#11)

* feat: use `zip_eq` for `Polynomial` add/sub (privacy-scaling-explorations#12)

* fix: git CI turn off all features

* fix: `rotate_scalar` misbehaves on `i32::MIN` (privacy-scaling-explorations#13)

Should never actually be callable with such a large negative rotation

* fix: cleanup code quality (privacy-scaling-explorations#14)

* fix: `split_by_ascii_whitespace` (privacy-scaling-explorations#15)

* fix: `batch_invert_and_mul` do not allow zeros (privacy-scaling-explorations#16)

* feat: verify proof in release mode (privacy-scaling-explorations#17)

Verify proof before caching it as extra safety

* fix: add better error messages/docs for catching empty inputs (privacy-scaling-explorations#18)

* chore: add Cargo.lock

* chore: update Cargo dependencies

* feat: fix versions and tags for dependencies

---------

Co-authored-by: Han <[email protected]>
Co-authored-by: DoHoon Kim <[email protected]>
Co-authored-by: Chih Cheng Liang <[email protected]>
Co-authored-by: Carlos Pérez <[email protected]>
Co-authored-by: dante <[email protected]>
Co-authored-by: Jonathan Wang <[email protected]>
zemse pushed a commit to zemse/snark-verifier that referenced this pull request Feb 27, 2024
* Rollback to pse halo2 and halo2wrong for first release (privacy-scaling-explorations#5)

* feat: move `Accumulator` to `accumulator.rs`

* feat: update due to halo2

* feat: upgrade to use branch `feature/generic-instructions` of `halo2wrong`

* refactor: rollback to `{halo2,halo2_wrong}` without challenge API and cleanup dependencies

* chore: rename statement to instance and auxliary to witness

* chore: use `finalize` instead of `code`

* feat: add `Code::deployment` and `EvmLoader::deployment_code`; add example `evm-verifier-codegen`

* fix: typo

* feat: reduce generated evm verifier size; rename to `evm-verifier` and add another example `evm-verifier-with-accumulator`

* fix: due to `halo2wrong`

* feat: reorganize mods and traits

* fix: allow empty `values` in `sum_*` and move them under `ScalarLoader`

* ci: use `--all-features` for `cargo test`

* fix: use same strategy for aggregation testing

* fix: simplify trait `PlonkVerifier` again

* fix: move system specified transcript under mod `system`

* feat: add `quotient_poly` info in `Protocol`

* feat: implement linearization for circom integration

* feat: re-export loader's dependency for consumer

* refactor: for circom's integration

* tmp: pin `revm` to rev

* fix: remove parentheses

* fix: upgrade for multi-phase halo2

* feat: improve error reporting

* chore: rename crate to snake case

* feat: add `Domain` as an input of `PolynomialCommitmentScheme::read_proof`

* refactor: for further integration

* feat: generalize to suppoer also ipa and add builder fns to `system::halo2::Config`

* feat: add `KzgDecider` for simple evm verifier

* refactor: split `AccumulationScheme` and `AccumulatorEncoding`

* refactor: split `PolynomialCommitmentScheme` and `MultiOpenScheme`

* fix: not need sealed actually

* fix: `chunk_size` should be `LIMBS` when recovering accumulator

* feat: add `Expression::DistributePowers` to avoid stack overflow

* fix: update and pin foundry

* fix: move testing circuits under `system/halo2`

* fix: allow accumulate single accumulator

* feat: remove all patch and make less depending `halo2wrong`

* Generalized `Halo2Loader` (privacy-scaling-explorations#12)

* feat: generalize `Protocol` for further usage

* feat: add `EccInstruction::{fixed_base_msm,variable_base_msm,sum_with_const}`

* chore: move `rand_chacha` as dev dependency

* General refactor for further integration (privacy-scaling-explorations#13)

* feat: remove dev-dependency `foundry` and vendor necessary part of it

* refactor: simplify traits and remove unused stuff

* refactor: much less clone

* feat: generalized `AccumulatorEncoding` for `EccInstructions`

* feat: implement ipa pcs and accumulation (privacy-scaling-explorations#14)

* ci: add `svm-rs` and install `[email protected]` in job `test` (privacy-scaling-explorations#16)

* Update `EvmLoader` to generate Yul code instead of bytecode (privacy-scaling-explorations#15)

* Update `EvmLoader` to generate Yul instead of bytecode

* feat: simplify

* feat: Add missing end_gas_metering impl

Co-authored-by: Han <[email protected]>

Co-authored-by: Han <[email protected]>

* fix: pin all `revm` dependencies (privacy-scaling-explorations#18)

* fix: looser trait bound on impl `CostEstimation` for `Plonk` (privacy-scaling-explorations#20)

* Restructure for more kind of verifier (privacy-scaling-explorations#21)

* feat: restructure to monorepo and expand the project scope to be generic (s)nark verifier

* feat: reorganize mods and traits for further new features

* refactor: simplify trait bounds

* chore: use hyphen case for crate name (`snark_verifier` -> `snark-verifier`)

* docs: add `#![deny(missing_docs)]` and simple documents

* refactor: remove redudant check `validate_ec_point` (still doesn not support identity)

* feat: expand more things and fix typos

Co-authored-by: Chih Cheng Liang <[email protected]>
Co-authored-by: Carlos Pérez <[email protected]>

* fix: rustdoc warnings

* chore: update dependencies (privacy-scaling-explorations#24)

* chore: update `halo2` and `halo2wrong` version (privacy-scaling-explorations#25)

* fix: enable `util::hash::poseidon` only when `feature = loader_halo2` (privacy-scaling-explorations#27)

* feat: working update to halo2-lib v0.3.0

* feat: update zkevm bench

* feat: update recursion example

* feat: switch poseidon native implementation to Scroll's audited version

* fix: invert determinant only once in Cramer's rule

* chore: fix doc

* chore

* chore: forgot to update halo2-base dependency in snark-verifier-sdk

* Minor update (privacy-scaling-explorations#8)

* feat(sdk): remove duplicate code in
`RangeWithInstanceCircuitBuilder::synthesize`

* feat(sdk): Proof caching when using feature 'halo2-pse'

* chore: sync with halo2-lib

* chore: switch to halo2-lib release-0.3.0 branch

* Moved `RangeWithInstanceCircuitBuilder` to halo2-lib (privacy-scaling-explorations#9)

* chore: sync with halo2-lib

* fix: clippy

* chore: fix halo2-base branch in sdk

* feat: update to halo2-lib new types (privacy-scaling-explorations#10)

* feat: add `assert` for non-empty accumulators in `decide_all` (privacy-scaling-explorations#11)

* feat: use `zip_eq` for `Polynomial` add/sub (privacy-scaling-explorations#12)

* fix: git CI turn off all features

* fix: `rotate_scalar` misbehaves on `i32::MIN` (privacy-scaling-explorations#13)

Should never actually be callable with such a large negative rotation

* fix: cleanup code quality (privacy-scaling-explorations#14)

* fix: `split_by_ascii_whitespace` (privacy-scaling-explorations#15)

* fix: `batch_invert_and_mul` do not allow zeros (privacy-scaling-explorations#16)

* feat: verify proof in release mode (privacy-scaling-explorations#17)

Verify proof before caching it as extra safety

* fix: add better error messages/docs for catching empty inputs (privacy-scaling-explorations#18)

* chore: add Cargo.lock

* chore: update Cargo dependencies

* chore: update dependencies

* Minor: merge v0.1.1 to develop (privacy-scaling-explorations#21)

Just cargo fixes

* feat: remove use of env vars for circuit configuration (privacy-scaling-explorations#22)

* feat: remove use of env vars for circuit configuration

This is a companion to axiom-crypto/halo2-lib#92

* chore: remove rustfmt CI check

PSE upstream uses different rustfmt configuration than us, so some files
disagree in formatting

* chore: fix dependencies

* Feat/read pk buffer capacity (privacy-scaling-explorations#24)

* feat: change default `read_pk` buffer capacity to 1MB

* feat: add bench for read_pk

* [Update] use ff v0.13 (privacy-scaling-explorations#28)

* feat(snark-verifier): update to ff v0.13

* feat(snark-verifier): update examples

* feat(snark-verifier-sdk): update to ff v0.13

* fix: conversion from BaseConfigParams to AggregationConfigParams

* chore: pin poseidon rev

* refactor(sdk): add `AggregationCtxBuilder` for aggregation

Contains the populated builder after aggregating, without creating the
`AggregationCircuit`. Doesn't need config parameters and break points.

* chore: update cargo

* [Feat] Universal verifier circuit (privacy-scaling-explorations#26)

* feat: add example with different vkey as private witness

Same aggregation circuit, verifies different snarks with different vkeys
(same standard plonk gate, but different selectors / copy constraints)

* fix: save break points when generating agg circuit for first time (privacy-scaling-explorations#23)

* fix: save break points when generating agg circuit for first time

* chore: add circuit files to gitignore

* feat: halo2-lib universal verifier example

* chore: cargo fix

* feat: allow circuit size (number of rows) to be loaded as witness

* chore: clippy fix

* fix(n_as_witness): computation of shifts depends on `omega`

`omega` which changes when `k` changes, so all shift computations need
to be done as witness. Current implementation is likely not the most
optimal. Instead of storing `shift` as `omega^i`, we store just
`Rotation(i)`. We de-duplicate when possible using `BTreeMap` of
`Rotation`. Note you must use `Rotation` instead of `F` for `BTreeMap`
because the ordering of `omega^i` may change depending on `omega`.

* fix: temp remove pow_var

* add universal verifier range check test

* chore: do not serialize domain_as_witness if none

* Revert "fix: temp remove pow_var"

This reverts commit 69f648e.

* fix: halo2_lib example

* test: halo2_lib with variable lookup table passes

* Bump version to 0.1.3

---------

Co-authored-by: Roshan <[email protected]>

* chore: derive Default for VerifierUniversality

* feat: upgrade `revm` to support lastest hardfork (privacy-scaling-explorations#40)

* Update: use `halo2-lib` v0.4.0 (privacy-scaling-explorations#29)

* feat: update snark-verifier

* update: use `halo2-lib` v0.4.0

* feat: load `k` as witness and compute `n = 2^k` and `omega` from `k` (privacy-scaling-explorations#30)

* feat: load `k` as witness and compute `n = 2^k` and `omega` from `k`

Removes need to make `omega` a public output in universal verifier.

* fix: bit_length

* Move `OptimizedPoseidonSpec` to `halo2-base` (privacy-scaling-explorations#31)

* chore: move `OptimizedPoseidonSpec` to `halo2-base`

* Bump version to 0.1.5 and remove poseidon-rs dep

* chore: util::hash available without loader_halo2 feature

* chore: nit

* [feat] change yul code into Solidity assembly (privacy-scaling-explorations#32)

feat: change yul code into Solidity assembly

Just changes to wrapping yul in solidity assembly block

* chore: change halo2-lib branch

* chore: `snark-verifier` re-exports `halo2-base`, `halo2-ecc` (privacy-scaling-explorations#38)

and `snark-verifier-sdk` re-exports `snark-verifier`. This is to
minimize cargo dependency / version issues until we publish to crates.io.

* [feat] make assembly block Solidity memory safe (privacy-scaling-explorations#39)

* feat: make assembly block Solidity memory safe

* chore: add back example solidity code

* chore: switch CI to solidity v0.8.19

* [chore] add `cargo audit` and update dependencies (privacy-scaling-explorations#40)

* chore: add `cargo audit` and update dependencies

disabling zkevm benches due to versioning issues

* chore: autobenches false

* fix: audit-check CI

* [chore] fix hyperlinks in docs (privacy-scaling-explorations#41)

chore: fix hyperlinks in docs

* [chore] separate `revm` import into separate feature (privacy-scaling-explorations#44)

* chore: import `ruint` separately to not rely on `revm`

* feat: add feature `revm` to toggle `revm` import

Separates `revm` import, which is only used for testing EVM execution,
from `loader_evm` feature, which is used more broadly for generating
proofs to be sent to on-chain verifier.

* chore: add CI to check `--release` compilation with assembly

* fix: import under revm

* fix: compute `system_challenge_offset` correctly (privacy-scaling-explorations#53)

* chore: turn off halo2-pse for crates.io

* chore: add package descriptions

* chore: update description

* chore: add snark-verifier version

---------

Co-authored-by: Han <[email protected]>
Co-authored-by: DoHoon Kim <[email protected]>
Co-authored-by: Chih Cheng Liang <[email protected]>
Co-authored-by: Carlos Pérez <[email protected]>
Co-authored-by: dante <[email protected]>
Co-authored-by: Jonathan Wang <[email protected]>
Co-authored-by: Roshan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant