Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: BW6-761 emulated pairing #846

Merged
merged 41 commits into from
Oct 17, 2023
Merged

feat: BW6-761 emulated pairing #846

merged 41 commits into from
Oct 17, 2023

Conversation

yelhousni
Copy link
Contributor

@yelhousni yelhousni commented Oct 4, 2023

Description

This PR implements BW6-761 emulated pairing circuit. It is based on #589 (mainly for the tower arithmetic). It follows https://hackmd.io/@gnark/BW6-761-changes:

All these equations are implemented using similar tricks to BN254 and BLS12-381 emulated pairings (tower hints, affine coordinates, sparser lines, iterations isolation...).
I was expecting (4') to be the optimal choice, for a single pairing, in terms of constraints but it seems not. I will need to try with the optimal Tate equations.

Todo:

  • equation (5)
  • equation (5')
  • equation (6) ⭐️ 7fc5925
  • equation (6')
  • Torus-based final exponentiation
    (not enough multiplications in Expt for torus-based FE to be worth it vs. 46 Karabina squarings in a row)

Type of change

  • New feature (non-breaking change which adds functionality)

How has this been tested?

  • Pairing (and multi-pairing) and final exponentiation results compared to gnark-crypto. This needs to change the gnark-crypto branch to feat/bw6761-pairing. The final chosen algorithm is the same as gnark-crypto master, so no nothing to do there anymore.

How has this been benchmarked?

  • Number of constraints:
Miller Loop Final exp 1 pairing 2 pairings
R1CS 1,548,496 1,037,242 2,585,738 3,746,835
SCS 16,520,533 11,372,377 27,892,910 39,656,303

Checklist:

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • I did not modify files generated from templates
  • golangci-lint does not output errors locally
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

@yelhousni yelhousni added this to the v0.9.0 milestone Oct 4, 2023
@yelhousni yelhousni requested a review from ivokub October 4, 2023 14:14
@yelhousni yelhousni self-assigned this Oct 4, 2023
@yelhousni yelhousni marked this pull request as draft October 4, 2023 14:14
std/algebra/emulated/sw_bw6761/pairing.go Outdated Show resolved Hide resolved
std/algebra/emulated/sw_bw6761/pairing.go Outdated Show resolved Hide resolved
std/algebra/emulated/sw_bw6761/pairing.go Outdated Show resolved Hide resolved
std/algebra/emulated/fields_bw6761/e6_pairing.go Outdated Show resolved Hide resolved
std/algebra/emulated/fields_bw6761/e3.go Show resolved Hide resolved
std/algebra/emulated/fields_bw6761/e3.go Outdated Show resolved Hide resolved
std/algebra/emulated/fields_bw6761/e3.go Outdated Show resolved Hide resolved
std/algebra/emulated/fields_bw6761/e3.go Outdated Show resolved Hide resolved
std/algebra/emulated/fields_bw6761/e6.go Outdated Show resolved Hide resolved
std/algebra/emulated/fields_bw6761/e6.go Outdated Show resolved Hide resolved
@yelhousni yelhousni requested a review from ivokub October 5, 2023 23:35
Copy link
Collaborator

@ivokub ivokub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think almost good. I'm not understanding the thing with Ext6.Expc1, but I guess it is some ongoing optimization? Otherwise, there is still one open issue from last review (method signature to take *baseEl instead of baseEl for consistency)

std/algebra/emulated/fields_bw6761/e3.go Outdated Show resolved Hide resolved
std/algebra/emulated/sw_bw6761/pairing_test.go Outdated Show resolved Hide resolved
@yelhousni
Copy link
Contributor Author

I think almost good. I'm not understanding the thing with Ext6.Expc1, but I guess it is some ongoing optimization? Otherwise, there is still one open issue from last review (method signature to take *baseEl instead of baseEl for consistency)

Re-posting here for visibility #846 (comment).

@github-actions
Copy link

Summary

✅ Passed: 5737
❌ Failed: 0
🚧 Skipped: 21

🚧 Skipped

  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls12-377/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls12-377/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls12-381/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls12-381/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls24-315/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls24-315/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls24-317/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls24-317/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bn254/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bw6-633/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bw6-633/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bw6-761/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bw6-761/mpcsetup)
  • TestCircuitInclusionProof (github.com/consensys/gnark/examples/rollup)
  • TestCircuitUpdateAccount (github.com/consensys/gnark/examples/rollup)
  • TestCircuitFull (github.com/consensys/gnark/examples/rollup)
  • TestFrobeniusFp12 (github.com/consensys/gnark/std/algebra/native/fields_bls12377)
  • TestMulByNonResidueFp2 (github.com/consensys/gnark/std/algebra/native/fields_bls12377)
  • TestMulByFp2Fp6 (github.com/consensys/gnark/std/algebra/native/fields_bls12377)
  • TestIssue348UnconstrainedLimbs (github.com/consensys/gnark/std/math/emulated)
  • TestSolverConsistency (github.com/consensys/gnark/test)

@github-actions
Copy link

Summary

✅ Passed: 5737
❌ Failed: 0
🚧 Skipped: 21

🚧 Skipped

  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls12-377/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls12-377/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls12-381/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls12-381/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls24-315/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls24-315/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls24-317/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls24-317/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bn254/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bw6-633/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bw6-633/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bw6-761/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bw6-761/mpcsetup)
  • TestCircuitInclusionProof (github.com/consensys/gnark/examples/rollup)
  • TestCircuitUpdateAccount (github.com/consensys/gnark/examples/rollup)
  • TestCircuitFull (github.com/consensys/gnark/examples/rollup)
  • TestFrobeniusFp12 (github.com/consensys/gnark/std/algebra/native/fields_bls12377)
  • TestMulByNonResidueFp2 (github.com/consensys/gnark/std/algebra/native/fields_bls12377)
  • TestMulByFp2Fp6 (github.com/consensys/gnark/std/algebra/native/fields_bls12377)
  • TestIssue348UnconstrainedLimbs (github.com/consensys/gnark/std/math/emulated)
  • TestSolverConsistency (github.com/consensys/gnark/test)

@ivokub
Copy link
Collaborator

ivokub commented Oct 11, 2023

SCS cost when rebasing #749 on top of this:

OP SCS Count
FinalExp 6008875
MillerLoop 5973754
1 pairing 11722930
2 pairing 14963190

@github-actions
Copy link

Summary

✅ Passed: 5737
❌ Failed: 0
🚧 Skipped: 21

🚧 Skipped

  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls12-377/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls12-377/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls12-381/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls12-381/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls24-315/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls24-315/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls24-317/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls24-317/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bn254/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bw6-633/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bw6-633/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bw6-761/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bw6-761/mpcsetup)
  • TestCircuitInclusionProof (github.com/consensys/gnark/examples/rollup)
  • TestCircuitUpdateAccount (github.com/consensys/gnark/examples/rollup)
  • TestCircuitFull (github.com/consensys/gnark/examples/rollup)
  • TestFrobeniusFp12 (github.com/consensys/gnark/std/algebra/native/fields_bls12377)
  • TestMulByNonResidueFp2 (github.com/consensys/gnark/std/algebra/native/fields_bls12377)
  • TestMulByFp2Fp6 (github.com/consensys/gnark/std/algebra/native/fields_bls12377)
  • TestIssue348UnconstrainedLimbs (github.com/consensys/gnark/std/math/emulated)
  • TestSolverConsistency (github.com/consensys/gnark/test)

@github-actions
Copy link

Summary

✅ Passed: 5737
❌ Failed: 0
🚧 Skipped: 21

🚧 Skipped

  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls12-377/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls12-377/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls12-381/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls12-381/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls24-315/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls24-315/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bls24-317/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bls24-317/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bn254/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bw6-633/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bw6-633/mpcsetup)
  • TestContributionSerialization (github.com/consensys/gnark/backend/groth16/bw6-761/mpcsetup)
  • TestSetupCircuit (github.com/consensys/gnark/backend/groth16/bw6-761/mpcsetup)
  • TestCircuitInclusionProof (github.com/consensys/gnark/examples/rollup)
  • TestCircuitUpdateAccount (github.com/consensys/gnark/examples/rollup)
  • TestCircuitFull (github.com/consensys/gnark/examples/rollup)
  • TestFrobeniusFp12 (github.com/consensys/gnark/std/algebra/native/fields_bls12377)
  • TestMulByNonResidueFp2 (github.com/consensys/gnark/std/algebra/native/fields_bls12377)
  • TestMulByFp2Fp6 (github.com/consensys/gnark/std/algebra/native/fields_bls12377)
  • TestIssue348UnconstrainedLimbs (github.com/consensys/gnark/std/math/emulated)
  • TestSolverConsistency (github.com/consensys/gnark/test)

@yelhousni
Copy link
Contributor Author

SCS cost when rebasing #749 on top of this:

OP SCS Count
FinalExp 6008875
MillerLoop 5973754
1 pairing 11722930
2 pairing 14963190

After rebasing #749 on top of last commit cb9ff67, I get different SCS counts (e.g. 2 pairings 16713265). Normally without #749 the SCS count reduced from 44M to 39M for 2 pairings. Otherwise, I think we're good to go with this PR.

@ivokub
Copy link
Collaborator

ivokub commented Oct 11, 2023

SCS cost when rebasing #749 on top of this:

OP SCS Count
FinalExp 6008875
MillerLoop 5973754
1 pairing 11722930
2 pairing 14963190

After rebasing #749 on top of last commit cb9ff67, I get different SCS counts (e.g. 2 pairings 16713265). Normally without #749 the SCS count reduced from 44M to 39M for 2 pairings. Otherwise, I think we're good to go with this PR.

May it be because I measured without ExpC1 call and without asserting equality to one

@yelhousni
Copy link
Contributor Author

SCS cost when rebasing #749 on top of this:
OP SCS Count
FinalExp 6008875
MillerLoop 5973754
1 pairing 11722930
2 pairing 14963190

After rebasing #749 on top of last commit cb9ff67, I get different SCS counts (e.g. 2 pairings 16713265). Normally without #749 the SCS count reduced from 44M to 39M for 2 pairings. Otherwise, I think we're good to go with this PR.

May it be because I measured without ExpC1 call and without asserting equality to one

hmm even with that it's 16610107 so I'm not sure if there is a regression when combined with #749 ..

@github-actions
Copy link

📦 github.com/consensys/gnark/std/algebra/emulated/sw_bw6761
TestFinalExponentiationTestSolve 7.28s

    pairing_test.go:61: 
        	Error Trace:	/home/runner/work/gnark/gnark/std/algebra/emulated/sw_bw6761/pairing_test.go:61
        	Error:      	Received unexpected error:
        	            	[assertIsEqual] 4043109907697756569110837963364858198648652661637310544680583168 == 4043109907697756569110838191677820252800915128780054250950959763
        	            	emulated.(*Field[...]).rsh
        	            		field_assert.go:56
        	            	emulated.(*Field[...]).assertLimbsEqualitySlow
        	            		field_assert.go:35
        	            	emulated.(*Field[...]).AssertLimbsEquality
        	            		field_assert.go:86
        	            	emulated.(*Field[...]).AssertIsEqual
        	            		field_assert.go:144
        	            	fields_bw6761.Ext3.AssertIsEqual
        	            		e3.go:382
        	            	fields_bw6761.Ext6.AssertIsEqual
        	            		e6.go:362
        	            	sw_bw6761.Pairing.AssertIsEqual
        	            		pairing.go:136
        	            	sw_bw6761.(*FinalExponentiationCircuit).Define
        	            		pairing_test.go:47
        	Test:       	TestFinalExponentiationTestSolve

TestMultiPairTestSolve 10.35s

    pairing_test.go:141: 
        	Error Trace:	/home/runner/work/gnark/gnark/std/algebra/emulated/sw_bw6761/pairing_test.go:141
        	Error:      	Received unexpected error:
        	            	[assertIsEqual] 4043112557480694090907116479823769882374099025771673745203658752 == 4043112557480694090907116752333972745856101443798687026332477818
        	            	emulated.(*Field[...]).rsh
        	            		field_assert.go:56
        	            	emulated.(*Field[...]).assertLimbsEqualitySlow
        	            		field_assert.go:35
        	            	emulated.(*Field[...]).AssertLimbsEquality
        	            		field_assert.go:86
        	            	emulated.(*Field[...]).AssertIsEqual
        	            		field_assert.go:144
        	            	fields_bw6761.Ext3.AssertIsEqual
        	            		e3.go:382
        	            	fields_bw6761.Ext6.AssertIsEqual
        	            		e6.go:362
        	            	sw_bw6761.Pairing.AssertIsEqual
        	            		pairing.go:136
        	            	sw_bw6761.(*MultiPairCircuit).Define
        	            		pairing_test.go:118
        	Test:       	TestMultiPairTestSolve

TestPairTestSolve 10.59s

    pairing_test.go:94: 
        	Error Trace:	/home/runner/work/gnark/gnark/std/algebra/emulated/sw_bw6761/pairing_test.go:94
        	Error:      	Received unexpected error:
        	            	[assertIsEqual] 4043109475235519274826500990840458599565201170399850156791955456 == 4043109475235519274826501138481677465620205991927730968288058791
        	            	emulated.(*Field[...]).rsh
        	            		field_assert.go:56
        	            	emulated.(*Field[...]).assertLimbsEqualitySlow
        	            		field_assert.go:35
        	            	emulated.(*Field[...]).AssertLimbsEquality
        	            		field_assert.go:86
        	            	emulated.(*Field[...]).AssertIsEqual
        	            		field_assert.go:144
        	            	fields_bw6761.Ext3.AssertIsEqual
        	            		e3.go:382
        	            	fields_bw6761.Ext6.AssertIsEqual
        	            		e6.go:362
        	            	sw_bw6761.Pairing.AssertIsEqual
        	            		pairing.go:136
        	            	sw_bw6761.(*PairCircuit).Define
        	            		pairing_test.go:79
        	Test:       	TestPairTestSolve

@yelhousni
Copy link
Contributor Author

📦 github.com/consensys/gnark/std/algebra/emulated/sw_bw6761TestFinalExponentiationTestSolve 7.28s

    pairing_test.go:61: 
        	Error Trace:	/home/runner/work/gnark/gnark/std/algebra/emulated/sw_bw6761/pairing_test.go:61
        	Error:      	Received unexpected error:
        	            	[assertIsEqual] 4043109907697756569110837963364858198648652661637310544680583168 == 4043109907697756569110838191677820252800915128780054250950959763
        	            	emulated.(*Field[...]).rsh
        	            		field_assert.go:56
        	            	emulated.(*Field[...]).assertLimbsEqualitySlow
        	            		field_assert.go:35
        	            	emulated.(*Field[...]).AssertLimbsEquality
        	            		field_assert.go:86
        	            	emulated.(*Field[...]).AssertIsEqual
        	            		field_assert.go:144
        	            	fields_bw6761.Ext3.AssertIsEqual
        	            		e3.go:382
        	            	fields_bw6761.Ext6.AssertIsEqual
        	            		e6.go:362
        	            	sw_bw6761.Pairing.AssertIsEqual
        	            		pairing.go:136
        	            	sw_bw6761.(*FinalExponentiationCircuit).Define
        	            		pairing_test.go:47
        	Test:       	TestFinalExponentiationTestSolve

TestMultiPairTestSolve 10.35s

    pairing_test.go:141: 
        	Error Trace:	/home/runner/work/gnark/gnark/std/algebra/emulated/sw_bw6761/pairing_test.go:141
        	Error:      	Received unexpected error:
        	            	[assertIsEqual] 4043112557480694090907116479823769882374099025771673745203658752 == 4043112557480694090907116752333972745856101443798687026332477818
        	            	emulated.(*Field[...]).rsh
        	            		field_assert.go:56
        	            	emulated.(*Field[...]).assertLimbsEqualitySlow
        	            		field_assert.go:35
        	            	emulated.(*Field[...]).AssertLimbsEquality
        	            		field_assert.go:86
        	            	emulated.(*Field[...]).AssertIsEqual
        	            		field_assert.go:144
        	            	fields_bw6761.Ext3.AssertIsEqual
        	            		e3.go:382
        	            	fields_bw6761.Ext6.AssertIsEqual
        	            		e6.go:362
        	            	sw_bw6761.Pairing.AssertIsEqual
        	            		pairing.go:136
        	            	sw_bw6761.(*MultiPairCircuit).Define
        	            		pairing_test.go:118
        	Test:       	TestMultiPairTestSolve

TestPairTestSolve 10.59s

    pairing_test.go:94: 
        	Error Trace:	/home/runner/work/gnark/gnark/std/algebra/emulated/sw_bw6761/pairing_test.go:94
        	Error:      	Received unexpected error:
        	            	[assertIsEqual] 4043109475235519274826500990840458599565201170399850156791955456 == 4043109475235519274826501138481677465620205991927730968288058791
        	            	emulated.(*Field[...]).rsh
        	            		field_assert.go:56
        	            	emulated.(*Field[...]).assertLimbsEqualitySlow
        	            		field_assert.go:35
        	            	emulated.(*Field[...]).AssertLimbsEquality
        	            		field_assert.go:86
        	            	emulated.(*Field[...]).AssertIsEqual
        	            		field_assert.go:144
        	            	fields_bw6761.Ext3.AssertIsEqual
        	            		e3.go:382
        	            	fields_bw6761.Ext6.AssertIsEqual
        	            		e6.go:362
        	            	sw_bw6761.Pairing.AssertIsEqual
        	            		pairing.go:136
        	            	sw_bw6761.(*PairCircuit).Define
        	            		pairing_test.go:79
        	Test:       	TestPairTestSolve

Should work with Consensys/gnark-crypto#459

Copy link
Collaborator

@ivokub ivokub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ivokub
Copy link
Collaborator

ivokub commented Oct 12, 2023

SCS cost when rebasing #749 on top of this:
OP SCS Count
FinalExp 6008875
MillerLoop 5973754
1 pairing 11722930
2 pairing 14963190

After rebasing #749 on top of last commit cb9ff67, I get different SCS counts (e.g. 2 pairings 16713265). Normally without #749 the SCS count reduced from 44M to 39M for 2 pairings. Otherwise, I think we're good to go with this PR.

May it be because I measured without ExpC1 call and without asserting equality to one

hmm even with that it's 16610107 so I'm not sure if there is a regression when combined with #749 ..

Yes, I'm now getting the same results you are. I guess I removed too much when I was measuring :/ Or it may also be that I had some dirty state in the tree as I was debugging the dereferences.

In any case, I think the PR is good to go - we only have to merge Consensys/gnark-crypto#459 first and then update go.mod here.

Great work!

@ivokub ivokub mentioned this pull request Oct 13, 2023
10 tasks
@yelhousni yelhousni mentioned this pull request Oct 16, 2023
9 tasks
@yelhousni
Copy link
Contributor Author

In any case, I think the PR is good to go - we only have to merge Consensys/gnark-crypto#459 first and then update go.mod here.

PR Consensys/gnark-crypto#459 is merged and go.mod updated here. So I'm merging and closing this.

@yelhousni yelhousni merged commit 063cad6 into master Oct 17, 2023
7 checks passed
@yelhousni yelhousni deleted the feat/bw6761-pairing branch October 17, 2023 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants