-
Notifications
You must be signed in to change notification settings - Fork 33
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'feat/integrate-bg-router' into chore/introduce-submodule
* feat/integrate-bg-router: Update E2E_FEATURE_LIST.md [IBC] Implement ICS-23 CommitmentProof verification for the SMT (#845) [Utility] trustless relays servicer token validation (#803)
- Loading branch information
Showing
28 changed files
with
1,222 additions
and
82 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| # ICS-23 Vector Commitments <!-- omit in toc --> | ||
|
|
||
| - [Overview](#overview) | ||
| - [Implementation](#implementation) | ||
| - [Custom SMT `ProofSpec`](#custom-smt-proofspec) | ||
| - [Converting `SparseMerkleProof` to `CommitmentProof`](#converting-sparsemerkleproof-to-commitmentproof) | ||
| - [Proof Verification](#proof-verification) | ||
|
|
||
| ## Overview | ||
|
|
||
| [ICS-23][ics23] defines the types and functions needed to verify membership of a key-value pair in a `CommitmentState`. As the Pocket IBC implementation uses the [SMT][smt] for its provable stores, this is referred to as the `CommitmentState` object. Cosmos has a library `cosmos/ics23` which is already SDK agnostic and defines many of the types necessary for ICS-23. This library was able to be used _mostly_ out of the box, with some minor adjustments detailed below. | ||
|
|
||
| ## Implementation | ||
|
|
||
| The benefit of using `cosmos/ics23` over implementing similar types ourselves is twofold: | ||
|
|
||
| 1. It is already SDK agnostic, so can be used by Pocket (a non-cosmos chain) without any issues or major changes. | ||
| 2. The functions defined for proof verification are decoupled from the underlying tree structure, meaning proof verification is tree agnostic. | ||
|
|
||
| However, there were some changes made specifically for Pocket's implementation of ICS-23. | ||
|
|
||
| See: [`cosmos/ics23` #152](https://github.com/cosmos/ics23/issues/152) and [`cosmos/ics23` #153](https://github.com/cosmos/ics23/pull/153ß) for the details of the changes made to allow for `ExclusionProof` verification. | ||
|
|
||
| ### Custom SMT `ProofSpec` | ||
|
|
||
| The `ProofSpec` type in `cosmos/ics23` is used to define: | ||
|
|
||
| 1. The steps needed to verify a proof | ||
| 2. The hash functions used | ||
| 3. Node prefixes | ||
| 4. Etc... | ||
|
|
||
| The `ProofSpec` is then passed into the verification functions in order to verify a proof instead of having to interact with the tree itself. This is useful as proofs must be verified via an (IBC) light client, and as such being able to verify a proof without reconstructing a tree is much more memory efficient. | ||
|
|
||
| As the SMT used by Pocket Network only stores hashed values by default, the IBC store uses the `WithValueHasher(nil)` option which stores the source value (as raw bytes) in the tree. The following `ProofSpec` was created to support this: | ||
|
|
||
| ```go | ||
| smtSpec *ics23.ProofSpec = &ics23.ProofSpec{ | ||
| LeafSpec: &ics23.LeafOp{ | ||
| Hash: ics23.HashOp_SHA256, | ||
| PrehashKey: ics23.HashOp_SHA256, | ||
| PrehashValue: ics23.HashOp_NO_HASH, | ||
| Length: ics23.LengthOp_NO_PREFIX, | ||
| Prefix: []byte{0}, | ||
| }, | ||
| InnerSpec: &ics23.InnerSpec{ | ||
| ChildOrder: []int32{0, 1}, | ||
| ChildSize: 32, | ||
| MinPrefixLength: 1, | ||
| MaxPrefixLength: 1, | ||
| EmptyChild: make([]byte, 32), | ||
| Hash: ics23.HashOp_SHA256, | ||
| }, | ||
| MaxDepth: 256, | ||
| PrehashKeyBeforeComparison: true, | ||
| } | ||
| ``` | ||
|
|
||
| The main difference from the `cosmos/ics23` `SmtSpec` object is that the `PrehashValue` field is set to not hash values before hashing the key-value pair. | ||
|
|
||
| ### Converting `SparseMerkleProof` to `CommitmentProof` | ||
|
|
||
| In order to convert the proofs generated by the SMT into a serialisable proof used by `cosmos/ics23`, the `SideNodes` field of the `SparseMerkleProof` must be converted into a list of `InnerOp` types which define the order of the hashes. The order of the hashes is important as depending on whether the next hash is the left or right neighbour of the current hash, they will be hashed in a different order, ultimately creating a different root hash. This conversion allows the verification to produce the same root hash as the SMT would have produced when verifying the proof. | ||
|
|
||
| As `SparseMerkleProof` objects represent both inclusion and exclusion proofs as defined in the [JMT whitepaper][jmt]. The conversion step will convert the SMT proof into either an `ExistenceProof` or `ExclusionProof` as defined in `cosmos/ics23`. | ||
|
|
||
| ### Proof Verification | ||
|
|
||
| Membership proofs are verified as follows: | ||
|
|
||
| 1. Use the key-value pair to generate a leaf hash | ||
| 2. Hash the leaf with the `SideNodes` found in the `path` field of the `ExistenceProof` to generate the root hash | ||
| 3. Compare the root hash with the one provided and expect them to be identical | ||
|
|
||
| Non-membership proofs are verified as follows: | ||
|
|
||
| 1. If the `ActualValueHash` field in the `ExclusionProof` is the SMT's placeholder value (`[32]byte`, i.e. the key is not set in the tree), then use the placeholder value as the leaf node hash and skip to step 3 below | ||
| 2. If the `ActualValueHash` field is not the placeholder value, then use the `ActualPath` and `ActualValueHash` fields (provided via `NonMembershipLeafData`) to generate the leaf node hash. | ||
| - **IMPORTANT**: DO NOT hash these values before hashing the node as they are populated from the SMT proof's `NonMembershipLeafData` field and thus are already hashed | ||
| 3. Hash the leaf node hash with the `SideNodes` found in the `Path` field of the `ExclusionProof` to generate the root hash | ||
| 4. Compare the root hash with the one provided | ||
| - if `computedRootHash == providedRootHash` | ||
| - `key` not in tree -> `Proof` is valid -> exclusion QED | ||
| - if `computedRootHash != providedRootHash` | ||
| - `key` is in tree -> `Proof` is invalid -> exclusion QED | ||
|
|
||
| ```mermaid | ||
| flowchart TD | ||
| I["Proof,Key"] | ||
| NMD{"proof.NonMembershipLeafData == nil ?"} | ||
| KP1["actualPath = sha256(key) \n actualValue = placeholder\ncurrentHash = [32]byte"] | ||
| KP2["actualPath = ProvidedKeyHash \n actualValue = ProvidedValueHash\ncurrentHash = sha256([]byte{0}+actualPath+actualValueHash)"] | ||
| C["nextHash = sha256(currentHash+sideNodeHash)"] | ||
| Compare{"ComputedRootHash == ProvidedRootHash ?"} | ||
| EV["Exclusion Prove VALID"] | ||
| EI["Exclusion Prove INVALID"] | ||
| I --> NMD | ||
| NMD -- Yes --> KP1 | ||
| NMD -- No --> KP2 | ||
| KP1 -- CurrentHash --> C | ||
| KP2 -- CurrentHash --> C | ||
| C -- while NextSideNode != nil --> C | ||
| C -- ComputedRootHash --> Compare | ||
| Compare -- Yes --> EV | ||
| Compare -- No --> EI | ||
| ``` | ||
|
|
||
| The full implementation of this logic can be found [here](../store/proofs_ics23.go) as well as in the `cosmos/ics23` [library](https://github.com/h5law/ics23/blob/56d948cafb83ded78dc4b9de3c8b04582734851a/go/proof.go#L171). | ||
|
|
||
| [ics23]: https://github.com/cosmos/ibc/blob/main/spec/core/ics-023-vector-commitments/README.md | ||
| [smt]: https://github.com/pokt-network/smt | ||
| [jmt]: https://developers.diem.com/papers/jellyfish-merkle-tree/2021-01-14.pdf |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,159 @@ | ||
| package store | ||
|
|
||
| import ( | ||
| "crypto/sha256" | ||
|
|
||
| ics23 "github.com/cosmos/ics23/go" | ||
| coreTypes "github.com/pokt-network/pocket/shared/core/types" | ||
| "github.com/pokt-network/smt" | ||
| ) | ||
|
|
||
| // position refers to whether the node is either the left or right child of its parent | ||
| // for the binary SMT | ||
| // Ref: https://github.com/pokt-network/smt/blob/main/types.go | ||
| const ( | ||
| left int = iota // 0 | ||
| right // 1 | ||
| hashSize = 32 | ||
| ) | ||
|
|
||
| var ( | ||
| // Custom SMT spec as the store does not hash values | ||
| smtSpec *ics23.ProofSpec = &ics23.ProofSpec{ | ||
| LeafSpec: &ics23.LeafOp{ | ||
| Hash: ics23.HashOp_SHA256, | ||
| PrehashKey: ics23.HashOp_SHA256, | ||
| PrehashValue: ics23.HashOp_NO_HASH, | ||
| Length: ics23.LengthOp_NO_PREFIX, | ||
| Prefix: []byte{0}, | ||
| }, | ||
| InnerSpec: &ics23.InnerSpec{ | ||
| ChildOrder: []int32{0, 1}, | ||
| ChildSize: hashSize, | ||
| MinPrefixLength: 1, | ||
| MaxPrefixLength: 1, | ||
| EmptyChild: make([]byte, hashSize), | ||
| Hash: ics23.HashOp_SHA256, | ||
| }, | ||
| MaxDepth: 256, | ||
| PrehashKeyBeforeComparison: true, | ||
| } | ||
| innerPrefix = []byte{1} | ||
|
|
||
| // defaultValue is the default placeholder value in a SparseMerkleTree | ||
| defaultValue = make([]byte, hashSize) | ||
| ) | ||
|
|
||
| // VerifyMembership verifies the CommitmentProof provided, checking whether it produces the same | ||
| // root as the one given. If it does, the key-value pair is a member of the tree | ||
| func VerifyMembership(root ics23.CommitmentRoot, proof *ics23.CommitmentProof, key, value []byte) bool { | ||
| // verify the proof | ||
| return ics23.VerifyMembership(smtSpec, root, proof, key, value) | ||
| } | ||
|
|
||
| // VerifyNonMembership verifies the CommitmentProof provided, checking whether it produces the same | ||
| // root as the one given. If it does, the key-value pair is not a member of the tree as the proof's | ||
| // value is either the default nil value for the SMT or an unrelated value at the path | ||
| func VerifyNonMembership(root ics23.CommitmentRoot, proof *ics23.CommitmentProof, key []byte) bool { | ||
| // verify the proof | ||
| return ics23.VerifyNonMembership(smtSpec, root, proof, key) | ||
| } | ||
|
|
||
| // createMembershipProof generates a CommitmentProof object verifying the membership of a key-value pair | ||
| // in the SMT provided | ||
| func createMembershipProof(tree *smt.SMT, key, value []byte) (*ics23.CommitmentProof, error) { | ||
| proof, err := tree.Prove(key) | ||
| if err != nil { | ||
| return nil, coreTypes.ErrCreatingProof(err) | ||
| } | ||
| return convertSMPToExistenceProof(proof, key, value), nil | ||
| } | ||
|
|
||
| // createNonMembershipProof generates a CommitmentProof object verifying the membership of an unrealted key at the given key in the SMT provided | ||
| func createNonMembershipProof(tree *smt.SMT, key []byte) (*ics23.CommitmentProof, error) { | ||
| proof, err := tree.Prove(key) | ||
| if err != nil { | ||
| return nil, coreTypes.ErrCreatingProof(err) | ||
| } | ||
|
|
||
| return convertSMPToExclusionProof(proof, key), nil | ||
| } | ||
|
|
||
| // convertSMPToExistenceProof converts a SparseMerkleProof to an ics23 | ||
| // ExistenceProof to verify membership of an element | ||
| func convertSMPToExistenceProof(proof *smt.SparseMerkleProof, key, value []byte) *ics23.CommitmentProof { | ||
| path := sha256.Sum256(key) | ||
| steps := convertSideNodesToSteps(proof.SideNodes, path[:]) | ||
| return &ics23.CommitmentProof{ | ||
| Proof: &ics23.CommitmentProof_Exist{ | ||
| Exist: &ics23.ExistenceProof{ | ||
| Key: key, | ||
| Value: value, | ||
| Leaf: smtSpec.LeafSpec, | ||
| Path: steps, | ||
| }, | ||
| }, | ||
| } | ||
| } | ||
|
|
||
| // convertSMPToExclusionProof converts a SparseMerkleProof to an ics23 | ||
| // ExclusionProof to verify non-membership of an element | ||
| func convertSMPToExclusionProof(proof *smt.SparseMerkleProof, key []byte) *ics23.CommitmentProof { | ||
| path := sha256.Sum256(key) | ||
| steps := convertSideNodesToSteps(proof.SideNodes, path[:]) | ||
| leaf := &ics23.LeafOp{ | ||
| Hash: ics23.HashOp_SHA256, | ||
| // Do not re-hash already hashed fields from NonMembershipLeafData | ||
| PrehashKey: ics23.HashOp_NO_HASH, | ||
| PrehashValue: ics23.HashOp_NO_HASH, | ||
| Length: ics23.LengthOp_NO_PREFIX, | ||
| Prefix: []byte{0}, | ||
| } | ||
| actualPath := path[:] | ||
| actualValue := defaultValue | ||
| if proof.NonMembershipLeafData != nil { | ||
| actualPath = proof.NonMembershipLeafData[1 : 1+hashSize] // len(prefix): len(prefix) + hashSize | ||
| actualValue = proof.NonMembershipLeafData[1+hashSize:] | ||
| } | ||
| return &ics23.CommitmentProof{ | ||
| Proof: &ics23.CommitmentProof_Exclusion{ | ||
| Exclusion: &ics23.ExclusionProof{ | ||
| Key: key, | ||
| ActualPath: actualPath, | ||
| ActualValueHash: actualValue, | ||
| Leaf: leaf, | ||
| Path: steps, | ||
| }, | ||
| }, | ||
| } | ||
| } | ||
|
|
||
| // convertSideNodesToSteps converts the SideNodes field in the SparseMerkleProof | ||
| // into a list of InnerOps for the ics23 CommitmentProof | ||
| func convertSideNodesToSteps(sideNodes [][]byte, path []byte) []*ics23.InnerOp { | ||
| steps := make([]*ics23.InnerOp, 0, len(sideNodes)) | ||
| for i := 0; i < len(sideNodes); i++ { | ||
| var prefix, suffix []byte | ||
| prefix = append(prefix, innerPrefix...) | ||
| if isLeft(path, len(sideNodes)-1-i) { | ||
| // path is on the left so sidenode must be on the right | ||
| suffix = make([]byte, 0, len(sideNodes[i])) | ||
| suffix = append(suffix, sideNodes[i]...) | ||
| } else { | ||
| // path is on the right so sidenode must be on the left | ||
| prefix = append(prefix, sideNodes[i]...) | ||
| } | ||
| op := &ics23.InnerOp{ | ||
| Hash: ics23.HashOp_SHA256, | ||
| Prefix: prefix, | ||
| Suffix: suffix, | ||
| } | ||
| steps = append(steps, op) | ||
| } | ||
| return steps | ||
| } | ||
|
|
||
| // isLeft returns true is the i-th bit of path is a left child in the SMT | ||
| func isLeft(path []byte, i int) bool { | ||
| return smt.GetPathBit(path, i) == left | ||
| } |
Oops, something went wrong.