Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: improves documentation #96

Merged
merged 56 commits into from
Feb 14, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
57481a0
clarifies the unit of nidSize
staheri14 Jan 26, 2023
d842ff6
adds function description for foundInRange
staheri14 Jan 26, 2023
5495171
adds missing func descriptions or makes further clarifications
staheri14 Jan 26, 2023
f1f1a53
adds TODOs and missing function descriptions
staheri14 Jan 27, 2023
0d3d318
refactors gotLeafHashes to leafHashes
staheri14 Jan 27, 2023
be869c9
corrects the description of generateLeafData
staheri14 Jan 27, 2023
69dd211
adds todos and comments and missing description of methods/functions
staheri14 Feb 1, 2023
b02c28b
adds some optimization ideas
staheri14 Feb 1, 2023
da16c5d
adds todo for early hash calculation
staheri14 Feb 1, 2023
2c82fbb
adds two more questions about the namespace len size
staheri14 Feb 1, 2023
12cc868
modifies the Prove documentation
staheri14 Feb 1, 2023
66fd0a8
updates the comments
staheri14 Feb 1, 2023
256a705
adds godoc for calculateAbsenceIndex
staheri14 Feb 2, 2023
810de44
adds the function name to the description
staheri14 Feb 2, 2023
3dbcd83
adds further clarification about the absence proof
staheri14 Feb 2, 2023
184a0a7
deletes excess line
staheri14 Feb 2, 2023
bf514d0
removes a todo
staheri14 Feb 2, 2023
d659113
elaborates on the return values of foundInRange
staheri14 Feb 2, 2023
ef7c36f
removes a todo
staheri14 Feb 2, 2023
dc838f1
adds a suggestion for early return in VerifyNamespace
staheri14 Feb 2, 2023
28b38b3
removes some old questions
staheri14 Feb 2, 2023
393db51
explains the order of nodes in the proof
staheri14 Feb 2, 2023
33cf50f
removes a question
staheri14 Feb 2, 2023
ac87f1c
revises the type of the proof
staheri14 Feb 2, 2023
4466bf8
changes the capitalization of the end
staheri14 Feb 8, 2023
f4504d1
adds documentation for id
staheri14 Feb 8, 2023
8310cea
expands the proof module by incorporating additional questions and re…
staheri14 Feb 9, 2023
2d81750
adds documentation for the Proof and VerifyNamespace
staheri14 Feb 9, 2023
5d7a9a7
adds further documentation
staheri14 Feb 9, 2023
6b76848
explains the ignore max logic
staheri14 Feb 10, 2023
ae4ef1b
reformats the code and adds description for the IgnoreMaxNamespace flag
staheri14 Feb 10, 2023
dab15e2
cleans up todos in data.go
staheri14 Feb 10, 2023
52c87da
removes todos from nmt
staheri14 Feb 10, 2023
ebbadea
removes todos from proof
staheri14 Feb 10, 2023
f91831c
removes a todo
staheri14 Feb 10, 2023
3d295c9
revises the hasher description to better illustrate the Ignore max idea
staheri14 Feb 10, 2023
36c4ebd
fixes variable name inconsistency
staheri14 Feb 10, 2023
d161c16
adds periods
staheri14 Feb 10, 2023
0d497e7
shortens the description of IgnoreMaxNamespace
staheri14 Feb 10, 2023
b9c7736
adds description of leafRange
staheri14 Feb 10, 2023
c820e57
adds further description about NamespacedMerkleTree and its fields
staheri14 Feb 10, 2023
19d6620
minor update
staheri14 Feb 10, 2023
7dea8b9
expands on Prove and ProveRange
staheri14 Feb 10, 2023
de897e4
elaborates on ProveNamespace
staheri14 Feb 10, 2023
82f9a1e
expands on the godoc for validateandextract namespace, computeRoot, f…
staheri14 Feb 10, 2023
c60ee20
adds explanation of isMaxNamespaceIDIgnored to the Proof data type, a…
staheri14 Feb 10, 2023
0095ca6
fixes wrapped function arguments
staheri14 Feb 10, 2023
530a279
fix bugs introduced by wrapped lines
staheri14 Feb 10, 2023
9effd59
Merge tag 'v0.14.0' into nmt-improved-documentation
staheri14 Feb 10, 2023
8742307
adds newlines to the end of the files
staheri14 Feb 10, 2023
e68e750
minor change
staheri14 Feb 10, 2023
87437b1
reverts the minor change
staheri14 Feb 10, 2023
bb85479
fixes a bug
staheri14 Feb 10, 2023
f53c3af
fixes another bug
staheri14 Feb 10, 2023
26b1ea6
addresses comments
staheri14 Feb 14, 2023
04dd6f9
fixes CI failures
staheri14 Feb 14, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ import (
)

func main() {
// the tree will use this namespace size
// the tree will use this namespace size (number of bytes)
nidSize := 1
// the leaves that will be pushed
data := [][]byte{
Expand Down
96 changes: 57 additions & 39 deletions hasher.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,35 +15,33 @@ const (

var _ hash.Hash = (*Hasher)(nil)

// defaultHasher uses sha256 as a base-hasher, 8 bytes
// for the namespace IDs and ignores the maximum possible namespace.
// defaultHasher uses sha256 as a base-hasher, 8 bytes for the namespace IDs and
// ignores the maximum possible namespace.
var defaultHasher = NewNmtHasher(sha256.New(), DefaultNamespaceIDLen, true)

// Sha256Namespace8FlaggedLeaf uses sha256 as a base-hasher, 8 bytes
// for the namespace IDs and ignores the maximum possible namespace.
// Sha256Namespace8FlaggedLeaf uses sha256 as a base-hasher, 8 bytes for the
// namespace IDs and ignores the maximum possible namespace.
//
// Sha256Namespace8FlaggedLeaf(namespacedData) results in:
// ns(rawData) || ns(rawData) || sha256(LeafPrefix || rawData),
// where rawData is the leaf's data minus the namespace.ID prefix
// (namely namespacedData[NamespaceLen:]).
// Sha256Namespace8FlaggedLeaf(namespacedData) results in: ns(rawData) ||
// ns(rawData) || sha256(LeafPrefix || rawData), where rawData is the leaf's
// data minus the namespace.ID prefix (namely namespacedData[NamespaceLen:]).
//
// Note that different from other cryptographic hash functions, this here
// makes assumptions on the input:
// len(namespacedData) >= DefaultNamespaceIDLen has to hold,
// as the first DefaultNamespaceIDLen bytes are interpreted as the namespace ID).
// If the input does not fulfil this, we will panic.
// The output will be of length 2*DefaultNamespaceIDLen+sha256.Size = 48 bytes.
// Note that different from other cryptographic hash functions, this here makes
// assumptions on the input: len(namespacedData) >= DefaultNamespaceIDLen has to
// hold, as the first DefaultNamespaceIDLen bytes are interpreted as the
// namespace ID). If the input does not fulfil this, we will panic. The output
// will be of length 2*DefaultNamespaceIDLen+sha256.Size = 48 bytes.
func Sha256Namespace8FlaggedLeaf(namespacedData []byte) []byte {
return defaultHasher.HashLeaf(namespacedData)
}

// Sha256Namespace8FlaggedInner hashes inner nodes to:
// minNID || maxNID || sha256(NodePrefix || leftRight), where leftRight consists of the full
// left and right child node bytes, including their respective min and max namespace IDs.
// Hence, the input has to be of size:
// 48 = 32 + 8 + 8 = sha256.Size + 2*DefaultNamespaceIDLen bytes.
// If the input does not fulfil this, we will panic.
// The output will also be of length 2*DefaultNamespaceIDLen+sha256.Size = 48 bytes.
// Sha256Namespace8FlaggedInner hashes inner nodes to: minNID || maxNID ||
// sha256(NodePrefix || leftRight), where leftRight consists of the full left
// and right child node bytes, including their respective min and max namespace
// IDs. Hence, the input has to be of size: 48 = 32 + 8 + 8 = sha256.Size +
// 2*DefaultNamespaceIDLen bytes. If the input does not fulfil this, we will
// panic. The output will also be of length 2*DefaultNamespaceIDLen+sha256.Size
// = 48 bytes.
func Sha256Namespace8FlaggedInner(leftRight []byte) []byte {
const flagLen = DefaultNamespaceIDLen * 2
sha256Len := defaultHasher.baseHasher.Size()
Expand All @@ -57,6 +55,13 @@ type Hasher struct {
baseHasher hash.Hash
NamespaceLen namespace.IDSize

// The "ignoreMaxNs" flag influences the calculation of the namespace ID
// range for intermediate nodes in the tree i.e., HashNode method. This flag
// signals that, when determining the upper limit of the namespace ID range
// for a tree node, the maximum possible namespace ID (equivalent to
// "NamespaceLen" bytes of 0xFF, or 2^NamespaceLen-1) should be omitted if
// feasible. For a more in-depth understanding of this field, refer to the
// "HashNode".
ignoreMaxNs bool
precomputedMaxNs namespace.ID

Expand Down Expand Up @@ -88,8 +93,8 @@ func (n *Hasher) Size() int {

// Write writes the namespaced data to be hashed.
//
// Requires data of fixed size to match leaf or inner NMT nodes.
// Only a single write is allowed.
// Requires data of fixed size to match leaf or inner NMT nodes. Only a single
// write is allowed.
func (n *Hasher) Write(data []byte) (int, error) {
if n.data != nil {
panic("only a single Write is allowed")
Expand All @@ -109,8 +114,8 @@ func (n *Hasher) Write(data []byte) (int, error) {
return ln, nil
}

// Sum computes the hash.
// Does not append the given suffix, violating the interface.
// Sum computes the hash. Does not append the given suffix, violating the
// interface.
func (n *Hasher) Sum([]byte) []byte {
switch n.tp {
case LeafPrefix:
Expand Down Expand Up @@ -165,19 +170,30 @@ func (n *Hasher) HashLeaf(leaf []byte) []byte {
return h.Sum(res)
}

// HashNode hashes inner nodes to:
// minNID || maxNID || hash(NodePrefix || left || right), where left and right are the full
// left and right child node bytes, including their respective min and max namespace IDs:
// left = left.Min() || left.Max() || l.Hash().
func (n *Hasher) HashNode(l, r []byte) []byte {
// HashNode calculates a namespaced hash of a node using the supplied left and
// right children. The input values, "left" and "right," are namespaced hash
// values with the format "minNID || maxNID || hash." By default, the normal
// namespace hash calculation is followed, which is "res = min(left.minNID,
// right.minNID) || max(left.maxNID, right.maxNID) || H(NodePrefix, left,
// right)". "res" refers to the return value of the HashNode. However, if the
// "ignoreMaxNs" property of the Hasher is set to true, the calculation of the
// namespace ID range of the node slightly changes. In this case, when setting
// the upper range, the maximum possible namespace ID (i.e.,
// 2^NamespaceIDSize-1) should be ignored if possible. This is achieved by
// taking the maximum value among the namespace IDs available in the range of
// its left and right children (i.e., max(left.minNID, left.maxNID ,
// right.minNID, right.maxNID)), which is not equal to the maximum possible
// namespace ID value. If such a namespace ID does not exist, the maximum NID is
// calculated as normal, i.e., "res.maxNID = max(left.maxNID , right.maxNID).
func (n *Hasher) HashNode(left, right []byte) []byte {
h := n.baseHasher
h.Reset()

// the actual hash result of the children got extended (or flagged) by their
// children's minNs || maxNs; hence the flagLen = 2 * NamespaceLen:
flagLen := 2 * n.NamespaceLen
leftMinNs, leftMaxNs := l[:n.NamespaceLen], l[n.NamespaceLen:flagLen]
rightMinNs, rightMaxNs := r[:n.NamespaceLen], r[n.NamespaceLen:flagLen]
leftMinNs, leftMaxNs := left[:n.NamespaceLen], left[n.NamespaceLen:flagLen]
rightMinNs, rightMaxNs := right[:n.NamespaceLen], right[n.NamespaceLen:flagLen]

minNs := min(leftMinNs, rightMinNs)
var maxNs []byte
Expand All @@ -189,15 +205,17 @@ func (n *Hasher) HashNode(l, r []byte) []byte {
maxNs = max(leftMaxNs, rightMaxNs)
}

res := append(append(make([]byte, 0), minNs...), maxNs...)
res := make([]byte, 0)
res = append(res, minNs...)
res = append(res, maxNs...)

// Note this seems a little faster than calling several Write()s on the
// underlying Hash function (see: https://github.com/google/trillian/pull/1503):
data := append(append(append(
make([]byte, 0, 1+len(l)+len(r)),
NodePrefix),
l...),
r...)
// underlying Hash function (see:
// https://github.com/google/trillian/pull/1503):
data := make([]byte, 0, 1+len(left)+len(right))
data = append(data, NodePrefix)
data = append(data, left...)
data = append(data, right...)
//nolint:errcheck
h.Write(data)
return h.Sum(res)
Expand Down
17 changes: 8 additions & 9 deletions namespace/data.go
Original file line number Diff line number Diff line change
@@ -1,16 +1,15 @@
package namespace

// PrefixedData simply represents a slice of bytes which consists of
// a namespace.ID and raw data.
// The user has to guarantee that the bytes are valid namespace prefixed data.
// Go's type system does not allow enforcing the structure we want:
// [namespaceID, rawData ...], especially as this type does not expect any
// particular size for the namespace.
// PrefixedData simply represents a slice of bytes which consists of a
// namespace.ID and raw data. The user has to guarantee that the bytes are valid
// namespace prefixed data. Go's type system does not allow enforcing the
// structure we want: [namespaceID, rawData ...], especially as this type does
// not expect any particular size for the namespace.
type PrefixedData []byte

// PrefixedData8 like PrefixedData is just a slice of bytes.
// It assumes that the slice it represents is at least 8 bytes.
// This assumption is not enforced by the type system though.
// PrefixedData8 like PrefixedData is just a slice of bytes. It assumes that the
// slice it represents is at least 8 bytes. This assumption is not enforced by
// the type system though.
type PrefixedData8 []byte

func (d PrefixedData8) NamespaceID() ID {
Expand Down
5 changes: 5 additions & 0 deletions namespace/id.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,27 @@ import "bytes"

type ID []byte

// Less returns true if nid < other, otherwise, false.
func (nid ID) Less(other ID) bool {
return bytes.Compare(nid, other) < 0
}

// Equal returns true if nid == other, otherwise, false.
func (nid ID) Equal(other ID) bool {
return bytes.Equal(nid, other)
}

// LessOrEqual returns true if nid <= other, otherwise, false.
func (nid ID) LessOrEqual(other ID) bool {
return bytes.Compare(nid, other) <= 0
}

// Size returns the byte size of the nid.
func (nid ID) Size() IDSize {
return IDSize(len(nid))
}

// String stringifies the nid.
func (nid ID) String() string {
return string(nid)
}
Loading