Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] EIP-7594: PeerDAS protocol #3574

Merged
merged 29 commits into from
Apr 4, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
d6a37ec
Copied from Danny's ethresearch post
hwwhww Nov 9, 2023
93dddd1
wip
hwwhww Nov 14, 2023
504b4f9
Migrating to latest crypto APIs
hwwhww Jan 11, 2024
696d443
Merge branch 'dev' into peer-das
hwwhww Jan 19, 2024
2cc7c87
Fix conflict
hwwhww Jan 19, 2024
665e6fa
Add `RowIndex`, `ColumnIndex` custom types in crypto doc
hwwhww Jan 19, 2024
9553d54
fix typo
hwwhww Jan 19, 2024
a72ece8
Apply suggestions from code review
hwwhww Jan 19, 2024
65be5b0
Make `CUSTODY_REQUIREMENT` unit be subnets; move some depended helper…
hwwhww Jan 19, 2024
55db861
Apply suggestions from code review
hwwhww Jan 20, 2024
4477cc6
Fix column computation
hwwhww Jan 20, 2024
56e6a98
`verify_data_column_sidecar_kzg_proof` -> `verify_data_column_sidecar…
hwwhww Jan 20, 2024
edeef07
toc
hwwhww Jan 28, 2024
b2a4657
Merge branch 'peer-das-req-subnet-count' into peer-das
hwwhww Jan 29, 2024
7aab577
Merge branch 'dev' into peer-das
hwwhww Jan 29, 2024
170dae5
Apply suggestions from code review
hwwhww Jan 29, 2024
547460c
Apply PR feedback
hwwhww Jan 30, 2024
d23452d
Deprecate `blob_sidecar_{subnet_id}`
hwwhww Jan 31, 2024
87e9702
Fix `DataColumnSidecarsByRoot`
hwwhww Jan 31, 2024
428c166
Apply suggestions from code review
hwwhww Feb 1, 2024
c47d5f3
Add `recover_matrix` and remove unused `FlatExtendedMatrix` type
hwwhww Feb 1, 2024
91dbbb3
Implement `compute_extended_matrix`
hwwhww Feb 1, 2024
e7c0d5f
Update specs/_features/eip7594/das-core.md
hwwhww Feb 2, 2024
8150f76
Apply @cskiraly's suggestion
hwwhww Feb 20, 2024
bb33f90
Change List length of `DataColumn` from `MAX_BLOBS_PER_BLOCK` to `MAX…
hwwhww Feb 20, 2024
1acb1ff
minor arrange
hwwhww Feb 20, 2024
cebf78a
Apply PR feedback
hwwhww Feb 27, 2024
8728561
Merge branch 'dev' into peer-das
hwwhww Apr 4, 2024
5535e6a
fix conflict
hwwhww Apr 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions specs/_features/eip7594/das-core.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@ We define the following Python custom types for type hinting and readability:
| `DataColumn` | `List[Cell, MAX_BLOBS_PER_BLOCK]` | The data of each column in EIP7594 |
hwwhww marked this conversation as resolved.
Show resolved Hide resolved
| `ExtendedMatrix` | `List[Cell, MAX_BLOBS_PER_BLOCK * NUMBER_OF_COLUMNS]` | The full data with blobs and one-dimensional erasure coding extension |
hwwhww marked this conversation as resolved.
Show resolved Hide resolved
| `FlatExtendedMatrix` | `List[BLSFieldElement, MAX_BLOBS_PER_BLOCK * FIELD_ELEMENTS_PER_BLOB * NUMBER_OF_COLUMNS]` | The flattened format of `ExtendedMatrix` |
hwwhww marked this conversation as resolved.
Show resolved Hide resolved
| `LineIndex` | `uint64` | The index of the rows or columns in `FlatExtendedMatrix` matrix |

## Configuration

Expand All @@ -68,11 +67,10 @@ We define the following Python custom types for type hinting and readability:
#### `get_custody_lines`

```python
def get_custody_lines(node_id: NodeID, custody_size: uint64) -> Sequence[LineIndex]:
def get_custody_lines(node_id: NodeID, custody_size: uint64) -> Sequence[ColumnIndex]:
hwwhww marked this conversation as resolved.
Show resolved Hide resolved
assert custody_size <= NUMBER_OF_COLUMNS
all_items = list(range(NUMBER_OF_COLUMNS))
line_index = node_id % NUMBER_OF_COLUMNS
return [LineIndex(all_items[(line_index + i) % len(all_items)]) for i in range(custody_size)]
column_index = node_id % NUMBER_OF_COLUMNS
return [ColumnIndex((column_index + i) % NUMBER_OF_COLUMNS) for i in range(custody_size)]
```

#### `compute_extended_data`
Expand Down
20 changes: 10 additions & 10 deletions specs/_features/eip7594/p2p-interface.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
- [Configuration](#configuration)
- [Containers](#containers)
- [`DataColumnSidecar`](#datacolumnsidecar)
- [`DataColumnIdentifier`](#datacolumnidentifier)
- [`DataColumnIndexentifier`](#dataColumnIndexentifier)
- [Helpers](#helpers)
- [`verify_data_column_sidecar_kzg_proof`](#verify_data_column_sidecar_kzg_proof)
- [`verify_data_column_sidecar_inclusion_proof`](#verify_data_column_sidecar_inclusion_proof)
Expand Down Expand Up @@ -49,20 +49,20 @@

```python
class DataColumnSidecar(Container):
index: LineIndex # Index of column in extended matrix
index: ColumnIndex # Index of column in extended matrix
column: DataColumn
kzg_commitments: List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK]
kzg_proofs: List[KZGProof, MAX_BLOB_COMMITMENTS_PER_BLOCK]
signed_block_header: SignedBeaconBlockHeader
kzg_commitments_inclusion_proof: Vector[Bytes32, KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH]
```

#### `DataColumnIdentifier`
#### `DataColumnIndexentifier`

```python
class DataColumnIdentifier(Container):
class DataColumnIndexentifier(Container):
hwwhww marked this conversation as resolved.
Show resolved Hide resolved
block_root: Root
index: LineIndex
index: ColumnIndex
```

### Helpers
Expand All @@ -74,14 +74,14 @@ def verify_data_column_sidecar_kzg_proof(sidecar: DataColumnSidecar) -> bool:
"""
Verify if the proofs are correct
"""
row_ids = [LineIndex(i) for i in range(len(sidecar.column))]
row_ids = [RowIndex(i) for i in range(len(sidecar.column))]
assert len(sidecar.column) == len(sidecar.kzg_commitments) == len(sidecar.kzg_proofs)

# KZG batch verifies that the cells match the corresponding commitments and proofs
return verify_cell_proof_batch(
row_commitments=sidecar.kzg_commitments,
row_ids=row_ids, # all rows
column_ids=[sidecar.index],
row_indices=row_ids, # all rows
column_indices=[sidecar.index],
datas=sidecar.column,
proofs=sidecar.kzg_proofs,
)
Expand All @@ -107,7 +107,7 @@ def verify_data_column_sidecar_inclusion_proof(sidecar: DataColumnSidecar) -> bo
##### `compute_subnet_for_data_column_sidecar`

```python
def compute_subnet_for_data_column_sidecar(column_index: LineIndex) -> SubnetID:
def compute_subnet_for_data_column_sidecar(column_index: ColumnIndex) -> SubnetID:
hwwhww marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @ppopth that the name here is misleading. I would suspect it to take in a sidecar not a column_index value. I might argue for a retro-patch to the naming convenction in deneb

return SubnetID(column_index % DATA_COLUMN_SIDECAR_SUBNET_COUNT)
```

Expand Down Expand Up @@ -167,7 +167,7 @@ Request Content:

```
(
DataColumnIdentifier
DataColumnIndexentifier
)
```

Expand Down
12 changes: 7 additions & 5 deletions specs/_features/eip7594/polynomial-commitments-sampling.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,8 @@ Public functions MUST accept raw bytes as input and perform the required cryptog
| `PolynomialCoeff` | `List[BLSFieldElement, 2 * FIELD_ELEMENTS_PER_BLOB]` | A polynomial in coefficient form |
| `Cell` | `Vector[BLSFieldElement, FIELD_ELEMENTS_PER_CELL]` | The unit of blob data that can come with their own KZG proofs |
| `CellID` | `uint64` | Cell identifier |
| `RowIndex` | `uint64` | Row identifier |
| `ColumnIndex` | `uint64` | Column identifier |

## Constants

Expand Down Expand Up @@ -415,8 +417,8 @@ def verify_cell_proof(commitment: KZGCommitment,

```python
def verify_cell_proof_batch(row_commitments: Sequence[KZGCommitment],
row_ids: Sequence[int],
column_ids: Sequence[int],
row_indices: Sequence[RowIndex],
column_indices: Sequence[ColumnIndex],
cells: Sequence[Cell],
proofs: Sequence[KZGProof]) -> bool:
"""
Expand All @@ -432,11 +434,11 @@ def verify_cell_proof_batch(row_commitments: Sequence[KZGCommitment],
"""

# Get commitments via row IDs
commitments = [row_commitments[row_id] for row_id in row_ids]
commitments = [row_commitments[row_index] for row_index in row_indices]

return all(
verify_kzg_proof_multi_impl(commitment, coset_for_cell(column_id), cell, proof)
for commitment, column_id, cell, proof in zip(commitments, column_ids, cells, proofs)
verify_kzg_proof_multi_impl(commitment, coset_for_cell(column_index), cell, proof)
for commitment, column_index, cell, proof in zip(commitments, column_indices, cells, proofs)
)
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,8 @@ def test_verify_cell_proof_batch(spec):

assert spec.verify_cell_proof_batch(
row_commitments=[commitment],
row_ids=[0],
column_ids=[0, 1],
row_indices=[0],
column_indices=[0, 1],
cells=cells[0:1],
proofs=proofs,
)
Expand Down
Loading