Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add blob download endpoint (getBlobs) #286

Merged
merged 29 commits into from
May 17, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
5502da1
Update types to support EIP-4844 (#271)
jimmygchen Dec 25, 2022
85a715d
Add getBlobsSidecar endpoint
jimmygchen Jan 9, 2023
e702695
Merge remote-tracking branch 'upstream/master' into EIP4844
rolfyone Jan 18, 2023
fd02cd2
Introduce Deneb to remove EIP-4844 references
rolfyone Jan 18, 2023
ca3dd1b
Remove `_sidecar` from blobs endpoint path. Add endpoint to CHANGES.md
jimmygchen Jan 19, 2023
09ca1dd
Merge remote-tracking branch 'upstream/master' into eip_rename
rolfyone Feb 8, 2023
709ccd6
fix lint - couldnt find any evidence of capella transactions, so put …
rolfyone Feb 8, 2023
652b865
Merge branch 'eip_rename' into download-blob-api
jimmygchen Feb 14, 2023
f300afe
Merge branch 'master' into eip_rename
rolfyone Feb 15, 2023
b1bbe28
Merge branch 'master' into download-blob-api
jimmygchen Feb 20, 2023
30b2d05
Update endpoint to return `BlobSidecar` container from latest CL spec
jimmygchen Feb 24, 2023
1d8e599
Add recommendation to do blob retrieval by slot
jimmygchen Feb 24, 2023
39efcbe
Merge branch 'eip_rename' into download-blob-api
jimmygchen Feb 24, 2023
1ec6931
Merge branch 'master' into download-blob-api
jimmygchen Feb 27, 2023
fe595c3
Merge branch 'master' into download-blob-api
jimmygchen Mar 12, 2023
27ac4b8
Add blob indices query paraemter
jimmygchen Apr 4, 2023
7f5e792
Merge branch 'master' into download-blob-api
jimmygchen May 3, 2023
7ee3525
Remove old block_and_blobs_sidecar type
jimmygchen May 3, 2023
4ccaaa6
Address review comments.
jimmygchen May 5, 2023
4ded8d7
Merge branch 'master' into download-blob-api
rolfyone May 9, 2023
61b4663
Merge branch 'master' into download-blob-api
rolfyone May 10, 2023
27e4451
Merge branch 'master' into download-blob-api
jimmygchen May 11, 2023
e3dfda2
Apply review comment, change RanDAO to RANDAO
jimmygchen May 11, 2023
354ff9f
Add more details to getBlobs response
jimmygchen May 11, 2023
eb7bc1f
Merge branch 'download-blob-api' of github.com:jimmygchen/beacon-APIs…
jimmygchen May 11, 2023
df43224
Update spacing
jimmygchen May 11, 2023
775e93c
Merge branch 'master' into download-blob-api
jimmygchen May 17, 2023
a6d1c36
Update getBlobSidecars endpoint path
jimmygchen May 17, 2023
096aadc
Reuse BlobSidecars type in getBlobSidecars response
jimmygchen May 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions apis/beacon/blobs_sidecars/blobs_sidecar.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
get:
operationId: getBlobsSidecar
summary: Get blobs sidecar
description: |
Retrieves blobs sidecar for given block id.
Depending on `Accept` header it can be returned either as json or as bytes serialized by SSZ
tags:
- Beacon
parameters:
- name: block_id
in: path
required: true
$ref: '../../../beacon-node-oapi.yaml#/components/parameters/BlockId'
responses:
"200":
description: "Successful response"
content:
application/json:
schema:
title: GetBlobsSidecarResponse
type: object
properties:
data:
oneOf:
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/BlobsSidecar"
application/octet-stream:
schema:
description: "SSZ serialized block bytes. Use Accept header to choose this response type"
"400":
description: "The block ID supplied could not be parsed"
content:
application/json:
schema:
$ref: "../../../beacon-node-oapi.yaml#/components/schemas/ErrorMessage"
example:
code: 400
message: "Invalid block ID: current"
"404":
description: "Blob sidecar not found"
content:
application/json:
schema:
$ref: "../../../beacon-node-oapi.yaml#/components/schemas/ErrorMessage"
example:
code: 404
message: "Blob sidecar not found"
"500":
$ref: "../../../beacon-node-oapi.yaml#/components/responses/InternalError"
3 changes: 2 additions & 1 deletion apis/beacon/blocks/blinded_block.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ get:
properties:
version:
type: string
enum: [phase0, altair, bellatrix, capella]
enum: [phase0, altair, bellatrix, capella, eip4844]
example: "phase0"
execution_optimistic:
$ref: "../../../beacon-node-oapi.yaml#/components/schemas/ExecutionOptimistic"
Expand All @@ -37,6 +37,7 @@ get:
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Altair.SignedBeaconBlock"
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Bellatrix.SignedBlindedBeaconBlock"
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Capella.SignedBlindedBeaconBlock"
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/EIP4844.SignedBlindedBeaconBlock"
application/octet-stream:
schema:
description: "SSZ serialized block bytes. Use Accept header to choose this response type"
Expand Down
1 change: 1 addition & 0 deletions apis/beacon/blocks/blinded_blocks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ post:
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Altair.SignedBeaconBlock"
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Bellatrix.SignedBlindedBeaconBlock"
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Capella.SignedBlindedBeaconBlock"
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/EIP4844.SignedBlindedBeaconBlock"
application/octet-stream:
schema:
description: "SSZ serialized block bytes. Use content type header to indicate that SSZ data is contained in the request body."
Expand Down
3 changes: 2 additions & 1 deletion apis/beacon/blocks/block.v2.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ get:
properties:
version:
type: string
enum: [phase0, altair, bellatrix, capella]
enum: [phase0, altair, bellatrix, capella, eip4844]
example: "phase0"
execution_optimistic:
$ref: "../../../beacon-node-oapi.yaml#/components/schemas/ExecutionOptimistic"
Expand All @@ -37,6 +37,7 @@ get:
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Altair.SignedBeaconBlock"
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Bellatrix.SignedBeaconBlock"
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Capella.SignedBeaconBlock"
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/EIP4844.SignedBeaconBlock"
application/octet-stream:
schema:

Expand Down
1 change: 1 addition & 0 deletions apis/beacon/blocks/blocks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ post:
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Altair.SignedBeaconBlock"
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Bellatrix.SignedBeaconBlock"
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/Capella.SignedBeaconBlock"
- $ref: "../../../beacon-node-oapi.yaml#/components/schemas/EIP4844.SignedBeaconBlock"
application/octet-stream:
schema:
description: "SSZ serialized block bytes. Use content type header to indicate that SSZ data is contained in the request body."
Expand Down
3 changes: 2 additions & 1 deletion apis/debug/state.v2.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ get:
properties:
version:
type: string
enum: [ phase0, altair, bellatrix, capella ]
enum: [ phase0, altair, bellatrix, capella, eip4844 ]
example: "phase0"
execution_optimistic:
$ref: "../../beacon-node-oapi.yaml#/components/schemas/ExecutionOptimistic"
Expand All @@ -37,6 +37,7 @@ get:
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/Altair.BeaconState"
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/Bellatrix.BeaconState"
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/Capella.BeaconState"
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/EIP4844.BeaconState"
application/octet-stream:
schema:
description: "SSZ serialized state bytes. Use Accept header to choose this response type"
Expand Down
3 changes: 2 additions & 1 deletion apis/validator/blinded_block.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,14 +54,15 @@ get:
properties:
version:
type: string
enum: [ phase0, altair, bellatrix, capella ]
enum: [ phase0, altair, bellatrix, capella, eip4844 ]
example: "bellatrix"
data:
oneOf:
- $ref: '../../beacon-node-oapi.yaml#/components/schemas/BeaconBlock'
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/Altair.BeaconBlock"
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/Bellatrix.BlindedBeaconBlock"
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/Capella.BlindedBeaconBlock"
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/EIP4844.BlindedBeaconBlock"
application/octet-stream:
schema:
description: "SSZ serialized block bytes. Use Accept header to choose this response type, version string is sent in header `Eth-Consensus-Version`."
Expand Down
3 changes: 2 additions & 1 deletion apis/validator/block.v2.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -51,14 +51,15 @@ get:
properties:
version:
type: string
enum: [ phase0, altair, bellatrix, capella ]
enum: [ phase0, altair, bellatrix, capella, eip4844 ]
example: "phase0"
data:
oneOf:
- $ref: '../../beacon-node-oapi.yaml#/components/schemas/BeaconBlock'
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/Altair.BeaconBlock"
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/Bellatrix.BeaconBlock"
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/Capella.BeaconBlock"
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/EIP4844.BeaconBlock"
application/octet-stream:
schema:
description: "SSZ serialized block bytes. Use Accept header to choose this response type, version string is sent in header `Eth-Consensus-Version`."
Expand Down
16 changes: 15 additions & 1 deletion beacon-node-oapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,8 @@ paths:
$ref: "./apis/beacon/blocks/root.yaml"
/eth/v1/beacon/blocks/{block_id}/attestations:
$ref: "./apis/beacon/blocks/attestations.yaml"
/eth/v1/beacon/blobs_sidecars/{block_id}:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd actually be inclined to make this the same pattern as attestations, so the path would be more like
/eth/v1/beacon/blocks/{block_id}/blobs

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't that imply that the blobs are part of the block structure, though?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to /eth/v1/beacon/blobs/{block_id}

$ref: "./apis/beacon/blobs_sidecars/blobs_sidecar.yaml"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do both need to be plurals? is blob_sidecars accurate?

if the paths / files / object names can be consistent that'll make it easier to navigate

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm I think this is on an old commit as well, I just did a global search for blobs_ and not finding anything. I'll make sure the paths / file name are matching if we end up changing the paths 👍

/eth/v1/beacon/rewards/sync_committee/{block_id}:
$ref: "./apis/beacon/rewards/sync_committee.yaml"
/eth/v1/beacon/deposit_snapshot:
Expand Down Expand Up @@ -198,6 +200,8 @@ components:
$ref: './types/state.yaml#/BeaconState'
BeaconBlock:
$ref: './types/block.yaml#/BeaconBlock'
BlobsSidecar:
$ref: './types/blobs_sidecar.yaml#/BlobsSidecar'
DepositSnapshotResponse:
$ref: './types/api.yaml#/DepositSnapshotResponse'
SignedBeaconBlock:
Expand Down Expand Up @@ -311,7 +315,7 @@ components:
Bellatrix.SignedBlindedBeaconBlock:
$ref: './types/bellatrix/block.yaml#/Bellatrix/SignedBlindedBeaconBlock'
ConsensusVersion:
enum: [phase0, altair, bellatrix, capella]
enum: [phase0, altair, bellatrix, capella, eip4844]
rolfyone marked this conversation as resolved.
Show resolved Hide resolved
example: "phase0"
SignedValidatorRegistration:
$ref: './types/registration.yaml#/SignedValidatorRegistration'
Expand All @@ -325,6 +329,16 @@ components:
$ref: './types/capella/block.yaml#/Capella/BlindedBeaconBlock'
Capella.SignedBlindedBeaconBlock:
$ref: './types/capella/block.yaml#/Capella/SignedBlindedBeaconBlock'
EIP4844.BeaconState:
$ref: './types/eip4844/state.yaml#/EIP4844/BeaconState'
EIP4844.BeaconBlock:
$ref: './types/eip4844/block.yaml#/EIP4844/BeaconBlock'
EIP4844.SignedBeaconBlock:
$ref: './types/eip4844/block.yaml#/EIP4844/SignedBeaconBlock'
EIP4844.BlindedBeaconBlock:
$ref: './types/eip4844/block.yaml#/EIP4844/BlindedBeaconBlock'
EIP4844.SignedBlindedBeaconBlock:
$ref: './types/eip4844/block.yaml#/EIP4844/SignedBlindedBeaconBlock'
Node:
$ref: './types/fork_choice.yaml#/Node'
ExtraData:
Expand Down
1 change: 1 addition & 0 deletions types/bellatrix/execution_payload.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ Bellatrix:
allOf:
- $ref: '#/Bellatrix/ExecutionPayloadCommon'
- type: object
additionalProperties: false
properties:
transactions_root:
$ref: '../primitive.yaml#/Root'
28 changes: 28 additions & 0 deletions types/blobs_sidecar.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
BlobsSidecar:
type: object
description: "The `BlobsSidecar` object from the EIP-4844 CL spec."
properties:
beacon_block_root:
$ref: "./primitive.yaml#/Root"
beacon_block_slot:
$ref: "./primitive.yaml#/Uint64"
blobs:
type: array
maxItems: 4
minItems: 0
items:
$ref: "#/Blob"
kzg_aggregated_proof:
$ref: "#/KZGProof"

KZGProof:
type: string
format: hex
pattern: "^0x[a-fA-F0-9]{96}$"
description: "An aggregated KZG proof. Same check as `KZGCommitment`"

Blob:
type: string
format: hex
pattern: "^0x[a-fA-F0-9]{262144}$"
description: "A blob is `FIELD_ELEMENTS_PER_BLOB * size_of(BLSFieldElement) = 4096 * 32 = 131072` bytes (`DATA`) representing a SSZ-encoded Blob as defined in EIP-4844"
12 changes: 4 additions & 8 deletions types/capella/block.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ Capella:
$ref: '../voluntary_exit.yaml#/SignedVoluntaryExit'
sync_aggregate:
$ref: '../altair/sync_aggregate.yaml#/Altair/SyncAggregate'
bls_to_execution_changes:
type: array
items:
$ref: '../bls_to_execution_change.yaml#/SignedBLSToExecutionChange'

BeaconBlockBody:
allOf:
Expand All @@ -42,10 +46,6 @@ Capella:
properties:
execution_payload:
$ref: './execution_payload.yaml#/Capella/ExecutionPayload'
bls_to_execution_changes:
type: array
items:
$ref: '../bls_to_execution_change.yaml#/SignedBLSToExecutionChange'

BeaconBlock:
description: "The [`BeaconBlock`](https://github.com/ethereum/consensus-specs/blob/master/specs/Capella/beacon-chain.md#beaconblock) object from the CL Capella spec."
Expand Down Expand Up @@ -73,10 +73,6 @@ Capella:
properties:
execution_payload_header:
$ref: './execution_payload.yaml#/Capella/ExecutionPayloadHeader'
bls_to_execution_changes:
type: array
items:
$ref: '../bls_to_execution_change.yaml#/SignedBLSToExecutionChange'

BlindedBeaconBlock:
description: "A variant of the the [`BeaconBlock`](https://github.com/ethereum/consensus-specs/blob/master/specs/capella/beacon-chain.md#beaconblock) object from the CL Capella spec, which contains a `BlindedBeaconBlockBody` rather than a `BeaconBlockBody`."
Expand Down
1 change: 1 addition & 0 deletions types/capella/execution_payload.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ Capella:
allOf:
- $ref: '#/Capella/ExecutionPayloadCommon'
- type: object
additionalProperties: false
properties:
transactions_root:
$ref: '../primitive.yaml#/Root'
Expand Down
97 changes: 97 additions & 0 deletions types/eip4844/block.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
EIP4844:
BeaconBlockBodyCommon:
# An abstract object to collect the common fields between the BeaconBlockBody and the BlindedBeaconBlockBody objects
type: object
description: "The [`BeaconBlockBody`](https://github.com/ethereum/consensus-specs/blob/master/specs/eip4844/beacon-chain.md#beaconblockbody) object from the CL EIP-4844 spec."
rolfyone marked this conversation as resolved.
Show resolved Hide resolved
properties:
randao_reveal:
allOf:
- $ref: '../primitive.yaml#/Signature'
- description: "The RanDAO reveal value provided by the validator."
eth1_data:
$ref: '../eth1.yaml#/Eth1Data'
graffiti:
$ref: '../primitive.yaml#/Graffiti'
proposer_slashings:
type: array
items:
$ref: '../proposer_slashing.yaml#/ProposerSlashing'
attester_slashings:
type: array
items:
$ref: '../attester_slashing.yaml#/AttesterSlashing'
attestations:
type: array
items:
$ref: '../attestation.yaml#/Attestation'
deposits:
type: array
items:
$ref: '../deposit.yaml#/Deposit'
voluntary_exits:
type: array
items:
$ref: '../voluntary_exit.yaml#/SignedVoluntaryExit'
sync_aggregate:
$ref: '../altair/sync_aggregate.yaml#/Altair/SyncAggregate'
bls_to_execution_changes:
type: array
items:
$ref: '../bls_to_execution_change.yaml#/SignedBLSToExecutionChange'
blob_kzg_commitments:
type: array
items:
$ref: '../primitive.yaml#/KZGCommitment'

BeaconBlockBody:
allOf:
- $ref: '#/EIP4844/BeaconBlockBodyCommon'
- type: object
properties:
execution_payload:
$ref: './execution_payload.yaml#/EIP4844/ExecutionPayload'

BeaconBlock:
description: "The [`BeaconBlock`](https://github.com/ethereum/consensus-specs/blob/master/specs/EIP4844/beacon-chain.md#beaconblock) object from the CL EIP-4844 spec."
rolfyone marked this conversation as resolved.
Show resolved Hide resolved
allOf:
- $ref: '../altair/block.yaml#/Altair/BeaconBlockCommon'
- type: object
properties:
body:
$ref: '#/EIP4844/BeaconBlockBody'

SignedBeaconBlock:
type: object
description: "The [`SignedBeaconBlock`](https://github.com/ethereum/consensus-specs/blob/master/specs/eip4844/beacon-chain.md#signedbeaconblock) object envelope from the CL EIP-4844 spec."
rolfyone marked this conversation as resolved.
Show resolved Hide resolved
properties:
message:
$ref: "#/EIP4844/BeaconBlock"
signature:
$ref: "../primitive.yaml#/Signature"

BlindedBeaconBlockBody:
description: "A variant of the [`BeaconBlockBody`](https://github.com/ethereum/consensus-specs/blob/master/specs/eip4844/beacon-chain.md#beaconblockbody) object from the CL EIP-4844 spec, which contains a transactions root rather than a full transactions list."
rolfyone marked this conversation as resolved.
Show resolved Hide resolved
allOf:
- $ref: '#/EIP4844/BeaconBlockBodyCommon'
- type: object
properties:
execution_payload_header:
$ref: './execution_payload.yaml#/EIP4844/ExecutionPayloadHeader'

BlindedBeaconBlock:
description: "A variant of the the [`BeaconBlock`](https://github.com/ethereum/consensus-specs/blob/master/specs/eip4844/beacon-chain.md#beaconblock) object from the CL EIP-4844 spec, which contains a `BlindedBeaconBlockBody` rather than a `BeaconBlockBody`."
rolfyone marked this conversation as resolved.
Show resolved Hide resolved
allOf:
- $ref: '../altair/block.yaml#/Altair/BeaconBlockCommon'
- type: object
properties:
body:
$ref: '#/EIP4844/BlindedBeaconBlockBody'

SignedBlindedBeaconBlock:
type: object
description: "A variant of the the the [`SignedBeaconBlock`](https://github.com/ethereum/consensus-specs/blob/master/specs/eip4844/beacon-chain.md#signedbeaconblock) object envelope from the CL EIP-4844 spec, which contains a `BlindedBeaconBlock` rather than a `BeaconBlock`."
rolfyone marked this conversation as resolved.
Show resolved Hide resolved
properties:
message:
$ref: "#/EIP4844/BlindedBeaconBlock"
signature:
$ref: "../primitive.yaml#/Signature"
Loading