bidcollect docs update

flashbots · Jun 13, 2024 · ba3261f · ba3261f
1 parent d8a96df
commit ba3261f
Show file tree

Hide file tree

Showing 2 changed files with 44 additions and 16 deletions.
diff --git a/docs/2024-06_bidcollect.md b/docs/2024-06_bidcollect.md
@@ -1,20 +1,49 @@
-# Bid Collection
+# Relayscan Bid Archive
 
-Relayscan collects bids across [relays](../vars/relays.go) with these methods:
+Relayscan can run in a special bid-collection mode to build an archive of bids across [relays](../vars/relays.go).
 
-1. [getHeader polling](https://ethereum.github.io/builder-specs/#/Builder/getHeader)
-2. [Data API polling](https://flashbots.github.io/relay-specs/#/Data/getReceivedBids)
-3. [Ultrasound top-bid websocket stream](https://github.com/ultrasoundmoney/docs/blob/main/top-bid-websocket.md)
+Public bid archive: https://bidarchive.relayscan.io
 
-Output:
+---
 
-1. CSV file archive
+### Bids source types
 
-See also:
+0. [getHeader polling](https://ethereum.github.io/builder-specs/#/Builder/getHeader)
+1. [Data API polling](https://flashbots.github.io/relay-specs/#/Data/getReceivedBids)
+2. [Ultrasound top-bid websocket stream](https://github.com/ultrasoundmoney/docs/blob/main/top-bid-websocket.md)
+
+### Output
+
+The output is (daily) CSV files:
+1. All bids
+2. Top bids only
+
+
+### Collected fields
+
+| Field                  | Description                                            | Source Types |
+| ---------------------- | ------------------------------------------------------ | ------------ |
+| source_type            | 0: getHeader, 1: Data API, 2: Ultrasound stream        | all          |
+| received_at_ms         | Timestamp when the bid was first seen at the collector | all          |
+| timestamp_ms           | Timestamp when the bid was received by the relay       | 1 + 2        |
+| slot                   | Slot the bid was submitted for                         | all          |
+| slot_t_ms              | Slot timestamp                                         | all          |
+| value                  | Bid value in wei                                       | all          |
+| block_hash             | Block hash                                             | all          |
+| parent_hash            | Parent hash                                            | all          |
+| builder_pubkey         | Builder pubkey                                         | 1 + 2        |
+| block_number           | Block number                                           | all          |
+| block_fee_recipient    | Block fee recipient                                    | all          |
+| relay                  | Relay name                                             | all          |
+| proposer_pubkey        | Proposer pubkey                                        | 1 + 2        |
+| proposer_fee_recipient | Proposer fee recipient                                 | 1 + 2        |
+| optimistic_submission  | Optimistic submission flag                             | 0 + 1        |
+
+### See also
 
-- [Example output](https://gist.github.com/metachris/061c0443afb8b8d07eed477a848fa395)
-- [Pull request #37](https://github.com/flashbots/relayscan/pull/37)
 - Live data: https://bidarchive.relayscan.io
+- [Pull request #37](https://github.com/flashbots/relayscan/pull/37)
+- [Example output](https://gist.github.com/metachris/061c0443afb8b8d07eed477a848fa395)
 
 ---
 
@@ -27,18 +56,18 @@ Source types:
 
 Different data sources have different limitations:
 
-- `getHeader` polling:
+- `getHeader` polling ([code](/services/bidcollect/getheader-poller.go)):
   - The received header only has limited information, with these implications:
     - Optimistic is always `false`
     - No `builder_pubkey`
     - No bid timestamp (need to use receive timestamp)
     - getHeader bid timestamps are always when the response from polling at t=1s comes back (but not when the bid was received at a relay)
   - Some relays only allow a single `getHeader` request per slot, so we time it at `t=1s`
-- Data API polling:
+- Data API polling ([code](/services/bidcollect/data-api-poller.go):
     - Has all the necessary information
     - Due to rate limits, we only poll at specific times
-    - Polling at t-4, t-2, t-0.5, t+0.5, t+2 (see also [`services/bidcollect/data-api-poller.go`](services/bidcollect/data-api-poller.go#64-69))
-- Ultrasound websocket stream
+    - Polling at t-4, t-2, t-0.5, t+0.5, t+2 (see also [`/services/bidcollect/data-api-poller.go`](/services/bidcollect/data-api-poller.go#64-69))
+- Ultrasound websocket stream ([code](/services/bidcollect/ultrasound-stream.go):
   - doesn't expose optimistic, thus that field is always `false`
 
 ## Other notes

diff --git a/services/bidcollect/website/templates/base.html b/services/bidcollect/website/templates/base.html
@@ -113,8 +113,7 @@
         </a>
         <h1>{{ $title }}</h1>
         <p>
-            <!-- <a href="https://github.com/flashbots/relayscan">https://github.com/flashbots/relayscan</a> &middot; <a href="https://github.com/flashbots/relayscan/blob/bidstream/docs/2024-06_bidcollect.md">Bid Archive Docs</a> -->
-            <a href="https://github.com/flashbots/relayscan/blob/bidstream/docs/2024-06_bidcollect.md">docs</a> &middot; <a href="https://github.com/flashbots/relayscan">https://github.com/flashbots/relayscan</a>
+            <a href="https://github.com/flashbots/relayscan/blob/main/docs/2024-06_bidcollect.md">docs</a> &middot; <a href="https://github.com/flashbots/relayscan">https://github.com/flashbots/relayscan</a>
         </p>
         <p><i>Illuminate, Democratize, Distribute</i></p>
     </div>