Broken command ImportReceiptsOp

[emhane]

Describe the feature

Importing receipts on commit 16577e0 throws error

-08-27T20:56:35.768013Z  INFO reth::cli: Importing receipt file chunk first_receipts_block=104896808 total_receipts_chunk=338256
[crates/net/downloaders/src/file_client.rs:410:13] self.chunk.is_empty() = true
2024-08-27T20:56:38.042297Z ERROR reth::cli: Receipts were partially imported total_decoded_receipts=105235062 total_imported_receipts=105235057 total_filtered_out_dup_txns=6
2024-08-27T20:56:38.042312Z ERROR reth::cli: Receipts inconsistent with transactions total_imported_receipts=105235057 total_imported_txns=105235056
2024-08-27T20:56:38.042320Z ERROR reth::cli: Height of receipts inconsistent with transactions highest_block_receipts=105235065 highest_block_transactions=105235063
2024-08-27T20:56:38.042325Z  INFO reth::cli: Receipt file imported total_imported_receipts=105235057 total_decoded_receipts=105235062 total_filtered_out_dup_txns=6

Additional context

No response

Tasks

Beta Give feedback

1. fix(op): empty receipts for block not counted by file client #10581
   A-cli A-op-reth C-bug +3

[emhane]

with #10581, the output of import-receipts-op now is

2024-08-28T08:58:09.818796Z  INFO reth::cli: Importing receipt file chunk first_receipts_block=104896808 total_receipts_chunk=338256
2024-08-28T08:58:12.012154Z ERROR reth::cli: Receipts inconsistent with transactions total_imported_receipts=105235057 total_imported_txns=105235056
2024-08-28T08:58:12.012165Z ERROR reth::cli: Height of receipts inconsistent with transactions highest_block_receipts=105235065 highest_block_transactions=105235063
2024-08-28T08:58:12.012170Z  INFO reth::cli: Receipt file imported total_imported_receipts=105235057 total_decoded_receipts=105235063 total_filtered_out_dup_txns=6

remaining bugs are

• inconsistency of one more receipt being imported than transactions
• receipts ending up at 2 blocks higher static files than transactions. txns are at correct height, since they match block height (see output from db stats cmd below)

$ ./target/maxperf/op-reth db --datadir ~/.local/share/reth/op-mainnet stats
2024-08-28T11:13:44.167318Z  INFO Initialized tracing, debug log directory: /home/ubuntu/.cache/reth/logs/optimism
2024-08-28T11:13:44.171014Z  INFO Opening storage db_path="/home/ubuntu/.local/share/reth/op-mainnet/db" sf_path="/home/ubuntu/.local/share/reth/op-mainnet/static_files"
2024-08-28T11:13:44.185470Z  INFO Skipping storage verification for OP mainnet, expected inconsistency in OVM chain
| Segment      | Block Range   | Transaction Range | Shape (columns x rows) | Size      |
|--------------|---------------|-------------------|------------------------|-----------|
| Headers      | 0..=105235063 | N/A               | 3 x 105235064          | 44 GiB    |
| Transactions | 0..=105235063 | 0..=105235055     | 1 x 105235056          | 22 GiB    |
| Receipts     | 0..=105235065 | 0..=105235056     | 1 x 105235057          | 34.2 GiB  |
| ------------ | ------------- | ----------------- | ---------------------- | --------  |
| Total        |               |                   |                        | 100.3 GiB |


| Table Name                 | # Entries | Branch Pages | Leaf Pages | Overflow Pages | Total Size |
|----------------------------|-----------|--------------|------------|----------------|------------|
| AccountChangeSets          | 0         | 0            | 0          | 0              | 0 B        |
| AccountsHistory            | 0         | 0            | 0          | 0              | 0 B        |
| AccountsTrie               | 0         | 0            | 0          | 0              | 0 B        |
| BlockBodyIndices           | 105235064 | 2782         | 622660     | 0              | 2.4 GiB    |
| BlockOmmers                | 0         | 0            | 0          | 0              | 0 B        |
| BlockRequests              | 0         | 0            | 0          | 0              | 0 B        |
| BlockWithdrawals           | 0         | 0            | 0          | 0              | 0 B        |
| Bytecodes                  | 0         | 0            | 0          | 0              | 0 B        |
| CanonicalHeaders           | 0         | 0            | 0          | 0              | 0 B        |
| ChainState                 | 0         | 0            | 0          | 0              | 0 B        |
| HashedAccounts             | 0         | 0            | 0          | 0              | 0 B        |
| HashedStorages             | 0         | 0            | 0          | 0              | 0 B        |
| HeaderNumbers              | 105235064 | 29111        | 1886591    | 0              | 7.3 GiB    |
| HeaderTerminalDifficulties | 0         | 0            | 0          | 0              | 0 B        |
| Headers                    | 0         | 0            | 0          | 0              | 0 B        |
| PlainAccountState          | 0         | 0            | 0          | 0              | 0 B        |
| PlainStorageState          | 0         | 0            | 0          | 0              | 0 B        |
| PruneCheckpoints           | 0         | 0            | 0          | 0              | 0 B        |
| Receipts                   | 0         | 0            | 0          | 0              | 0 B        |
| StageCheckpointProgresses  | 0         | 0            | 0          | 0              | 0 B        |
| StageCheckpoints           | 14        | 0            | 1          | 0              | 4 KiB      |
| StorageChangeSets          | 0         | 0            | 0          | 0              | 0 B        |
| StoragesHistory            | 0         | 0            | 0          | 0              | 0 B        |
| StoragesTrie               | 0         | 0            | 0          | 0              | 0 B        |
| TransactionBlocks          | 105235056 | 3014         | 674584     | 0              | 2.6 GiB    |
| TransactionHashNumbers     | 105235056 | 29161        | 1887487    | 0              | 7.3 GiB    |
| TransactionSenders         | 105235056 | 4393         | 983506     | 0              | 3.8 GiB    |
| Transactions               | 0         | 0            | 0          | 0              | 0 B        |
| VersionHistory             | 1         | 0            | 1          | 0              | 4 KiB      |
| -------------------------- | --------- | ------------ | ---------- | -------------- | ---------- |
| Tables                     |           |              |            |                | 23.4 GiB   |
| Freelist                   | 1422924   |              |            |                | 5.4 GiB    |