fjord: Add FastLZ L1-Cost function specs

specs/fjord/derivation.md

@@ -13,6 +13,10 @@
     - [Rationale](#rationale-1)
     - [Security Considerations](#security-considerations-1)
 - [Brotli Channel Compression](#brotli-channel-compression)
+- [Network upgrade automation transactions](#network-upgrade-automation-transactions)
+  - [GasPriceOracle Deployment](#gaspriceoracle-deployment)
+  - [GasPriceOracle Proxy Update](#gaspriceoracle-proxy-update)
+  - [GasPriceOracle Enable Fjord](#gaspriceoracle-enable-fjord)
 
 <!-- END doctoc generated TOC please keep comment here to allow auto update -->
 
@@ -34,6 +38,9 @@ Changes to derivation are applied when it is considering data from a L1 Block wh
 is greater than or equal to the activation timestamp.
 The change of the `max_sequencer_drift` parameter activates with the L1 origin block timestamp.
 
+If Fjord is not activated at genesis, it must be activated at least one block after the Ecotone
+activation block. This ensures that the network upgrade transactions don't conflict.
+
 ## Constant Maximum Sequencer Drift
 
 With Fjord, the `max_sequencer_drift` parameter becomes a constant of value `1800` _seconds_,
@@ -137,3 +144,118 @@ Brotli compression algorithm (as specified in [RFC-7932][rfc7932]) with no custo
 
 [rfc7932]: https://datatracker.ietf.org/doc/html/rfc7932
 [rfc1950]: https://www.rfc-editor.org/rfc/rfc1950.html
+
+# Network upgrade automation transactions
+
+The Fjord hardfork activation block contains the following transactions, in this order:
+
+- L1 Attributes Transaction
+- User deposits from L1
+- Network Upgrade Transactions
+  - GasPriceOracle deployment
+  - Update GasPriceOracle Proxy ERC-1967 Implementation Slot
+  - GasPriceOracle Enable Fjord
+
+To not modify or interrupt the system behavior around gas computation, this block will not include any sequenced
+transactions by setting `noTxPool: true`.
+
+## GasPriceOracle Deployment
+
+The `GasPriceOracle` contract is upgraded to support the new Fjord L1 data fee computation. Post fork this contract
+will use FastLZ to compute the L1 data fee.
+
+To perform this upgrade, a deposit transaction is derived with the following attributes:
+
+- `from`: `0x4210000000000000000000000000000000000002`
+- `to`: `null`,
+- `mint`: `0`
+- `value`: `0`
+- `gasLimit`: `1,450,000`
+- `data`: `0x60806040523...` ([full bytecode](../static/bytecode/fjord-gas-price-oracle-deployment.txt))
+- `sourceHash`: `0x86122c533fdcb89b16d8713174625e44578a89751d96c098ec19ab40a51a8ea3`
+  computed with the "Upgrade-deposited" type, with `intent = "Fjord: Gas Price Oracle Deployment"
+
+This results in the Fjord GasPriceOracle contract being deployed to `0xa919894851548179A0750865e7974DA599C0Fac7`,
+to verify:
+
+```bash
+cast compute-address --nonce=0 0x4210000000000000000000000000000000000002
+Computed Address: 0xa919894851548179A0750865e7974DA599C0Fac7
+```
+
+Verify `sourceHash`:
+
+```bash
+cast keccak $(cast concat-hex 0x0000000000000000000000000000000000000000000000000000000000000002 $(cast keccak "Fjord: Gas Price Oracle Deployment"))
+# 0x86122c533fdcb89b16d8713174625e44578a89751d96c098ec19ab40a51a8ea3
+```
+
+Verify `data`:
+
+```bash
+git checkout 52abfb507342191ae1f960b443ae8aec7598755c
+pnpm clean && pnpm install && pnpm build
+jq -r ".bytecode.object" packages/contracts-bedrock/forge-artifacts/GasPriceOracle.sol/GasPriceOracle.json
+```
+
+This transaction MUST deploy a contract with the following code hash
+`0xa88fa50a2745b15e6794247614b5298483070661adacb8d32d716434ed24c6b2`.
+
+## GasPriceOracle Proxy Update
+
+This transaction updates the GasPriceOracle Proxy ERC-1967 implementation slot to point to the new GasPriceOracle
+deployment.
+
+A deposit transaction is derived with the following attributes:
+
+- `from`: `0x0000000000000000000000000000000000000000`
+- `to`: `0x420000000000000000000000000000000000000F` (Gas Price Oracle Proxy)
+- `mint`: `0`
+- `value`: `0`
+- `gasLimit`: `50,000`
+- `data`: `0x3659cfe6000000000000000000000000a919894851548179a0750865e7974da599c0fac7`
+- `sourceHash`: `0x1e6bb0c28bfab3dc9b36ffb0f721f00d6937f33577606325692db0965a7d58c6`
+  computed with the "Upgrade-deposited" type, with `intent = "Fjord: Gas Price Oracle Proxy Update"`
+
+Verify data:
+
+```bash
+cast concat-hex $(cast sig "upgradeTo(address)") $(cast abi-encode "upgradeTo(address)" 0xa919894851548179A0750865e7974DA599C0Fac7)
+# 0x3659cfe6000000000000000000000000a919894851548179a0750865e7974da599c0fac7
+```
+
+Verify `sourceHash`:
+
+```bash
+cast keccak $(cast concat-hex 0x0000000000000000000000000000000000000000000000000000000000000002 $(cast keccak "Fjord: Gas Price Oracle Proxy Update"))
+# 0x1e6bb0c28bfab3dc9b36ffb0f721f00d6937f33577606325692db0965a7d58c6
+```
+
+## GasPriceOracle Enable Fjord
+
+This transaction informs the GasPriceOracle to start using the Fjord gas calculation formula.
+
+A deposit transaction is derived with the following attributes:
+
+- `from`: `0xDeaDDEaDDeAdDeAdDEAdDEaddeAddEAdDEAd0001` (Depositer Account)
+- `to`: `0x420000000000000000000000000000000000000F` (Gas Price Oracle Proxy)
+- `mint`: `0`
+- `value`: `0`
+- `gasLimit`: `90,000`
+- `data`: `0x8e98b106`
+- `sourceHash`: `0xbac7bb0d5961cad209a345408b0280a0d4686b1b20665e1b0f9cdafd73b19b6b`,
+  computed with the "Upgrade-deposited" type, with `intent = "Fjord: Gas Price Oracle Set Fjord"
+
+Verify data:
+
+```bash
+cast sig "setFjord()"
+0x8e98b106
+```
+
+Verify `sourceHash`:
+
+```bash
+cast keccak $(cast concat-hex 0x0000000000000000000000000000000000000000000000000000000000000002 $(cast keccak "Fjord: Gas Price Oracle Set Fjord"))
+# 0xbac7bb0d5961cad209a345408b0280a0d4686b1b20665e1b0f9cdafd73b19b6b
+```

specs/fjord/exec-engine.md

@@ -0,0 +1,80 @@
+# L2 Execution Engine
+
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+**Table of Contents**
+
+- [Fees](#fees)
+  - [L1-Cost fees (L1 Fee Vault)](#l1-cost-fees-l1-fee-vault)
+    - [Fjord L1-Cost fee changes (FastLZ estimator)](#fjord-l1-cost-fee-changes-fastlz-estimator)
+      - [FastLZ Implementation](#fastlz-implementation)
+      - [L1-Cost linear regression details](#l1-cost-linear-regression-details)
+  - [L1 Gas Usage Estimation](#l1-gas-usage-estimation)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
+## Fees
+
+### L1-Cost fees (L1 Fee Vault)
+
+#### Fjord L1-Cost fee changes (FastLZ estimator)
+
+Fjord updates the L1 cost calculation function to use a FastLZ-based compression estimator.
+The L1 cost is computed as:
+
+```pseudocode
+l1FeeScaled = baseFeeScalar*l1BaseFee*16 + blobFeeScalar*l1BlobBaseFee
+estimatedSize = max(minTransactionSize, intercept + fastlzCoef*fastlzSize)
+l1Cost = estimatedSize * l1FeeScaled / 1e12
+```
+
+The final `l1Cost` computation is an unlimited precision unsigned integer computation, with the result in Wei and
+having `uint256` range. The values in this computation, are as follows:
+
+| Input arg            | Type      | Description                                                       | Value                    |
+|----------------------|-----------|-------------------------------------------------------------------|--------------------------|
+| `l1BaseFee`          | `uint256` | L1 base fee of the latest L1 origin registered in the L2 chain    | varies, L1 fee           |
+| `l1BlobBaseFee`      | `uint256` | Blob gas price of the latest L1 origin registered in the L2 chain | varies, L1 fee           |
+| `fastlzSize`         | `uint256` | Size of the FastLZ-compressed RLP-encoded signed tx               | varies, per transaction  |
+| `baseFeeScalar`      | `uint32`  | L1 base fee scalar, scaled by `1e6`                               | varies, L2 configuration |
+| `blobFeeScalar`      | `uint32`  | L1 blob fee scalar, scaled by `1e6`                               | varies, L2 configuration |
+| `intercept`          | `int32`   | Intercept constant, scaled by `1e6` (can be negative)             | -42_585_600              |
+| `fastlzCoef`         | `uint32`  | FastLZ coefficient, scaled by `1e6`                               | 836_500                  |
+| `minTransactionSize` | `uint32`  | A lower bound on transaction size, in bytes                       | 100                      |
+
+Previously, `baseFeeScalar` and `blobFeeScalar` were used to encode the compression ratio, due to the inaccuracy of
+the L1 cost function. However, the new cost function takes into account the compression ratio, so these scalars should
+be adjusted to account for any previous compression ratio they encoded.
+
+##### FastLZ Implementation
+
+All compression algorithms must be implemented equivalently to the `fastlz_compress` function in `fastlz.c` at the
+following [commit](https://github.com/ariya/FastLZ/blob/344eb4025f9ae866ebf7a2ec48850f7113a97a42/fastlz.c#L482-L506).
+
+##### L1-Cost linear regression details
+
+The `intercept` and `fastlzCoef` constants are calculated by linear regression using a dataset
+of previous L2 transactions. The dataset is generated by iterating over all transactions in a given time range, and
+performing the following actions. For each transaction:
+
+1. Compress the payload using FastLZ. Record the size of the compressed payload as `fastlzSize`.
+2. Emulate the change in batch size adding the transaction to a batch, compressed with Brotli 10. Record the change in
+   batch size as `bestEstimateSize`.
+
+Once this dataset is generated, a linear regression can be calculated using the `bestEstimateSize` as
+the dependent variable and `fastlzSize` as the independent variable.
+
+We generated a dataset from two weeks of post-Ecotone transactions on Optimism Mainnet, as we found that was
+the most representative of performance across multiple chains and time periods. More details on the linear regression
+and datasets used can be found in this [repository](https://github.com/roberto-bayardo/compression-analysis/tree/main).
+
+### L1 Gas Usage Estimation
+
+The `L1GasUsed` property on the transaction receipt is updated to take into account the improvement in
+[compression estimation](./exec-engine.md#fees) accuracy. The value will be calculated by
+multiplying the `estimatedSize` of the transaction from the above L1 cost formula by 16. The value of 16 assumes most
+of the bytes in the compressed data are non-zero.
+
+The `L1GasUsed` property will be deprecated due to it not accurately calculating the L1 gas used
+by a transaction. Users can continue to use the `L1Fee` field to retrieve the L1 fee for a given transaction. This field
+will be removed in a future network upgrade.

specs/fjord/overview.md

@@ -14,6 +14,9 @@ This document is not finalized and should be considered experimental.
 ## Execution Layer
 
 - [RIP-7212: Precompile for secp256r1 Curve Support](../protocol/precompiles.md#P256VERIFY)
+- [FastLZ compression for L1 data fee calculation](./exec-engine.md#fees)
+- [Deprecate the `getL1GasUsed` method on the `GasPriceOracle` contract](./predeploys.md#l1-gas-usage-estimation)
+- [Deprecate the `L1GasUsed` field on the transaction receipt](./exec-engine.md#l1-gas-usage-estimation)
 
 ## Consensus Layer
 

specs/fjord/predeploys.md

@@ -0,0 +1,77 @@
+# Predeploys
+
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+**Table of Contents**
+
+- [GasPriceOracle](#gaspriceoracle)
+  - [L1 Gas Usage Estimation](#l1-gas-usage-estimation)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
+## GasPriceOracle
+
+Following the Fjord upgrade, three additional values used for L1 fee computation are:
+
+- costIntercept
+- costFastlzCoef
+- minTransactionSize
+
+These values are hard-coded constants in the `GasPriceOracle` contract. The
+calculation follows the same formula outlined in the
+[Fjord L1-Cost fee changes (FastLZ estimator)](./exec-engine.md#fjord-l1-cost-fee-changes-fastlz-estimator)
+section.
+
+A new method is introduced: `getL1FeeUpperBound(uint256)`. This method returns an upper bound for the L1 fee
+for a given transaction size. It is provided for callers who wish to estimate L1 transaction costs in the
+write path, and is much more gas efficient than `getL1Fee`.
+
+The upper limit overhead is assumed to be `original/255+16`, borrowed from LZ4. According to historical data, this
+approach can encompass more than 99.99% of transactions.
+
+This is implemented as follows:
+
+```solidity
+function getL1FeeUpperBound(uint256 unsignedTxSize) external view returns (uint256) {
+    // Add 68 to account for unsigned tx
+    uint256 txSize = unsignedTxSize + 68;
+    // txSize / 255 + 16 is the pratical fastlz upper-bound covers 99.99% txs.
+    uint256 flzUpperBound = txSize + txSize / 255 + 16;
+
+    int256 estimatedSize = costIntercept + costFastlzCoef * flzUpperBound;
+    if (estimatedSize < minTransactionSize) {
+        estimatedSize = minTransactionSize;
+    }
+
+    uint256 l1FeeScaled = baseFeeScalar() * l1BaseFee() * 16 + blobBaseFeeScalar() * blobBaseFee();
+    return uint256(estimatedSize) * feeScaled / (10 ** (DECIMALS * 2));
+}
+```
+
+### L1 Gas Usage Estimation
+
+The `getL1GasUsed` method is updated to take into account the improved [compression estimation](./exec-engine.md#fees)
+accuracy as part of the Fjord upgrade.
+
+```solidity
+function getL1GasUsed(bytes memory _data) public view returns (uint256) {
+    if (isFjord) {
+      // Add 68 to the size to account for the unsigned tx
+      int256 flzSize = LibZip.flzCompress(_data).length + 68;
+
+      int256 estimatedSize = costIntercept + costFastlzCoef * flzSize;
+      if (estimatedSize < minTransactionSize) {
+        estimatedSize = minTransactionSize;
+      }
+
+      // Assume the compressed data is mostly non-zero, and would pay 16 gas per calldata byte
+      return estimatedSize * 16;
+    }
+    // ...
+}
+```
+
+The `getL1GasUsed` method will be deprecated. This is due to it not accurately estimating the
+L1 gas used, for a transaction. In a future network upgrade this function will revert when called.
+
+Users can continue to use the `getL1FeeUpperBound` or `getL1Fee` method to estimate the L1 fee for a given transaction.

specs/protocol/derivation.md

@@ -1040,7 +1040,7 @@ special transactions may be inserted as part of the derivation process.
 
 #### Ecotone
 
-The Ecotone hardfork activation block, contains the following transactions in this order:
+The Ecotone hardfork activation block contains the following transactions, in this order:
 
 - L1 Attributes Transaction, using the pre-Ecotone `setL1BlockValues`
 - User deposits from L1

specs/protocol/predeploys.md

@@ -244,8 +244,8 @@ has been hardcoded to 6.
 
 Following the Ecotone upgrade, the values used for L1 fee computation are:
 
-- l1BaseFeeScalar
-- l1BlobBaseFeeScalar
+- baseFeeScalar
+- blobBaseFeeScalar
 - decimals
 
 [ecotone-scalars]: system_config.md#ecotone-scalar-overhead-uint256uint256-change