feat: gas snapshots over arbitrary sections

[zerosnacks]

Motivation

Closes: #2065 + Closes: #2056 + Closes: #8713

Supports:

Uses a snapshots directory without . prefix to indicate it is meant to be checked in.

Cheatcodes:

• snapshotValue
• snapshotGas (wraps lastCallGas)
• startSnapshotGas
• stopSnapshotGas

All cheatcodes support grouping, allowing you to group arbitrary snapshots together by group name. If the group name is not supplied the name is used as filename.

We support multiple outputs:

If no group is supplied, uses the contract name as file name e.g. snapshots/GasSnapshotTest.json:

{
 "a": "123",
 "b": "456",
 "c": "789",
 "d": "123",
 "e": "456",
 "f": "789",
 "testSnapshotGasSection": "17439512"
}

Where the order is always sorted alphabetically and deterministic.

If a group name is supplied, uses the custom name e.g. CustomGroup (stored in snapshots/CustomGroup.json)

{
  "e": "456",
  "i": "456",
  "o": "123",
  "q": "789",
  "x": "123",
  "z": "789"
}

Supports FORGE_SNAPSHOT_CHECK=true flag to assert gas snapshots haven't changed across runs

Breaking

• Updates generic snapshot to snapshotState throughout the codebase in a breaking way

Notes

• As marked in feat(snapshots): track internal gas usage #3766 we do not support tracking internal gas.

To emulate internal gas tracking one could use:

    function testInternalSnapshotValue() public {
        uint256 a = gasleft();

        _run(1);

        uint256 b = gasleft();

        vm.snapshotValue("testInternalSnapshotValue", a - b);
    }

Alternatively this could be added to forge-std?

• Includes some formatting fixes raised by the Solidity compiler as warnings that were causing issues

To do

• Snapshots are currently not cleared by themselves at the start of running the test suite. It seems desirable that this would be the case to prevent staining
• A FORGE_SNAPSHOT_CHECK=true environment variable to revert on a mismatch in the snapshot with gas used
• If no group is passed automatically generate the snapshot group: for example testContractName as a default if not overridden with a specific string.
• Add vm.snapshotGasLastCall to wrap lastCallGas? Single call is a very common use case and would be nice to skip the wrapping steps

Follow up

Deriving function names automatically is quite complex as the test being ran isn't aware of the function specific context, only the contract.

I've set it up in a way (w/ name: Option<String>) to easily add it in a follow-up PR in a non-breaking way.

• If no name is passed automatically generate the snapshot name: for example testFunction as a default if not overridden with a specific string.

[marktoda]

would be nice to have a check for multiple snapshots with the same name (common mistake causing confusing snapshot thrashing based on test runner ordering)

[marktoda]

recognize this is probably harder but would love if name could be optional and autogenerated as testContract.testFunction

[zerosnacks]

Supportive, I think it should be possible

[zerosnacks]

I was able to add support for generating group names by contract name but haven't figured out a way to support derived function names yet. This can be added as a follow-up PR in a non-breaking way once a good way has been found.

[mds1]

I agree with this set of name change for state snapshots, since the term "snapshot" is now broad, but just noting this is a breaking change so we'll need to make sure to communicate that

[DaniPopes]

We can should keep the old names and mark them as deprecated (#[cheatcode(... status = Deprecated)],

foundry/crates/cheatcodes/spec/src/cheatcode.rs

Line 48 in cb109b1

Deprecated,

), these kinds of breaking changes are not a good experience.

I don't believe the deprecated warnings logic is implemented yet, but it should be straight forward to implement. CC @mattsse @klkvr

[zerosnacks]

Re-added the old cheatcodes and tagged them as Deprecated, sharing the inner logic with the new implementation

[marktoda]

Love the idea of groups - one thought is, does BTreeMap serialize deterministically? Given test run ordering isn't deterministic afaik, worried thrashing of snapshot output will make diffs more difficult to read. Would ideally have deterministic output

[marktoda]

I guess since BTreeMap is ordered on name and presumably serde_json just iterates over it, it's likely deterministic

[zerosnacks]

Good point!

Given that upon applying an update the json is read into a BTreeMap, changes are applied and then serialized again the order should be deterministic. I'll need to add more tests to confirm this assumption is correct, but that is the goal.

[zerosnacks]

Going to split out the state snapshot renaming into its own PR as it makes it difficult to iterate and review here

[zerosnacks]

Closing in favor of #8952

I've split out the work in this PR into: #8945 (snapshot renaming) and #8952 (gas snapshots over sections) that branches off #8945. This will make it easier to review and collaborate.