Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CircularBuffer data structure #4913

Merged
merged 16 commits into from
Apr 26, 2024
Merged
5 changes: 5 additions & 0 deletions .changeset/cold-cheetahs-check.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'openzeppelin-solidity': minor
---

`CircularBuffer`: Add a data structure that stores the last `N` values pushed to it.
2 changes: 2 additions & 0 deletions contracts/mocks/Stateless.sol
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import {AuthorityUtils} from "../access/manager/AuthorityUtils.sol";
import {Base64} from "../utils/Base64.sol";
import {BitMaps} from "../utils/structs/BitMaps.sol";
import {Checkpoints} from "../utils/structs/Checkpoints.sol";
import {CircularBuffer} from "../utils/structs/CircularBuffer.sol";
import {Clones} from "../proxy/Clones.sol";
import {Create2} from "../utils/Create2.sol";
import {DoubleEndedQueue} from "../utils/structs/DoubleEndedQueue.sol";
Expand All @@ -24,6 +25,7 @@ import {ERC721Holder} from "../token/ERC721/utils/ERC721Holder.sol";
import {Math} from "../utils/math/Math.sol";
import {MerkleProof} from "../utils/cryptography/MerkleProof.sol";
import {MessageHashUtils} from "../utils/cryptography/MessageHashUtils.sol";
import {Panic} from "../utils/Panic.sol";
ernestognw marked this conversation as resolved.
Show resolved Hide resolved
import {Packing} from "../utils/Packing.sol";
import {SafeCast} from "../utils/math/SafeCast.sol";
import {SafeERC20} from "../token/ERC20/utils/SafeERC20.sol";
Expand Down
3 changes: 3 additions & 0 deletions contracts/utils/Arrays.sol
Original file line number Diff line number Diff line change
Expand Up @@ -455,6 +455,7 @@ library Arrays {
* WARNING: this does not clear elements if length is reduced, of initialize elements if length is increased.
*/
function unsafeSetLength(address[] storage array, uint256 len) internal {
/// @solidity memory-safe-assembly
assembly {
ernestognw marked this conversation as resolved.
Show resolved Hide resolved
sstore(array.slot, len)
}
Expand All @@ -466,6 +467,7 @@ library Arrays {
* WARNING: this does not clear elements if length is reduced, of initialize elements if length is increased.
*/
function unsafeSetLength(bytes32[] storage array, uint256 len) internal {
/// @solidity memory-safe-assembly
assembly {
ernestognw marked this conversation as resolved.
Show resolved Hide resolved
sstore(array.slot, len)
}
Expand All @@ -477,6 +479,7 @@ library Arrays {
* WARNING: this does not clear elements if length is reduced, of initialize elements if length is increased.
*/
function unsafeSetLength(uint256[] storage array, uint256 len) internal {
/// @solidity memory-safe-assembly
assembly {
ernestognw marked this conversation as resolved.
Show resolved Hide resolved
sstore(array.slot, len)
}
Expand Down
3 changes: 3 additions & 0 deletions contracts/utils/README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Miscellaneous contracts and libraries containing utility functions you can use t
* {EnumerableMap}: A type like Solidity's https://solidity.readthedocs.io/en/latest/types.html#mapping-types[`mapping`], but with key-value _enumeration_: this will let you know how many entries a mapping has, and iterate over them (which is not possible with `mapping`).
* {EnumerableSet}: Like {EnumerableMap}, but for https://en.wikipedia.org/wiki/Set_(abstract_data_type)[sets]. Can be used to store privileged accounts, issued IDs, etc.
* {DoubleEndedQueue}: An implementation of a https://en.wikipedia.org/wiki/Double-ended_queue[double ended queue] whose values can be removed added or remove from both sides. Useful for FIFO and LIFO structures.
* {CircularBuffer}: A data structure to store the last N values pushed to it.
* {Checkpoints}: A data structure to store values mapped to an strictly increasing key. Can be used for storing and accessing values over time.
* {MerkleTree}: A library with https://wikipedia.org/wiki/Merkle_Tree[Merkle Tree] data structures and helper functions.
* {Create2}: Wrapper around the https://blog.openzeppelin.com/getting-the-most-out-of-create2/[`CREATE2` EVM opcode] for safe use without having to deal with low-level assembly.
Expand Down Expand Up @@ -95,6 +96,8 @@ Ethereum contracts have no native concept of an interface, so applications must

{{DoubleEndedQueue}}

{{CircularBuffer}}

{{Checkpoints}}

{{MerkleTree}}
Expand Down
130 changes: 130 additions & 0 deletions contracts/utils/structs/CircularBuffer.sol
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
// SPDX-License-Identifier: MIT
pragma solidity ^0.8.20;

import {Math} from "../math/Math.sol";
import {Arrays} from "../Arrays.sol";
import {Panic} from "../Panic.sol";

/**
* @dev A fixed-size buffer for keeping `bytes32` items in storage.
*
* This data structure allows for pushing elements to it, and when its length exceeds the specified fixed size,
* new items take the place of the oldest element in the buffer, keeping at most `N` elements in the
* structure.
*
* Elements can't be removed but the data structure can be cleared. See {clear}.
*
* Complexity:
* - insertion ({push}): O(1)
* - lookup ({last}): O(1)
* - inclusion ({includes}): O(N) (worst case)
* - reset ({clear}): O(1)
*
* * The struct is called `Bytes32CircularBuffer`. Other types can be cast to and from `bytes32`. This data structure
* can only be used in storage, and not in memory.
Comment on lines +23 to +24
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the description I'm proposing, I don't think it's needed to specify these apply only to storage.

Suggested change
* * The struct is called `Bytes32CircularBuffer`. Other types can be cast to and from `bytes32`. This data structure
* can only be used in storage, and not in memory.
* Example usage:
*

*
* Example usage:
*
* ```solidity
* contract Example {
* // Add the library methods
* using CircularBuffer for CircularBuffer.Bytes32CircularBuffer;
*
* // Declare a buffer storage variable
* CircularBuffer.Bytes32CircularBuffer private myBuffer;
* }
* ```
*/
library CircularBuffer {
/**
* @dev Counts the number of items that have been pushed to the buffer. The residuo modulo _data.length indicates
* where the next value should be stored.
*
* Struct members have an underscore prefix indicating that they are "private" and should not be read or written to
* directly. Use the functions provided below instead. Modifying the struct manually may violate assumptions and
* lead to unexpected behavior.
*
* The last item is at data[(index - 1) % data.length] and the last item is at data[index % data.length]. This
* range can wrap around.
*/
struct Bytes32CircularBuffer {
uint256 _count;
ernestognw marked this conversation as resolved.
Show resolved Hide resolved
bytes32[] _data;
}

/**
* @dev Initialize a new CircularBuffer of given size.
*
* If the CircularBuffer was already setup and used, calling that function again will reset it to a blank state.
*
* NOTE: The size of the buffer will affect the execution of {includes} function, as it has a complexity of O(N).
* Consider a large buffer size may render the function unusable.
*/
function setup(Bytes32CircularBuffer storage self, uint256 size) internal {
clear(self);
Arrays.unsafeSetLength(self._data, size);
}

/**
* @dev Clear all data in the buffer without resetting memory, keeping the existing size.
*/
function clear(Bytes32CircularBuffer storage self) internal {
self._count = 0;
}

/**
* @dev Push a new value to the buffer. If the buffer is already full, the new value replaces the oldest value in
* the buffer.
*/
function push(Bytes32CircularBuffer storage self, bytes32 value) internal {
uint256 index = self._count++;
uint256 modulus = self._data.length;
Arrays.unsafeAccess(self._data, index % modulus).value = value;
}

/**
* @dev Number of values currently in the buffer. This value is 0 for an empty buffer, and cannot exceed the size of
* the buffer.
*/
function count(Bytes32CircularBuffer storage self) internal view returns (uint256) {
return Math.min(self._count, self._data.length);
}

/**
* @dev Length of the buffer. This is the maximum number of elements kepts in the buffer.
*/
function length(Bytes32CircularBuffer storage self) internal view returns (uint256) {
return self._data.length;
}

/**
* @dev Getter for the i-th value in the buffer, from the end.
*
* Reverts with {Panic-ARRAY_OUT_OF_BOUNDS} if trying to access an element that was not pushed, or that was
* dropped to make room for newer elements.
*/
function last(Bytes32CircularBuffer storage self, uint256 i) internal view returns (bytes32) {
uint256 index = self._count;
uint256 modulus = self._data.length;
uint256 total = Math.min(index, modulus); // count(self)
if (i >= total) {
Panic.panic(Panic.ARRAY_OUT_OF_BOUNDS);
}
return Arrays.unsafeAccess(self._data, (index - i - 1) % modulus).value;
}

/**
* @dev Check if a given value is in the buffer.
*/
function includes(Bytes32CircularBuffer storage self, bytes32 value) internal view returns (bool) {
ernestognw marked this conversation as resolved.
Show resolved Hide resolved
uint256 index = self._count;
uint256 modulus = self._data.length;
uint256 total = Math.min(index, modulus); // count(self)
for (uint256 i = 0; i < total; ++i) {
if (Arrays.unsafeAccess(self._data, (index - i - 1) % modulus).value == value) {
return true;
}
}
return false;
}
}
1 change: 1 addition & 0 deletions scripts/generate/templates/Arrays.js
Original file line number Diff line number Diff line change
Expand Up @@ -356,6 +356,7 @@ const unsafeSetLength = type => `
* WARNING: this does not clear elements if length is reduced, of initialize elements if length is increased.
*/
function unsafeSetLength(${type}[] storage array, uint256 len) internal {
/// @solidity memory-safe-assembly
assembly {
sstore(array.slot, len)
}
Expand Down
79 changes: 79 additions & 0 deletions test/utils/structs/CircularBuffer.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
const { ethers } = require('hardhat');
const { expect } = require('chai');
const { loadFixture } = require('@nomicfoundation/hardhat-network-helpers');
const { PANIC_CODES } = require('@nomicfoundation/hardhat-chai-matchers/panic');

const { generators } = require('../../helpers/random');

const LENGTH = 4;

async function fixture() {
const mock = await ethers.deployContract('$CircularBuffer');
await mock.$setup(0, LENGTH);
return { mock };
}

describe('CircularBuffer', function () {
beforeEach(async function () {
Object.assign(this, await loadFixture(fixture));
});

it('starts empty', async function () {
expect(await this.mock.$count(0)).to.equal(0n);
expect(await this.mock.$length(0)).to.equal(LENGTH);
expect(await this.mock.$includes(0, ethers.ZeroHash)).to.be.false;
await expect(this.mock.$last(0, 0)).to.be.revertedWithPanic(PANIC_CODES.ARRAY_ACCESS_OUT_OF_BOUNDS);
});

it('push', async function () {
const values = Array.from({ length: LENGTH + 3 }, generators.bytes32);

for (const [i, value] of values.map((v, i) => [i, v])) {
// push value
await this.mock.$push(0, value);

// view of the values
const pushed = values.slice(0, i + 1);
const stored = pushed.slice(-LENGTH);
const dropped = pushed.slice(0, -LENGTH);

// check count
expect(await this.mock.$length(0)).to.equal(LENGTH);
expect(await this.mock.$count(0)).to.equal(stored.length);

// check last
for (const j in stored) {
expect(await this.mock.$last(0, j)).to.equal(stored.at(-j - 1));
}
await expect(this.mock.$last(0, stored.length + 1)).to.be.revertedWithPanic(
PANIC_CODES.ARRAY_ACCESS_OUT_OF_BOUNDS,
);

// check included and non-included values
for (const v of stored) {
expect(await this.mock.$includes(0, v)).to.be.true;
}
for (const v of dropped) {
expect(await this.mock.$includes(0, v)).to.be.false;
}
expect(await this.mock.$includes(0, ethers.ZeroHash)).to.be.false;
}
});

it('clear', async function () {
const value = generators.bytes32();
await this.mock.$push(0, value);

expect(await this.mock.$count(0)).to.equal(1n);
expect(await this.mock.$length(0)).to.equal(LENGTH);
expect(await this.mock.$includes(0, value)).to.be.true;
await this.mock.$last(0, 0); // not revert

await this.mock.$clear(0);

expect(await this.mock.$count(0)).to.equal(0n);
expect(await this.mock.$length(0)).to.equal(LENGTH);
expect(await this.mock.$includes(0, value)).to.be.false;
await expect(this.mock.$last(0, 0)).to.be.revertedWithPanic(PANIC_CODES.ARRAY_ACCESS_OUT_OF_BOUNDS);
});
});