Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transaction pool improvements to avoid filling the pool with not executable transactions #4425

Merged
merged 17 commits into from
Sep 23, 2022

Conversation

fab-10
Copy link
Contributor

@fab-10 fab-10 commented Sep 22, 2022

PR description

This PR is built on top of #4413 which was used to investigate the transaction pool, with the result that currently we keep many transactions that are not possible to include in a block, because a transaction with a lower nonce is missing or invalid.
To avoid that issues first we now remember (in a cache) that that sender has an invalid transaction, and we do not accept transactions with higher nonce from that sender, of course if a replacement for the invalid transaction is sent it is accepted.
The eviction when the pool is full has changed in order to pick the sender of the oldest transaction in the pool, and the evict the transaction with the higher nonce for that sender, so no gaps in the nonce list for that sender are created.

Fixed Issue(s)

fixes #4401

Documentation

  • I thought about documentation and added the doc-change-required label to this PR if
    updates are required.

Changelog

LOG,
"Adding {} to pending transactions, range type {}",
transactionInfo::toTraceLog,
kind::toString);

Check notice

Code scanning / CodeQL

Useless toString on String

Redundant call to 'toString' on a String object.
@fab-10 fab-10 changed the title Tx pool fixes Transaction pool improvement to avoid filling the pool with not executable transactions Sep 22, 2022
@fab-10 fab-10 changed the title Transaction pool improvement to avoid filling the pool with not executable transactions Transaction pool improvements to avoid filling the pool with not executable transactions Sep 22, 2022
@fab-10 fab-10 added the mainnet label Sep 22, 2022
@fab-10 fab-10 marked this pull request as ready for review September 22, 2022 18:28
Copy link
Contributor

@jflo jflo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestions below are non-blocking, would rather have this out sooner than adopt them. I have been running this branch successfully and when a proposal is triggered the block is full.

traceLambda(LOG, "Added remote transaction {}", transaction::toTraceLog);
addedTransactions.add(transaction);
break;
case ALREADY_KNOWN:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean the duplicate check at 145 is redundant with one the validateRemoteTransaction() method does? Optimization potential if we can avoid a full scan running twice.

}
}

public List<Transaction> signalInvalidTransaction(final Transaction transaction) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The return value here is not intuitive based on the method name. We should probably either add javadoc describing it, or rename to something like transactionsRemovedAfter(Transaction invalidTransaction)

.filter(tx::equals)
.isPresent())
.findFirst();
private static class LowestInvalidNonceCache {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty big for an inner class, maybe pull it out into its own class?

@garyschulte garyschulte merged commit 30bff31 into hyperledger:main Sep 23, 2022
@garyschulte
Copy link
Contributor

garyschulte commented Sep 23, 2022

merged after burn in on mainnet (4417, 4423, 4425) on a variety of hw and CL configurations

@fab-10
Copy link
Contributor Author

fab-10 commented Sep 23, 2022

Thanks @jflo for the review, I will implement your suggestion in a future PR

@fab-10 fab-10 deleted the tx-pool-fixes branch September 23, 2022 09:47
eum602 pushed a commit to lacchain/besu that referenced this pull request Nov 3, 2023
…utable transactions (hyperledger#4425)

* Add more tracing logs to transaction pool
* Prevent adding non executable transactions and evict based on age
* Move sender account in TransactionsForSenderInfo
* Implement a size expiration cache for lowest invalid nonce for sender

Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Justin Florentine <[email protected]>
Co-authored-by: Justin Florentine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Post-Merge some blocks contain 0 transactions
3 participants