-
Notifications
You must be signed in to change notification settings - Fork 804
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve eth/66 support #3616
Improve eth/66 support #3616
Conversation
Currently Besu handles hundreds of transaction message per seconds, so having a queue of 1M messages result in a lot of expired messages, make sense to reduce the capacity to drop incoming messages that could not be handled anyway. Signed-off-by: Fabio Di Fabio <[email protected]>
Currently on mainnet there are many thousand of pending transactions exchanged between peers, but Besu has a short memory of what has been exchanged with a specific peer, with the result that the same transaction is often exchanged back and forth with the same peer, expecially when the peer is another Besu. Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
f9e1182
to
e511d90
Compare
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
(int) Math.ceil(Math.sqrt(ethContext.getEthPeers().getMaxPeers())); | ||
} | ||
|
||
public void handlePeerConnection(final EthPeer peer) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
non-blocking suggestion: since this isn't an interface method, and it is public, I think a more descriptive name might help. What about "relayTransactionsTo"? That would describe what is being done to the peer a bit more obviously.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
make sense, rename to relayTransactionPoolTo
ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPool.java
Show resolved
Hide resolved
public class Utils { | ||
private Utils() {} | ||
|
||
static List<Hash> toHashList(final Collection<Transaction> transactions) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we move this to Transaction instead of introducing a Util class? Seems like a reasonable location for it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
make sense, done
@@ -544,7 +554,7 @@ public void shouldDiscardRemoteTransactionThatAlreadyExistsBeforeValidation() { | |||
transactionPool.addRemoteTransactions(singletonList(transaction1)); | |||
|
|||
verify(pendingTransactions).containsTransaction(transaction1.getHash()); | |||
verify(pendingTransactions).tryEvictTransactionHash(transaction1.getHash()); | |||
// verify(pendingTransactions).tryEvictTransactionHash(transaction1.getHash()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
missed comment removal
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good catch!
Signed-off-by: Fabio Di Fabio <[email protected]>
this.newPooledTransactionHashesMessageSender = newPooledTransactionHashesMessageSender; | ||
this.ethContext = ethContext; | ||
this.numPeersToSendFullTransactions = | ||
(int) Math.ceil(Math.sqrt(ethContext.getEthPeers().getMaxPeers())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it maybe be better if this calculation got performed outside of constructor?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this value never changes during the execution, so I calculate it once in the constructor, are you suggesting to pass the result as parameter of the constructor?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was just thinking that in case I would like to manually set up this value to some concrete number, I cannot because its calculation is hardcoded in the constructor. I don't think if I ever should or if it makes sense to do it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the value is taken from https://eips.ethereum.org/EIPS/eip-2464 where it suggests to send full transactions only to the square root of peers. Not sure we want to ever change this value, but in that case we could make it configurable in future.
peersWithTransactionHashesSupport.size()); | ||
// add peer from the other list to reach the required size | ||
Collections.shuffle(peersWithTransactionHashesSupport); | ||
IntStream.range(0, delta) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe I don't understand properly what is going on here, I feel this would be more understandable if we would just use something like peersWithOnlyTransactionSupport.addAll(peersWithTransactionHashesSupport.sublist(..)); peersWithTransactionHashesSupport.removeRange(...);
This way it looks like a for cycle obscured in a functional way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes it could be hard to read, I like your suggestion but removeRange
is protected, so I am rewriting this with a plain for cycle
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Kudos, SonarCloud Quality Gate passed! |
Currently Besu has a limited support for sending NewPooledTransactionHashes messages, and other aspect related to reduce transactions synchronization traffic, described in the Ethereum Wire Protocol version 66. Specifically: Besu only uses NewPooledTransactionHashes for new local transactions, while it could be extended to any transaction added to the transaction pool Besu does not limit the sending of the full transaction messages to a small fraction of the connected peers, and sends the new transaction hashes to all the remaining peers This PR, extends eth/66 support and does some code refactoring, to remove some reduntant code and rename some classes to identify they are related to the NewPooledTransactionHashes message. The main changes are: Do not have a separate tracker for transaction hashes, since for them we can reuse PeerTransactionTracker, that tracks full transactions exchange history and sending queue with a peer. So PeerPendingTransactionTracker has been removed. --tx-pool-hashes-max-size is now deprecated and has no more effect and it will be removed in a future release. When a new peer connects, if it support eth/6[56] then we send all the transaction hashes we have in the pool, otherwise we send the full transactions. When new transactions are added to the pool, we send full transactions to peers without eth/6[56] support, or to a small fractions of all peers, and then we send only transaction hashes to the remaining peer that support eth/6[56]. Both transactions and transaction hashes are only sent if not already exchanged with that specific peer. Signed-off-by: Fabio Di Fabio <[email protected]> Signed-off-by: Justin Florentine <[email protected]>
Currently Besu has a limited support for sending NewPooledTransactionHashes messages, and other aspect related to reduce transactions synchronization traffic, described in the Ethereum Wire Protocol version 66. Specifically: Besu only uses NewPooledTransactionHashes for new local transactions, while it could be extended to any transaction added to the transaction pool Besu does not limit the sending of the full transaction messages to a small fraction of the connected peers, and sends the new transaction hashes to all the remaining peers This PR, extends eth/66 support and does some code refactoring, to remove some reduntant code and rename some classes to identify they are related to the NewPooledTransactionHashes message. The main changes are: Do not have a separate tracker for transaction hashes, since for them we can reuse PeerTransactionTracker, that tracks full transactions exchange history and sending queue with a peer. So PeerPendingTransactionTracker has been removed. --tx-pool-hashes-max-size is now deprecated and has no more effect and it will be removed in a future release. When a new peer connects, if it support eth/6[56] then we send all the transaction hashes we have in the pool, otherwise we send the full transactions. When new transactions are added to the pool, we send full transactions to peers without eth/6[56] support, or to a small fractions of all peers, and then we send only transaction hashes to the remaining peer that support eth/6[56]. Both transactions and transaction hashes are only sent if not already exchanged with that specific peer. Signed-off-by: Fabio Di Fabio <[email protected]>
Currently Besu has a limited support for sending NewPooledTransactionHashes messages, and other aspect related to reduce transactions synchronization traffic, described in the Ethereum Wire Protocol version 66. Specifically: Besu only uses NewPooledTransactionHashes for new local transactions, while it could be extended to any transaction added to the transaction pool Besu does not limit the sending of the full transaction messages to a small fraction of the connected peers, and sends the new transaction hashes to all the remaining peers This PR, extends eth/66 support and does some code refactoring, to remove some reduntant code and rename some classes to identify they are related to the NewPooledTransactionHashes message. The main changes are: Do not have a separate tracker for transaction hashes, since for them we can reuse PeerTransactionTracker, that tracks full transactions exchange history and sending queue with a peer. So PeerPendingTransactionTracker has been removed. --tx-pool-hashes-max-size is now deprecated and has no more effect and it will be removed in a future release. When a new peer connects, if it support eth/6[56] then we send all the transaction hashes we have in the pool, otherwise we send the full transactions. When new transactions are added to the pool, we send full transactions to peers without eth/6[56] support, or to a small fractions of all peers, and then we send only transaction hashes to the remaining peer that support eth/6[56]. Both transactions and transaction hashes are only sent if not already exchanged with that specific peer. Signed-off-by: Fabio Di Fabio <[email protected]>
PR description
Currently Besu has a limited support for sending NewPooledTransactionHashes messages, and other aspect related to reduce transactions synchronization traffic, described in the Ethereum Wire Protocol version 66.
Specifically:
This PR, extends eth/66 support and does some code refactoring, to remove some reduntant code and rename some classes to identify they are related to the NewPooledTransactionHashes message.
The main changes are:
--tx-pool-hashes-max-size
is now deprecated and has no more effect and it will be removed in a future release.Fixed Issue(s)
Changelog