Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

p2p: fan in incoming txns into backlog worker #6126

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

algorandskiy
Copy link
Contributor

@algorandskiy algorandskiy commented Sep 10, 2024

Summary

While investigating p2p TX traffic and performance, I found transaction pool mutex congestion. This PR is a PoC to use backlogWorker as a pool only accessor similarly to wsnet.
Implementation summary:

  • Added syncCh channel to work item wi
  • When it is initiated from gossipsub validator, syncCh is initiated and awaited making validation synchronous as before.
  • At each tx validation check backlogWorker checks if syncCh is set and responds with validation result to the channel.

Additionally, there are couple more fixes that helped with TPS as well:

  • add some workers to pull sub.Next() faster.
  • cache signed transaction ID to reuse in remember and txpool recomputing on a next block arrival.

Test Plan

  1. Unit tests passed
  2. Cluster TPS test showed +700-900 TPS gain (total 7.5k TPS) on a single txn payment scenario.
    image

Copy link

codecov bot commented Sep 10, 2024

Codecov Report

Attention: Patch coverage is 61.22449% with 19 lines in your changes missing coverage. Please review.

Project coverage is 56.25%. Comparing base (8b6c443) to head (5635798).

Files with missing lines Patch % Lines
data/txHandler.go 58.33% 7 Missing and 3 partials ⚠️
data/transactions/signedtxn.go 0.00% 4 Missing and 1 partial ⚠️
network/p2pNetwork.go 80.00% 3 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6126      +/-   ##
==========================================
- Coverage   56.25%   56.25%   -0.01%     
==========================================
  Files         494      494              
  Lines       69946    69950       +4     
==========================================
- Hits        39350    39347       -3     
- Misses      27919    27925       +6     
- Partials     2677     2678       +1     
Flag Coverage Δ
56.25% <61.22%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@algorandskiy algorandskiy force-pushed the pavel/p2p-txhander-bg branch 4 times, most recently from a65acca to 2e055ca Compare September 20, 2024 18:54
@algorandskiy algorandskiy changed the title WIP: p2p: fan in incoming txns into backlog worker p2p: fan in incoming txns into backlog worker Sep 21, 2024
@algorandskiy algorandskiy marked this pull request as ready for review September 21, 2024 00:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant