Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blockchain: Add ancestor optimization to finding Ancestor #1688

Merged
merged 1 commit into from
Dec 7, 2023

Conversation

kcalvinalvin
Copy link
Collaborator

This mimics the behavior of Ancestor() in bitcoind

@coveralls
Copy link

coveralls commented Feb 2, 2021

Pull Request Test Coverage Report for Build 530293241

Warning: This coverage report may be inaccurate.

We've detected an issue with your CI configuration that might affect the accuracy of this pull request's coverage report.
To ensure accuracy in future PRs, please see these guidelines.
A quick fix for this PR: rebase it; your next report should be accurate.

  • 28 of 28 (100.0%) changed or added relevant lines in 1 file are covered.
  • 33 unchanged lines in 3 files lost coverage.
  • Overall coverage increased (+0.04%) to 53.439%

Files with Coverage Reduction New Missed Lines %
blockchain/accept.go 1 80.43%
blockchain/blockindex.go 8 93.92%
blockchain/chainio.go 24 60.35%
Totals Coverage Status
Change from base Build 512434868: 0.04%
Covered Lines: 20870
Relevant Lines: 39054

💛 - Coveralls

Copy link
Member

@Roasbeef Roasbeef left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like a cool optimization, but I think the core algorithm could be implemented a in simpler manner for clarity, and also tested better. Completed an initial pass, but will do another to really grok the algorithm.

blockchain/blockindex.go Show resolved Hide resolved
blockchain/blockindex.go Outdated Show resolved Hide resolved
blockchain/blockindex.go Outdated Show resolved Hide resolved
blockchain/blockindex.go Outdated Show resolved Hide resolved
blockchain/blockindex.go Outdated Show resolved Hide resolved
blockchain/blockindex.go Outdated Show resolved Hide resolved
blockchain/blockindex.go Outdated Show resolved Hide resolved
heightSkip := getSkipHeight(heightWalk)
heightSkipPrev := getSkipHeight(heightWalk - 1)

if indexWalk.ancestor != nil &&
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite a mouth-full of a conditional...

It also isn't immediately clear the significance (nor origin) of the -2 here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got rid of everything here and changed the logic to be simpler.

idk why that -2 is there as well... Some sipa magic

@davecgh
Copy link
Member

davecgh commented Nov 17, 2021

Commenting here because I just saw @Roasbeef's tweet linking to it. FYI I implemented something similar in dcrd back in 2019 in what is imo a simpler way and should be a simple backport:

decred/dcrd#2010

On startup, Ancestor call was taking a lot of time when the node was
loading the blockindex onto memory. This change speeds up the Ancestor
function significantly and speeds up the node during startup.

On testnet3 at blockheight ~2,500,000, the startup was around 30seconds
on current main and was 5 seconds with this change. Below is a benchstat
result showing the significant speedup.

goos: darwin
goarch: arm64
pkg: github.com/utreexo/utreexod/blockchain
           │     old.txt      │               new.txt                │
           │      sec/op      │    sec/op     vs base                │
Ancestor-8   120819.301µ ± 5%   7.013µ ± 19%  -99.99% (p=0.000 n=10)

           │  old.txt   │            new.txt             │
           │    B/op    │    B/op     vs base            │
Ancestor-8   0.000 ± 0%   0.000 ± 0%  ~ (p=1.000 n=10) ¹
¹ all samples are equal

           │  old.txt   │            new.txt             │
           │ allocs/op  │ allocs/op   vs base            │
Ancestor-8   0.000 ± 0%   0.000 ± 0%  ~ (p=1.000 n=10) ¹
¹ all samples are equal
@kcalvinalvin
Copy link
Collaborator Author

Reviving this as I noticed that the startup of a node is slowed down quite a bit by the blockindex loading.

Here's a pprof of the startup blockindex loading without this PR.
IMG_0424

Here's one with the PR.
image

Added a benchmark as well. The benchstat results were:

    goos: darwin
    goarch: arm64
    pkg: github.com/utreexo/utreexod/blockchain
               │     old.txt      │               new.txt                │
               │      sec/op      │    sec/op     vs base                │
    Ancestor-8   120819.301µ ± 5%   7.013µ ± 19%  -99.99% (p=0.000 n=10)
    
               │  old.txt   │            new.txt             │
               │    B/op    │    B/op     vs base            │
    Ancestor-8   0.000 ± 0%   0.000 ± 0%  ~ (p=1.000 n=10) ¹
    ¹ all samples are equal
    
               │  old.txt   │            new.txt             │
               │ allocs/op  │ allocs/op   vs base            │
    Ancestor-8   0.000 ± 0%   0.000 ± 0%  ~ (p=1.000 n=10) ¹
    ¹ all samples are equal

@Roasbeef Roasbeef requested review from guggero and removed request for Crypt-iQ December 6, 2023 23:46
Copy link
Member

@Roasbeef Roasbeef left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🎢

This significantly cut down on node start up time on testnet + mainnet.

@Roasbeef Roasbeef merged commit ac068e7 into btcsuite:master Dec 7, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants