Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ledger: move iterators into a store package (10 of N) #4874

Merged
merged 7 commits into from
Dec 15, 2022

Conversation

icorderi
Copy link
Contributor

@icorderi icorderi commented Dec 7, 2022

Summary

This PR creates a few more methods needed by the deltas.

Previous parts of this refactor:

What remains to be moved out:

  • iterators
    • orderedAccountsIter
    • catchpointPendingHashesIterator
  • deltas
    • accountsLoadOld and resourcesLoadOld have a couple queries that we might want to move out to avoid pulling all the deltas into store
  • misc queries left in ledger
    • acctdeltas_test.go
    • acctupdates_test.go
    • catchpointtracker.go
    • catchpointwriter.go
    • catchpointwriter_test.go

Test Plan

Existing tests.

@codecov
Copy link

codecov bot commented Dec 7, 2022

Codecov Report

Merging #4874 (1b984e7) into master (c0c5a94) will decrease coverage by 0.44%.
The diff coverage is 2.19%.

@@            Coverage Diff             @@
##           master    #4874      +/-   ##
==========================================
- Coverage   54.01%   53.57%   -0.45%     
==========================================
  Files         426      429       +3     
  Lines       53425    53426       +1     
==========================================
- Hits        28859    28624     -235     
- Misses      22305    22573     +268     
+ Partials     2261     2229      -32     
Impacted Files Coverage Δ
catchup/ledgerFetcher.go 39.32% <ø> (ø)
ledger/acctdeltas.go 80.15% <ø> (ø)
ledger/store/catchpointPendingHashesIter.go 0.00% <0.00%> (ø)
ledger/store/encodedAccountsIter.go 0.00% <0.00%> (ø)
ledger/store/orderedAccountsIter.go 0.00% <0.00%> (ø)
ledger/catchpointtracker.go 58.34% <40.00%> (ø)
ledger/catchpointwriter.go 60.00% <100.00%> (+0.31%) ⬆️
ledger/catchupaccessor.go 64.76% <100.00%> (ø)
ledger/roundlru.go 90.56% <0.00%> (-5.67%) ⬇️
ledger/blockqueue.go 85.63% <0.00%> (-2.88%) ⬇️
... and 9 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@brianolson
Copy link
Contributor

top level package 'encoded' makes me wonder what encoded? maybe ledger/encoded ? It's the in-database encoded form of some things, right?

@icorderi
Copy link
Contributor Author

@brianolson they are mostly used in catchpoints, but leak on some of the db interfaces.
I can move it down to ledger/encoded

brianolson
brianolson previously approved these changes Dec 14, 2022
Copy link
Contributor

@brianolson brianolson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@algorandskiy algorandskiy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix err=nil in loadOld, and maybe rename encoded?

ledger/encoded/recordsV6.go Show resolved Hide resolved
ledger/acctdeltas.go Show resolved Hide resolved
@@ -166,6 +167,14 @@ type catchpointCatchupAccessorImpl struct {
nextExpectedAccount basics.Address
}

// catchpointAccountResourceCounter keeps track of the resources processed for the current account
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is a copy in store/encodedAccountsIter.go

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's an accumulator used during iteration.
It seemed strange to export the type from store since it makes more sense for it to live here.

ledger/encoded/recordsV6.go Show resolved Hide resolved
@algorandskiy algorandskiy merged commit 9c72c4e into algorand:master Dec 15, 2022
@algorandskiy algorandskiy changed the title refactor: accountdb.go into a store package (10 of N) ledger: move iterators into a store package (10 of N) Dec 15, 2022
@icorderi icorderi deleted the refactor/accountdb-10 branch December 16, 2022 21:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants