Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

account min-balance #825

Closed
wants to merge 62 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
3e9c0c3
markdown lint
Jan 14, 2022
eadb8a7
Merge remote-tracking branch 'origin' into tzaffi/min-balance
Jan 14, 2022
1432816
update submodule
Jan 15, 2022
957070c
more control ovedr submodule
Jan 15, 2022
5a67bc5
typo
Jan 15, 2022
f0da64b
experimental differ scripts
Jan 15, 2022
bbfb7fe
reporting diffs
Jan 15, 2022
45babc0
reasonable looking reports
Jan 15, 2022
7f658fe
human friendly parity report
Jan 16, 2022
98205d0
maybe now it's human legible
Jan 16, 2022
0e0f61f
maybe now it human legible
Jan 16, 2022
cdd986e
reporting in good shape
Jan 16, 2022
779a347
summary code
Jan 16, 2022
2851191
wip
Jan 17, 2022
bde9d1a
rm
Jan 17, 2022
23a7264
rm
Jan 17, 2022
b727bdb
yaml is the best
Jan 17, 2022
ceb74ae
looking good
Jan 17, 2022
a7335a0
bare essentials diff of swaggers
Jan 17, 2022
997de01
set asserted diffs as drops and full
Jan 17, 2022
3405292
simplification
Jan 17, 2022
207fcfa
simplify
Jan 17, 2022
bf42127
imports reorg
Jan 17, 2022
8281a16
simplify
Jan 17, 2022
835b4b2
refresh submodule
Jan 17, 2022
7814815
pyaml in the right requirements.txt
Jan 17, 2022
81f5ab6
add algod v. indexer parity check to circle
Jan 17, 2022
1f73166
parity test
Jan 17, 2022
73ff3fe
try parity again (.PHONYness)
Jan 17, 2022
0f701cb
sync instead of force
Jan 17, 2022
b7dc9c0
don't need requirements in parity anymore
Jan 17, 2022
600c873
EOL
Jan 17, 2022
4ff2e35
Update parity/test_indexer_v_algod.py
tzaffi Jan 17, 2022
f3fe681
remove extra space
Jan 17, 2022
289bf2b
investigatory comment
Jan 18, 2022
e352b1c
update swagger for min-balance and a couple of others out of sync
Jan 19, 2022
a5ed930
stub the MinBalance using an Enricher()
Jan 20, 2022
e225d5a
pass lint
Jan 20, 2022
c5730ef
skip parity test by removing it from .PHONY
Jan 20, 2022
82341ba
do pg port changes bork circle?
Jan 20, 2022
73d6cae
did the tests get borked? yes - common.sh actually needs port 5434
Jan 20, 2022
a18bc0b
rename swagger make command
Jan 20, 2022
317ea61
remove parity test from Circle (memorializing issue https://github.co…
Jan 20, 2022
aa8673b
revert
Jan 20, 2022
84846e2
min-balance should not be required, especially when rewinding
Jan 21, 2022
a9906aa
enriching fetchAccounts with MinBalance
Jan 25, 2022
054de94
Update Makefile
tzaffi Jan 25, 2022
f8e5b39
Update Makefile
tzaffi Jan 25, 2022
7ef690b
oops == nil
tzaffi Jan 25, 2022
2bc1eb1
go fmt
Jan 25, 2022
e73ec07
lint
Jan 25, 2022
b11545f
Update api/handlers_test.go
tzaffi Jan 25, 2022
fe41ad9
Merge remote-tracking branch 'origin' into tzaffi/min-balance
Jan 25, 2022
6e8aa8b
breakout algodvindexer into its own PR
Jan 25, 2022
522a03a
re-merge
Jan 25, 2022
f31b049
Update accounting/calculated.go
tzaffi Jan 25, 2022
2610b8a
Merge remote-tracking branch 'origin/develop' into tzaffi/min-balance
Jan 25, 2022
e28e1d0
merge
Jan 25, 2022
bbd3c66
revert
Jan 25, 2022
aca823c
Merge branch 'develop' of https://github.com/algorand/indexer into tz…
Jan 28, 2022
481e45d
replace entire &blockheader with &blockheader.CurrentProtocol
Jan 31, 2022
d45da61
fmt
Jan 31, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,6 @@ __pycache__
*.tar.bz2

coverage.txt

# mac OS
.DS_store
156 changes: 156 additions & 0 deletions accounting/calculated.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
package accounting

import (
"fmt"

"github.com/algorand/go-algorand/config"
"github.com/algorand/go-algorand/data/basics"
"github.com/algorand/go-algorand/protocol"
"github.com/algorand/indexer/api/generated/v2"
)

// EnrichMinBalance addes the MinBalance information to a generated.Account by:
// 1. projecting its essential maintenance cost information to an initially empty basics.AccountData bAccount
// 2. calculating bAccount.MinBalance(protocol) using the protocol gotten from a recent blockheader
// 3. adding the result to the generated.Account
//
// TODO: this could present compatability challenges with rewind() as we need to also rewind the MinBalance calculation-logic and protocol along with the account.
//
// TODO: use new MinBalance() function that doesn't require slice lengths when branch feature/unlimited-assets is merged in go-algorand
func EnrichMinBalance(account *generated.Account, consensusVersion *protocol.ConsensusVersion) error {
if consensusVersion == nil {
return fmt.Errorf("cannot EnrichMinBalance as consensus version is missing")
}
proto, ok := config.Consensus[*consensusVersion]
if !ok {
return fmt.Errorf("cannot EnrichMinBalance as consensus version %s is unknown", *consensusVersion)
}
raw := minBalanceProjection(account).MinBalance(&proto).Raw
account.MinBalance = &raw
return nil
}

// inspired by api::pointer_utils.go and then renamed
// TODO: unify common utils

func boolZerovalIfNil(b *bool) bool {
if b != nil {
return *b
}
return false
}

func uint32ZerovalIfNil(x *uint64) uint32 {
if x != nil {
return uint32(*x)
}
return uint32(0)
}

func convertAssetHoldings(gAssets *[]generated.AssetHolding) map[basics.AssetIndex]basics.AssetHolding {
// assumes WYSIWYG ignoring round information except for Deleted which is skipped
// TODO: handle OptedInAtRound and OptedOutAtRound
// Make sure to add an integration test case that opts in and out and also back in
if gAssets == nil {
return nil
}
bAssets := make(map[basics.AssetIndex]basics.AssetHolding)
for _, gAsset := range *gAssets {
if boolZerovalIfNil(gAsset.Deleted) {
continue
}

bAssetID := basics.AssetIndex(gAsset.AssetId)
if _, alreadyExists := bAssets[bAssetID]; alreadyExists {
// skip dupes
continue
}

// for the purposes if MinBalance calculation, there is no need for Amount field, etc.
bAssets[bAssetID] = basics.AssetHolding{}
}
if len(bAssets) > 0 {
return bAssets
}
return nil
}

func convertAppsCreated(gApps *[]generated.Application) map[basics.AppIndex]basics.AppParams {
// assumes WYSIWYG ignoring most round information except for Deleted or DeletedAtRound which are skipped
if gApps == nil {
return nil
}
bAapps := make(map[basics.AppIndex]basics.AppParams)
for _, gApp := range *gApps {
if boolZerovalIfNil(gApp.Deleted) || gApp.DeletedAtRound != nil {
continue
}
bAppID := basics.AppIndex(gApp.Id)
if _, alreadyExists := bAapps[bAppID]; alreadyExists {
// skip dupes
continue
}

// MinBalance doesn't dig into AppParams, so provide a zeroval struct
bAapps[bAppID] = basics.AppParams{}
}
if len(bAapps) > 0 {
return bAapps
}
return nil
}

func convertAppsOptedIn(gApps *[]generated.ApplicationLocalState) map[basics.AppIndex]basics.AppLocalState {
// assumes WYSIWYG ignoring most round information except for Deleted or ClosedOutAtRound which are skipped
if gApps == nil {
return nil
}
bAapps := make(map[basics.AppIndex]basics.AppLocalState)
for _, gApp := range *gApps {
if boolZerovalIfNil(gApp.Deleted) || gApp.ClosedOutAtRound != nil {
continue
}
bAppID := basics.AppIndex(gApp.Id)
if _, alreadyExists := bAapps[bAppID]; alreadyExists {
// skip dupes
continue
}

// MinBalance doesn't dig into AppLocalStates, so provide a zeroval struct
bAapps[bAppID] = basics.AppLocalState{}
}
if len(bAapps) > 0 {
return bAapps
}
return nil
}

// minBalanceProjection projects a part of a generated.Account to a basics.AccountData struct.
// It does so for the purpose of calculating an account's minumum balance using the official
// calculator go-algorand/data/basics/userBalance.go::MinBalance()
// Some data which is relevant for MinBalance but duplicated is ignored. In particular:
// * TotalAppSchema - gotten from account.AppsTotalSchema and not re-calc'ed from CreatedApps
// * TotalExtraPages - gotten from account.AppsTotalExtraPages and not re-calc'ed from CreatedApps
func minBalanceProjection(account *generated.Account) basics.AccountData {
Copy link
Contributor Author

@tzaffi tzaffi Jan 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a similar function AccountToAccountData in go-algorand that fills out all of the fields (but has some minor issues when applied to Indexer's generated.Account). It is also possible to try and unmarshall the basics.Account directly from the database as is partially done elsewhere. After some thought, I decided that this was the simplest approach - get only the essential data into a basics.AccountData at the most convenient location and calculate MinBalance from it.

if boolZerovalIfNil(account.Deleted) {
return basics.AccountData{}
}

totalAppSchema := basics.StateSchema{}
if account.AppsTotalSchema != nil {
totalAppSchema.NumUint = account.AppsTotalSchema.NumUint
totalAppSchema.NumByteSlice = account.AppsTotalSchema.NumByteSlice
}
totalExtraAppPages := uint32(0)
if account.AppsTotalExtraPages != nil {
totalExtraAppPages = uint32ZerovalIfNil(account.AppsTotalExtraPages)
}

return basics.AccountData{
Assets: convertAssetHoldings(account.Assets),
AppParams: convertAppsCreated(account.CreatedApps),
AppLocalStates: convertAppsOptedIn(account.AppsLocalState),
TotalAppSchema: totalAppSchema,
TotalExtraAppPages: totalExtraAppPages,
}
}
Loading