Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go-algorand 3.5.1-stable #3798

Merged
merged 41 commits into from
Mar 18, 2022

Conversation

Algo-devops-service
Copy link
Contributor

@Algo-devops-service Algo-devops-service commented Mar 17, 2022

(Release notes being developed independently.)

michaeldiamant and others added 30 commits March 2, 2022 09:58
)

## Summary

Provides a pointer to instructions added in algorand/indexer#906 for updating Indexer E2E test input.

## Test Plan

N/A - Changes docs.  Though I expect to trial the instructions as discussed in algorand/indexer#906.
…4.2-remerge

go-algorand relstable3.4.2-remerge
## Summary

This PR adds support of for unlimited assets.

Done:
- [x] Update this title
- [x] Complete & Merge the implementation of the accounts prefetched during the validation (algorand#3666)
- [x] Create & Merge a spec update ( algorandfoundation/specs#62 (comment) )
- [x] Create & Merge a consensus upgrade (algorand#3674)

## Test Plan

Extend, update and add new unit tests.
## Summary
Refactored WriteThread such that it'll be easier to add new operations. 
Didn't delete the DBs`inner` functions, instead moved them to be handled by a struct that implements `dbOp` interface
…tion (algorand#3707)

## Summary

Follow-on to algorand#3702, the copy-pasted code in AccountApplicationInformation was checking the wrong value before returning AppParams.
## Summary

This adds configuration options to adjust the sizes of buffered channels used by the agreement system for incoming messages. 

## Test Plan

Existing tests should pass.
…ails message (algorand#3703)

## Summary

This pulls a bit of code from the the [pipelining branch](https://github.com/algorand/go-algorand/tree/feature/pipeline) to add a "validatedAt" duration field to the agreement.proposal type. This is used to provide an extra field in the BlockAcceptedEventDetails message describing how soon into the round each block was validated.

## Test Plan

Existing tests should pass, and a new test might be helpful to show that the value is getting filled in correctly.
Summary

The existing code was failing the fast catchup on node startup if the peers could not be retrieved from the network package.
Unfortunately, this is almost always the case. This change allow the node to retry fetching the peers list from the network package multiple times, while having a short delay between the iterations.

This issue was reported by a Algorand Forum member.

Test Plan

Test the change manually.
…nd#3710)

## Summary

The TestAssetConfig was not updated correctly to support unlimited assets.
This PR brings it up to date, as well as improve the runtime of the test.

## Test Plan

Use Circle-CI.
## Summary

Similar to algorand#1938 this adds the TagCounter to report message counters by type. It also removes labels from the existing Counters, which should improve their performance.

## Test Plan

Existing tests should pass.
…lgorand#3708)

## Summary

This replaces the various Ledger interfaces' `LookupResource` method with two more specific `LookupApplication` and `LookupAsset` methods. Following up on code review feedback from algorand#3652.

## Test Plan

Existing tests should pass, including the ones that implement their own mock ledger.
## Summary

There was one Clock implementation not updated in algorand#3703 in the agreement fuzzer testing package's NetworkFacade type. This updates it to fully implement the Clock interface.
## Summary

The `satori/go.uuid` dependency was reported to contain a vulnerability ( https://nvd.nist.gov/vuln/detail/CVE-2021-3538#VulnChangeHistorySection ).

While this vulnerability would have no negative impact on the Algorand blockchain, it does mislead automated vulnerability detectors and points the Algorand codebase to be considered insecure.

Given that this package was used in so few locations, and in such a small scope, I have removed the usage of it completely. Instead, I have implemented a random UUID which would work correctly for our needs.

## Test Plan

Unit tests added.
## Summary

In algorand#3341 I neglected to notice there are two places that generate telemetry configuration, ReadTelemetryConfigOrDefault and EnsureTelemetryConfigCreated. This updates the latter.

## Test Plan

Updated tests to check and handle new default.
…#3717)

## Summary

The `DeleteOldKeys` method was taking the lock for the duration of the keys deletion. This is not required, as the mutex really just need to be held to synchronize the list of participation keys. The underlying `OneTimeSignatureSecrets` already have a synchronization lock, which is taken as need.

## Test Plan

Unit test added.

The output of the test help to detect the timing issues addressed in this PR. Before this PR, calling Key() 10 times took 4.1 seconds. With this PR, it takes 255us.
## Summary

Tests for eval prefetcher checking that the prefetcher loads the same data (with some exceptions) that the evaluator requests.

Closes https://github.com/algorand/go-algorand-internal/issues/1922.

## Test Plan

This is tests.
* fixed off-by-one error in the assembler's typeDig() and improved unit test

* e2e test for CSP

* added additional test and cleaned up unused variables

* broadened test and corrected erroneous assumption about csp inner app calls
## Summary

Move eval prefetcher to its own package for better isolation. Some other refactoring was necessary.

## Test Plan

This is refactoring.
## Summary

Complementing algorand/indexer#916 this merges the AccountsErrorResponse fields into the "data" object in ErrorResponse, so SDKs do not to have to distinguish between different error response types.

## Test Plan

Update existing tests checking the old AccountsErrorResponse fields.
…-bit aligned address (algorand#3740)

## Summary

prefetcher: ensure the groupTask.incompleteCount is allocated on a 64-bit aligned address.

This is required in order to avoid the following on a ARM32:
```
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xb541efb0]

goroutine 4220 [running]:
runtime/internal/atomic.goLoad64(0x8ae33be4, 0xb67e0fec, 0x8cf54500)
	runtime/internal/atomic/atomic_arm.go:131 +0x1c
github.com/algorand/go-algorand/ledger/internal.(*accountPrefetcher).prefetch(0x8cf54500, 0xb68023f0, 0x8cf532c0)
	github.com/algorand/go-algorand/ledger/internal/evalprefetcher.go:384 +0x1268
created by github.com/algorand/go-algorand/ledger/internal.prefetchAccounts
	github.com/algorand/go-algorand/ledger/internal/evalprefetcher.go:87 +0x128
```

## Test Plan

Test manually.
Three opcodes that all pertain to FIDO2 authorization.  They are slated for Teal7 here, but they can be delayed by changing fidoVersion.

Co-authored-by: Igor Durovic <[email protected]>
Co-authored-by: algoidurovic <[email protected]>
Co-authored-by: [email protected] <[email protected]>
## Solution 

ledger refactoring: test 6M assets/apps per account
resolves https://github.com/algorand/go-algorand-internal/issues/1857


## Testing

This is a test
Summary
This PR replaces the usage of the LibGoal with the direct RestClient. The change could have performance implications when running against large sets of data.

Test Plan
This is a test.
* Fix typo in OpenAPI json and yml files

* Included output of make generate
## Summary

This PR improves the test in the following ways:
1. The checkPoint method is no longer timer-based; instead, it's using the WaitForBlock which moves the blocking operation to the node.
2. The generated accounts secret keys are now deterministic. This would allows repeated runs to have the same set of keys.
3. More testing was added for return variables across the test.

## Test Plan

This is a test.
Summary
The goal-partkey-information test randomly fails.
The changes in this PR improve the intermediate logs, which would be helpful in analyzing the source of the failure.

Test Plan
This is a test.
cce and others added 11 commits March 11, 2022 22:57
…rand#3759)

Summary
The ed25519 batch verification implementation in algorand#3031 provides a performance improvement for validating multiple signatures (such as multiple transaction signatures). Since each OneTimeSignature used by agreement votes is actually 3 ed25519 signatures, this hooks up the verifier to the batch verification implementation, yielding a ~12% performance improvement in the included benchmark on my computer.

Test Plan
Added benchmark, existing tests should pass.
Simplify handling of "active slots" to fix possible obo error. Unit tests added to confirm typing.
* SHA3-256 added

Co-authored-by: Matt-Ryley <[email protected]>
…nsus version (algorand#3730)

## Summary

When the agreement receives a proposal from a consensus version it cannot yet support, the node disconnects from the sender and logs the fact that it was unable to decode the proposal without providing guidance on the consensus version issue. This PR adds a warning message to the log specifying that the agreement cannot handle the consensus version's proposals
## Summary

LookupLatest combines base account and resources data, and in some cases it cannot
determine reliably how many resources account has, therefore it reads data directly from the DB.
The DB might advance causing cachedDBRound (and base account data) and  resourceDbRound
to be out of sync, and a retry needed.
If on retry baseAccount data is outdated, foundAccount flag incorrectly contains a value from
a previous iteration causing completion by checkDone because of empty ad (ledgercore.AccountData).
Having ad and foundAccount synchronized on retry eliminates the problem.

In addition checkDone improved for scenarios when accounts have only own assets.

## Test Plan

Added new test
## Summary

This PR completes the previous started efforts and ensure all the requests are tunneled directly to the participation registry for optimal performance.

## Test Plan

Unit tests added.
@codecov-commenter
Copy link

codecov-commenter commented Mar 17, 2022

Codecov Report

Merging #3798 (53cb3c5) into rel/stable (b56953f) will decrease coverage by 0.10%.
The diff coverage is 50.11%.

@@              Coverage Diff               @@
##           rel/stable    #3798      +/-   ##
==============================================
- Coverage       49.90%   49.79%   -0.11%     
==============================================
  Files             382      392      +10     
  Lines           64411    68780    +4369     
==============================================
+ Hits            32144    34251    +2107     
- Misses          28838    30769    +1931     
- Partials         3429     3760     +331     
Impacted Files Coverage Δ
agreement/proposal.go 71.96% <ø> (ø)
catchup/catchpointService.go 0.00% <0.00%> (ø)
catchup/ledgerFetcher.go 37.63% <0.00%> (-7.24%) ⬇️
cmd/goal/application.go 11.16% <0.00%> (-0.02%) ⬇️
cmd/goal/interact.go 3.35% <0.00%> (-0.01%) ⬇️
config/localTemplate.go 42.85% <ø> (ø)
config/version.go 9.09% <ø> (ø)
crypto/falconWrapper.go 76.00% <ø> (ø)
daemon/algod/api/server/router.go 14.70% <0.00%> (-0.45%) ⬇️
daemon/algod/api/server/v1/handlers/handlers.go 0.63% <0.00%> (+<0.01%) ⬆️
... and 128 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b56953f...53cb3c5. Read the comment docs.

@onetechnical onetechnical marked this pull request as ready for review March 18, 2022 00:26
@algojohnlee algojohnlee merged commit aa2fb0e into algorand:rel/stable Mar 18, 2022
@onetechnical onetechnical deleted the relstable3.5.1 branch March 18, 2022 00:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.