go-algorand 3.1.0-beta #2995
Merged
go-algorand 3.1.0-beta #2995
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Based on the matrix job @algojack added in #2749, this turns the other per-platform jobs into matrix jobs too. The most significant change in this PR is switching from building in the home directory (/home/circleci on Linux and /Users/distiller on Mac) to instead build out of /opt/cibuild to make it easier for the matrix jobs to share configuration.
Add a new ledger interface for Indexer that is significantly simpler and allows implementing batching in a more straight-forward way. As a result, the Indexer code doesn't need to know the specifics of go-algorand and doesn't need to implement its own accounts cache for batching.
TestWaitAndAddConnectionTimeLongtWindow is randomly failing when the execution of time sensitive statements is separated by more than the expected time gap. This issue is fixed by replacing the sleep with simulated time passing by changing the recorded time, thus giving the impression that the event happened timeout time ago.
The unit test TestLatestSigsFromThisNode was data racing. The fix is trivial.
## Summary The account totals are being tested once every round to ensure that the total amount of money in the system doesn't change. Prior to this PR, the accounts update newBlock method was responsible for testing that : this would ensure that we're not writing a new round to disk that violates the totals predicate. While this was working correctly, conducting this test at the time we're writing this information to the disk is too late. By that time, we've already agreed upon applying this (problematic) block to disk. Ideally, we could detect this situation ahead of time, and avoid agreeing on a block which would violate the totals predicate. In this PR, we've moving the totals calculation as the last step of the delta state calculation. Few thoughts: - From security perspective, it's just as secured, since the StateDeltas are always in-process and never shared across the network. - From agreement perspective - in case we have a bug in the evaluator that would cause it to generate a state deltas that would violate the totals predicate, the evaluator is going to fail assembling the state deltas on the generator, validator and applicator ( i.e. "consumer" ). Ideally, in the future, we would be able to propose the empty block in such case, allowing the problematic transactions to time-out. - From code-flow perspective - this seem to be a much better approach: a failure to store the block to the disk has only one potential outcome - panicking. Having the ability to detect that the state delta is invalid would give us a way out of that. ## Test Plan 1. Unit tests updated. 2. Tested against mainnet : catchup from scratch to force validation of the entire blockchain.
## Summary This PR is to take advantage of arm64's ability to run arm32 binaries. Using this, we are able to build for arm32 on arm64 hosts by running containers with arm32 OS's on them. This is much faster than our qemu set up because it doesn't rely on emulations. The binaries produced with this process work and the arm64/arm32 build times are as fast as the intel builds, reducing our build time by about an hour. ## Test Plan I ran our build pipeline against this branch using arm64 hosts and tested the produced artifacts using emulated cpu's to make sure they still work on their native architectures. Tested manually on raspberry pi 2 model b, and it's working as expected.
Rotated slack token in environment variable and removed the codecov token from Travis file. codecov token is removed because it's not needed for public repos.
## Summary <!-- Explain the goal of this change and what problem it is solving. Format this cleanly so that it may be used for a commit message, as your changes will be squash-merged. --> The test TestRequestBlockBytesErrors has failed, and the reason is unidentifiable. Various fixes are here, including more verbose error reporting to identify the issue when it occurs again. - Add waitgrop to ledger to wait before closing the ledger and blockQ when goroutines are using them. - In universalFetcher, return the error immediately in case of an error. - Proper closing of objects in TestRequestBlockBytesErrors, and more verbose error reporting. The waitGroup is needed for the following situation: in blockService.go listenForCatchupReq When, immediately after calling `bq.handleCatchupReq`, `BlockService.Stop()` is called, `blockQueue` will be `nil` by the time `blockQueue.getEncodedBlockCert` is called. ``` 0 0x0000000004e16846 in github.com/algorand/go-algorand/ledger.(*blockQueue).checkEntry at /Users/shantkarakashian/go/src/github.com/algorand/go-algorand/ledger/blockqueue.go:215 1 0x0000000004e174a9 in github.com/algorand/go-algorand/ledger.(*blockQueue).getEncodedBlockCert at /Users/shantkarakashian/go/src/github.com/algorand/go-algorand/ledger/blockqueue.go:295 2 0x0000000004e3a2a7 in github.com/algorand/go-algorand/ledger.(*Ledger).EncodedBlockCert at /Users/shantkarakashian/go/src/github.com/algorand/go-algorand/ledger/ledger.go:516 3 0x0000000004f46eed in github.com/algorand/go-algorand/rpcs.topicBlockBytes at /Users/shantkarakashian/go/src/github.com/algorand/go-algorand/rpcs/blockService.go:355 4 0x0000000004f45e1b in github.com/algorand/go-algorand/rpcs.(*BlockService).handleCatchupReq at /Users/shantkarakashian/go/src/github.com/algorand/go-algorand/rpcs/blockService.go:299 5 0x0000000004f45318 in github.com/algorand/go-algorand/rpcs.(*BlockService).listenForCatchupReq at /Users/shantkarakashian/go/src/github.com/algorand/go-algorand/rpcs/blockService.go:245 ``` ## Test Plan Test is enhanced. <!-- How did you test these changes? Please provide the exact scenarios you tested in as much detail as possible including commands, output and rationale. -->
Remove time ledger tracker as not used.
fix a typo.
Summary In the EvalForIndexer, we don't want to call finalValidation(). This would skip the calculation of the account totals in the state delta, which is a serious issue if it were to be used by algod, but it's perfectly fine for the indexer since it doesn't track any totals and therefore cannot calculate the new totals. Test Plan Test against the indexer.
The evaluator shouldn't ask the ledger who is the creator of an asset multiple times. This PR adds a cache for ledger's responses in cow base.
Timing issues between writing to and reading from the database cause the test to occasionally fail. Make the test more tolerant to such timing issues by polling.
Encoding/Decoding method for ABI Values. Serialization/Deserialization of ABI Types. Make/Get methods for ABI Types/Values. Basic testcases for Encoding/Decoding.
* moving logs e2e to scripts
AVM 1.0 introduces application accounts goal clerk needs to consider this account when creates dryrun request object Added a new option --dryrun-accounts for adding accounts that were rekeyed to an app account (or any other accounts).
go-algorand 3.0.1-stable
Similar to app-accounts test, but tests all sorts of asset txns instead of pays.
Made it nice to write e2e test in python by creating a "goal" module that emulates the convenience of goal, but in python so that structured results come out for easy asserts.
* Enable cost and fee pooling in tealdbg * CR fixes
Fixes found during recent cluster bandwidth testing
## Summary The plan to change the tracker database to maintain the latest 320 rounds for the online accounts only have some other required modification; one of them is the semantics of the `Totals` method: - At this time, it supports retrieving `AccountTotals` for any of the recent 320 rounds. - We need the method to support the `AccountTotals` for the latest round only, as well as provide the circulating supply for latest-320 rounds ( i.e. the circulating supply is a subset of the `AccountTotals` ). Once the database implementation is complete, the database would contain the `AccountTotals` information for the latest round only, plus the online circulation for the past 320 rounds. In order to support that, I've broken up the interface into: - `LatestTotals`, which would return the latest totals as well as the latest round associated with these totals. - `OnlineTotals`, which receive a round number and return the online totals associated with that round. This change is focused around the Ledger interface. Further changes would be required in the accounts update before the high-level goals could be achieved. ## Test Plan Unit tests updated.
Release3.0.1 mergeback manual
lower acct bal so it receives no reward
Return a bit more details on eval failure for app calls.
Start work on Teal 6 (AVM 1.1) vFuture, LogicVersion changed to 6 Support RekeyTo field, and Keyreg transactions
## Summary This PR implements the transaction sync 2.0. More details can be found in the design document and the feature presentation . ## Test Plan Unit tests, e2e tests and performance tests were executed against this branch successfully. ## Reviewer notes This PR is pretty large. Consider reviewing it in browsers other than Safari.
Codecov Report
@@ Coverage Diff @@
## rel/beta #2995 +/- ##
============================================
- Coverage 47.32% 43.84% -3.48%
============================================
Files 355 385 +30
Lines 57182 86309 +29127
============================================
+ Hits 27059 37845 +10786
- Misses 27069 42510 +15441
- Partials 3054 5954 +2900
Continue to review full report at Codecov.
|
algojack
requested review from
tsachiherman,
algobarb,
bricerisingalgorand,
egieseke and
onetechnical
…cleci nightly testing. (#3016) For CircleCI testing, increase machine size from medium to large and set parallelism to 4.
Our travisci keeps breaking for different reasons so we decided to move our deploy to circleci.
tsachizehub
approved these changes
Oct 14, 2021
tsachiherman
approved these changes
Oct 14, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Highlights
In this release we have added transaction sync 2.0 and started initial work on AVM 1.1. We also added a few other features and lots of cleanup/test fixes. See below for more details.
Changelog
New Features
Enhancements
Fixes
Protocol Upgrade
This release does not contain a protocol upgrade.
Additional Resources