go-algorand 3.2.0-beta #3231
Merged
go-algorand 3.2.0-beta #3231
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
itxn_next to give us inner groups Questions remain around how TxID and GroupID should be defined for inners.
… branch. (#3017) ## Summary Disable the participation key refresh on the TestPartitionHalfOffline test. The test was failing with the following threads running: ``` algod(20739) : github.com/algorand/go-algorand/agreement.(*asyncPseudonode).loadRoundParticipationKeys(0xc0033dad20, 0x2, 0xc0039c8180, 0xc000530000, 0xc000dcb0d0) ``` ``` algod(20739) : github.com/algorand/go-algorand/data/account.PersistedParticipation.DeleteOldKeys(0x92c6b21bbd8fefac, 0xed2d99ce769768ea, 0xac85a3b9466cead1, 0xc32937d7e4c532f9, 0xc000043260, 0xc0002b04b0, 0x0, 0x2dc6c0, 0x2710, 0xc0001892c0, ...) ``` ``` algod(20739) : github.com/algorand/go-algorand/data.(*AccountManager).AddParticipation(0xc0001191a0, 0xd175b5d042fa1c39, 0x2140fe15c6a19653, 0xc66f7b9c58cf5ed1, 0x107d657815b44d49, 0xc0037a4d20, 0xc0002b0690, 0x0, 0x2dc6c0, 0x2710, ...) ``` By avoiding the participation key reload ( which isn't needed for this test ), I'm hoping to avoid the issue. ## Test Plan This is a test.
## Summary Resolves #2738 Adds the following: - consensus.go: Adds the default MaxExpiredAccountsToProcess value for the new consensus protocol - block.go: Adds necessary block header entries - eval.go: Scans for expired accounts, modifies them to be offline and adds validation for this use case - eval_test.go: Basic unit tests - participationExpiration_test.go: Added e2e tests that verify that different consensus protocols behave differently ## Test Plan Unit test and e2e tests added
## Summary rebase Tolik's PR on master; this PR add the ability for the indexer to preload account data so that it would be used by the evaluator. ## Test Plan Existing unit tests updated.
## Summary * Moved code from **acctupdates.go** to **trackerdb.go**. * Extracted db creation and init from`accountsInitialize` to `trackerDBInitialize` * `reloadLedger` now calls `trackerDBInitialize` right after block db init * `reloadLedger` also has post-creation actions like vacuuming Limitations: 1. Tables creation still calls `accountsInit` for initial tables creation and inserting genesis accounts. 2. Migration code still have some business logic. ## Test Plan Use existing test suite
…cleci nightly testing. (#3016) For CircleCI testing, increase machine size from medium to large and set parallelism to 4.
In testing.T, Error, Errorf, and Fail do not terminate the test. Error and Errorf call Fail. There will be another call to FailNow to terminate the test. (FailNow also calls Fail.) When `dontReportFailures` is set in Error, Errorf, and Fail, the test will not be terminated. Moreover, the subsequent call to FailNow will be ignored because of the set dontReportFailures flag. In this change, dontReportFailures will not be set for Error, Errorf, and Fail so that the subsequent FailNow will terminate the test.
## Summary This PR reorganize the files inside the ledger package by move the evaluator related files into its own `internal` package. The files in the internal package cannot access the root ledger files, and therefore using the shared `ledgercore` as a place to share interfaces. ``` ledger/ ├── apply/ ├── internal/ ├── ledgercore/ └── testing/ ``` ## Test Plan use existing test, and update existing tests.
Summary This test has a special handling is case the protocol upgrade completes before we can validate that without an upgrade the rekey feature won't work. However, when that does happen, AccountA is already rekeyed, and would require a different handling. To avoid that while retaining the execution of the test, I've moved the second part to use a different set of accounts. Test Plan This is a test
## Summary This PR addresses a regression introduced in #2983. The culprit is that an interface might contain a nil pointer, which makes it insufficient to test the interface pointer itself. Fail cases: https://app.circleci.com/pipelines/github/algorand/go-algorand/2451/workflows/8807ced7-89ae-4b6b-96b0-1bc5bdf9d84c/jobs/27259/tests#failed-test-0 ## Test Plan Use existing unit testing. In particular TestCatchupOverGossip.
## Summary The TestMultiThreaded was running too slow on Travis machines, causing it to fail due to deadlock detection false-positive. The change in this PR modify the execution time of the test, ensuring that it won't run for more than 5 second ( which should be enough for data racing detection testing ) ## Test Plan This is a test.
## Summary During fast catchup, we restart the transaction sync service very quickly. This can cause a network message being sent, and the response would be returned to the "restarted" txnsync. Since we don't want to disconnect the network connection itself ( which could have some messages enqueued ), the transaction sync would need to store the "returned" channel before sending the message. That would avoid the data race ( and safely ignore the incoming message ). ## Test Plan Use existing testing, and confirm against that.
## Summary move StartEvaluator parameters into an object. Given that most of the time the parameters are using a "default" state, it reduce the number of passed parameters, improving the caller readability. ## Test Plan Unit tests updated.
Fix the test TestReproducibleCatchpointLabels, by provide proper calculation of the account totals.
Due to Travis not being able to handle our builds anymore, we are removing travis integration tests.
…n values (#3041) ## Summary The e2e test `TestPartitionHalfOffline` was failing due to the deadlock detection failing while the old keys are being deleted. This change would reduce the key dilution, and should shorten the duration it takes to generate a new batch. ## Test Plan This is a test.
## Summary The database handle is not closed in this code path. In normal situations, this passes unnoticed, since it will be closed once the test terminates. However, when running the test multiple times, the OS will complain about too many open files. ## Test Plan This is a test infrastructure fix.
Fix global-state-schema key name in REST API spec
## Summary 1. Add a new `OnlineAccountData` data type 2. Implement `LookupAgreement` ## Test Plan Fixed existed tests: added `LookupAgreement` to mocked ledgers
## Summary Use LookupAgreement in AlgorandFullNode.VotingKeys() Post #3046 fixes.
## Summary The test had a conceptual buggy pattern. It was using ```golang round, err = client.CurrentRound() ... _, err = client.BroadcastTransaction(signedTxn) ... _, err = client.WaitForRound(round + 3) ``` which is doomed to fail if the test process goes to sleep for a minute between the `client.CurrentRound()` call and the `client.BroadcastTransaction` call. The trivial solution is to reorder the calls so that ```golang _, err = client.BroadcastTransaction(signedTxn) ... round, err = client.CurrentRound() ... _, err = client.WaitForRound(round + 3) ``` ## Test Plan This is a test.
Summary Adds additional resilience to participation expiration test TestParticipationAccountsExpirationFuture. Test Plan This is a test.
## Summary Historically only `account updates` tracker was committing into DB. The PR changes this: 1. `dbRound` management and `commitSyncer` business moved to trackerRegistry. 2. `account updates` is used as a driver for committing tasks scheduling - by given `commttedRound` and `dbRound` it figures out offset in detlas for saving to the disk. This is basically old `au.committedUpTo` method`. 3. `commitRound` replaced by `prepareCommit` that returns commit and post commit callbacks 4. trackerRegistry has own `commitRound` that calls trackers' `prepareCommit` and commit/post commit procs if any. 5. `account updates` still have a cached dbRound to be used in Lookup methods. ## Test Plan It is refactoring, use existing tests
Our travisci keeps breaking for different reasons so we decided to move our deploy to circleci.
## Summary This PR eliminate the `totalsNewRounds`, which perform the exact same calculation previously performed by the block evaluator. ## Test Plan - [x] Unit tests updates. - [x] Catchpoint label testing : starting a node catching up to 16,800,000 and the correct catchpoint label was observed on round 16,810,000.
Noticed our environment passing variable is not working as intended, implemented a different way of passing it.
## Summary When a 2.1 (i.e. no txsync) client connects to a 3.0 relay (i.e. txsync), the relay needs to request the client to keep sending it a TX messages - otherwise, these transactions would not get propagated. The 2.1 and 3.0 above are network protocol versions, not a algod release version. ## Test Plan e2e test added.
Fixing naming due to new way of displaying CircleCI workflow.
## Summary This PR rolls back the txsync, retaining only the config file changes
## Summary Going to ledger directly prevents caching of accounts. I'm looking into poor indexer import performance on testnet at rounds > 17M, and I noticed that one account is looked up a lot of times in same round.
## Summary Extract catchpoint logic out of the accounts update tracker. ## Test Plan Refactor existing unit tests.
* Add error strings to v2 TEAL compile endpoint * Move number of errors to end so it is consistent with goal clerk compile * Refactor ReportProblems() to take io.Writer * Don't report number of errors on the API call
* add abi encoding * accord with go-algorand code format * minor modification * add partition test * resolve review, need more testcase rewrite * move from cmd/goal to data, resolve review * rewrite is-dynamic, dynamic array * update dynamic array 0 element support, test needed * minor * minor * minor * add more testcase for abi encode/decode * update comments in encoding test * minor * attempt to split abi.go into 2 files * separate abi files to smaller files * resolve reviews, work on random gen tuple value encode/decode * add random tuple test * remove math-rand, use crypto-rand * minor * minor * some change requested from community * fix for 1 corner case * resolve review comments * resolve review comments * minor * minor * update encode slot capacity * minor * resolve reviews * minor update on bool bytelen calculate * update encode/decode from types * random test remain to be modified * testing variable renaming, encode int support (u)int types * update test scripts and remove value struct * follow golint * partly resolving comments * whoops uint encoding update * update int decode to primitive types method * go fmt * update parseAppArg to accept abi input (attempt) * need to check cmdline arg validity * update unmarshal from JSON in ABI type * unmarshal from json for ABI type * update ABI type unmarshal values from JSON bytes * update ABI methods for string/array/address * update unmarshal from JSON in abi * fix for error in ufixed json unmarshal * fix * update on method sub command * minor * probably better separate abi json to a single file * i just want to add a required flag plz... * minor fix on interface from json * consider some rough test cases * minor * add partition test * update static uint test * update marshal/unmarshal json methods for abi * marshal byte array to b64 string * abi json polish * update golangci lint rules * revert golangci config * update method impl * update method signature return type check * minor * copy-paste code from call app cmd * minor * add method flag to txn flags * minor * update changes * minor * moving helper functions to abi * update comments * update method app call * resolve part in abi impl * add oncomplete support * minor * try to use stringarrayvar * minor * update goal return log handing process * go simple * add a line of e2e test for now * update * minor * minor * minor * go fmt * approval/clear prog nil * discard all changes to e2d-app-cross-round, going to write separately e2e test * update e2d tests * check ret valu * use constant * resolve review partly * resolve review on code reformatting * resolve review on code reformatting, use code chunk for datadir and client * go fmt * export tuple type maker * update comments in e2e test * update filter empty string * resolve issues with JSON abi * minor
* adding missing to opcode docs for external gets * update TEAL_opcodes.md
## Summary * new `ParticipationRegistry` interface designed to manage keys and collect usage metrics. * new REST endpoints for interacting with keys. * improved `goal account partkeyinfo` and `goal account listpartkeys` formatting. ## Test Plan New unit and integration tests.
Integration tests (specifically, e2e-go tests) seem to be having resource issues lately, so this returns integration tests to using large instances (4 cores, ~16GB RAM) after they were downsized to medium (2 cores ~8GB RAM) in #3095. Nightly integration tests are already using large VMs.
## Summary When implementing the catchpoint tracker, I missed the re-initilization location for some of the local variables. This would generate incorrect catchpoint labels after a node completes a fast-catchup. #3085 ## Test Plan - [x] Add unit tests - [x] Perform manual testing
## Summary This PR updates the websocket library to version 1.4.4, which includes a fix for deadlocked writers.
## Summary
This PR addresses two separate issues in the `pseudonodeNode` implementation:
1. `pseudonodeVotesTask.execute` and `pseudonodeProposalsTask.execute ` do not handle vote verification enqueueing failures ( to the execution pool ). This could lead the the pseudonode processing go-routine being stuck.
2. The `pseudonodeVotesTask.execute` could block forever in case the output channel is not being read from.
## Trigger
This issue was detected by the telemetry:
```
({num-occurances}): {telemetry error string}
(67): pseudonode.MakeVotes call failed(attest) pseudonode input channel is full
```
## Test Plan
Few unit tests added. More to come.
Check if the outFilename argument is passed and write to the path specified as other commands
Re-merge go-algorand 3.1.3-beta
Codecov Report
@@ Coverage Diff @@
## rel/beta #3231 +/- ##
============================================
+ Coverage 43.93% 47.43% +3.50%
============================================
Files 386 369 -17
Lines 86471 59538 -26933
============================================
- Hits 37988 28243 -9745
+ Misses 42513 28010 -14503
+ Partials 5970 3285 -2685
Continue to review full report at Codecov.
|
egieseke
requested review from
tsachiherman,
jannotti,
winder,
algobarb,
algojack,
bricerisingalgorand,
onetechnical and
yaovi-a
algobarb
approved these changes
Nov 23, 2021
bricerisingalgorand
approved these changes
Nov 23, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Highlights
Enhancements to Ledger and Smart Contract functionality.
Changelog
Agreement
Algod
systo resolve golang 1.17 local build crash (Updatesysto resolve golang 1.17 local build crash #3157)Catchup
Consensus
DevOps
go-algorand
Goal
Indexer
Ledger
ledger.Creatablepublic (Make fields inledger.Creatablepublic #3125)endOfBlock()(ledger: do final validation inendOfBlock()#3132)roundCowBase: uselookup()instead of going to ledger directly (roundCowBase: uselookup()instead of going to ledger directly #3173)Network
REST API
Teal
Testing
stringfields (Add unit test to avoid newstringfields #3101)Protocol Upgrade
This release does not contain a protocol upgrade.
Additional Resources