(5/8) Test P2PDataStore expiration code and make testing time sensitive code easier. #3568
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
julianknutsen
changed the title
|
have a look at: #2996 for a similar Clock related proposal. I found this in P2PModule as an example of injecting a clock: At first glance I did not see you use injection to instantiate Clocks, sorry if I missed it. |
|
Thanks for taking a look. I use the same injection that you found. Work was done previously in #3037 to do the injection, it just wasn't actually used yet, so I pushed it across the finish line. |
julianknutsen
commented
Nov 7, 2019
p2p/src/main/java/bisq/network/p2p/storage/payload/ProtectedStorageEntry.java
Outdated
Show resolved
Hide resolved
julianknutsen
changed the title
Fix a bug where remove() was called in the addMailboxData() failure path. 1. Sender's can't remove mailbox entries. Only the receiver can remove it so even if the previous add() failed and left partial state, the remove() can never succeed. 2. Even if the sender could remove, this path used remove() instead of removeMailboxData() so it wouldn't have succeed anyway. This patch cleans up the failure path as well as adds a precondition for the remove() function to ensure future callers don't use them for ProtectedMailboxStorageEntrys.
Add comments and refactor the body in order to make it easier to understand.
Refactor addProtectedStorageEntry for more readability and add comments to help future readers.
Refactor for readability and add comments for future readers.
Refactor for readability and add comments to help future readers.
Refactor for readability and add comments for future readers.
Removed duplicate log messages that are handled inside the various helper methods and print more verbose state useful for debugging. Updated potentially misleading comments around hashing collisions
…icates Now returns false on duplicate sequence numbers. This matches more of the expected behavior for an add() function when the element previously exists. The only callers are either P2PService users that always increment the sequence number or the onMessage() handler which doesn't verify the return so there will be no visible change other than the increased readability of the code and deduplication of the code paths.
Now returns false if the sequence number of the refresh matches the last operation seen for the specified hash. This is a more expected return value when no state change occurs. The only callers are either P2PService users that always increment the sequence number or the onMessage() handler which doesn't verify the return so there will be no visible change other than the increased readability of the code and deduplication of the code paths.
…duplicate sequence #s Remove operations are now only processed if the sequence number is greater than the last operation seen for a specific payload. The only creator of new remove entrys is the P2PService layer that always increments the sequence number. So, this is either left around from a time where removes needed to work with non-incrementing sequence numbers or just a longstanding bug. With the completion of this patch, all operations now require increasing sequence numbers so it should be easier to reason about the behavior in future debugging.
|
I got some small out-of-band requests for style and naming in 5-9. I'll get those cleaned up tomorrow and 5-9 will be ready for final review. Thanks again to everyone for taking the time to look through these. I know it wasn't planned dev or review work and bringing on new devs is more time consuming than normal, but it is appreciated. |
Use the DI Clock object already available in P2PDataStore, instead of calling System.currentTimeMillis() directly. These two functions have the same behavior and switching over allows finer control of time in the tests.
Deduplicate some code in the ProtectedStorageEntry constructors in preparation for passing in a Clock parameter.
Switch from System.currentTimeMills() to Clock.millis() so dependency injection can be used for tests that need finer control of time. This involves attaching a Clock to the resolver so all fromProto methods have one available when they reconstruct a message. This uses the Injector for the APP and a default Clock.systemDefaultZone is used in the manual instantiations. Work was already done in bisq-network#3037 to make this possible. All tests still use the default system clock for now.
Reduces non-deterministic failures of the refreshTTL tests that resulted from the uncontrollable System.currentTimeMillis(). Now, all tests have extremely fine control over the elapsed time between calls which makes the current and future tests much better.
Add tests for removing expired entries and optionally purging the sequence number map. Now possible since these tests have control over time with the ClockFake. The remove validation needed to be improved since deletes through the expire path don't signal HashMap listeners or write sequence numbers.
The original test would take over 5 seconds. Allow tests to set the number of required entries before purge to a lower value so the tests can run faster with the same confidence.
julianknutsen
force-pushed
the
manual-clock-in-tests
branch
from
4b23485
to
454b2d7
Compare
freimair
suggested changes
Nov 15, 2019
freimair
approved these changes
Nov 18, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This new set of patches starts at de72d39 (The rest are just merges of my previous pull requests).
The end goal of this stack is to test the expiration code that runs periodically in P2PDataStore. Since the tests require modification of time and the current objects made use of System.currentTimeMillis() (an unmockable static function) a detour of plumbing and dependency injection was required to get the objects in a state where they could be tested. #3037 looks to have started the pattern in other modules.
The end result is that all tests in P2PDataStore now have fine-grained control of the clock and can easily test the various expiration and purge functions. The periodic expiration code is now fully tested.
With the addition of these tests, the code coverage now stands at 81% line and 70% branch coverage.