-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide ASIC resistant PoW scheme for BSQ swaps #5858
Commits on Nov 21, 2021
-
Code cleanup: remove unused PoW class & test
Remove (possible draft) 'ProofOfWorkService(Test)', which is a near duplicate of the class 'HashCashService' but is currently unused.
Configuration menu - View commit details
-
Copy full SHA for 5322049 - Browse repository at this point
Copy the full SHA 5322049View commit details -
Code cleanup: replace (Bi)Function<..,Boolean> with (Bi)Predicate<..>
Replace 'BiFunction<T, U, Boolean>' with the primitive specialisation 'BiPredicate<T, U>' in HashCashService & FilterManager. As part of this, replace similar predicate constructs found elsewhere. NOTE: This touches the DAO packages (trivially @ VoteResultService).
Configuration menu - View commit details
-
Copy full SHA for 5da8df2 - Browse repository at this point
Copy the full SHA 5da8df2View commit details
Commits on Nov 23, 2021
-
Add Equihash implementation for use in ASIC-resistant PoW
Implement the Equihash (https://eprint.iacr.org/2015/946.pdf) algorithm for solving/verifying memory-hard client-puzzles/proof-of-work problems for ASIC-resistant DoS attack protection. The scheme is asymmetric, so that even though solving a puzzle is slow and memory-intensive, needing 100's of kB to MB's of memory, the solution verification is instant. Instead of a single 64-bit counter/nonce, as in the case of Hashcash, Equihash solutions are larger objects ranging from 10's of bytes to a few kB, depending on the puzzle parameters used. These need to be stored in entirety, in the proof-of-work field of each offer payload. Include logic for fine-grained difficulty control in Equihash with a double-precision floating point number. This is based on lexicographic comparison with a target hash, like in Bitcoin, instead of just counting the number of leading zeros of a hash. The code is unused at present. Also add some simple unit tests.
Configuration menu - View commit details
-
Copy full SHA for c2b3a07 - Browse repository at this point
Copy the full SHA c2b3a07View commit details -
Add Equihash.IntListMultimap (private) class for speedup
Provide a (vastly cut down) drop-in replacement for the Guava multimap instance 'indexMultimap', of type 'ListMultimap<Integer, Integer>', used to map table row indices to block values, to detect collisions at a given block position (that is, in a given table column). The replacement stores (multi-)mappings from ints to ints in a flat int- array, only spilling over to a ListMultimap if there are more than 4 values added for a given key. This vastly reduces the amount of boxing and memory usage when running 'Equihash::findCollisions' to build up the next table as part of Wagner's algorithm.
Configuration menu - View commit details
-
Copy full SHA for a5f5a55 - Browse repository at this point
Copy the full SHA a5f5a55View commit details -
Further Equihash optimisation: avoid lambda+stream in tight loop
Manually iterate over colliding table rows using a while- loop and a custom 'PrimitiveIterator.OfInt' implementation, instead of a foreach lambda called on an IntStream, in 'Equihash::findCollisions'. Profiling shows that this results in a slight speedup.
Configuration menu - View commit details
-
Copy full SHA for 6bb0f27 - Browse repository at this point
Copy the full SHA 6bb0f27View commit details -
Further Equihash optimisation: partial parallelisation
Run the initial XorTable fillup in 'Equihash::computeAllHashes' in parallel, using a parallel stream, to get an easy speed up. (The solver spends about half its time computing BLAKE2b hashes before iteratively building tables of partial collisions using 'Equihash::findCollisions'.) As part of this, replace the use of 'java.nio.ByteBuffer' array wrapping in 'Utilities::(bytesToIntsBE|intsToBytesBE)' with manual for-loops, as profiling reveals an unexpected bottleneck in the former when used in a multithreaded setting. (Lock contention somewhere in unsafe code?)
Configuration menu - View commit details
-
Copy full SHA for 52f4981 - Browse repository at this point
Copy the full SHA 52f4981View commit details
Commits on Nov 24, 2021
-
Add PoW version(-list) fields to ProofOfWork & Filter
Add a numeric version field to the 'ProofOfWork' protobuf object, along with a list of allowed version numbers, 'enabled_pow_versions', to the filter. The versions are taken to be in order of preference from most to least preferred when creating a PoW, with an empty list signifying use of the default algorithm only (that is, version 0: Hashcash). An explicit list is used instead of an upper & lower version bound, in case a new PoW algorithm (or changed algorithm params) turns out to provide worse resistance than an earlier version. (The fields are unused for now, to be enabled in a later commit.)
Configuration menu - View commit details
-
Copy full SHA for d8d8ec3 - Browse repository at this point
Copy the full SHA d8d8ec3View commit details
Commits on Nov 25, 2021
-
Add difficulty adjustment & benchmarking to Equihash(Test)
Provide a utility method, 'Equihash::adjustDifficulty', to linearise and normalise the expected time taken to solve a puzzle, as a function of the provided difficulty, by taking into account the fact that there could be 0, 1, 2 or more puzzle solutions for any given nonce. (Wagner's algorithm is supposed to give 2 solutions on average, but the observed number is fewer, possibly due to duplicate removal.) For tractability, assume that the solution count has a Poisson distribution, which seems to have good agreement with the tests. Also add some (disabled) benchmarks to EquihashTest. These reveal an Equihash-90-5 solution time of ~146ms per puzzle per unit difficulty on a Core i3 laptop, with a verification time of ~50 microseconds.
Configuration menu - View commit details
-
Copy full SHA for cb7481d - Browse repository at this point
Copy the full SHA cb7481dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 647fc86 - Browse repository at this point
Copy the full SHA 647fc86View commit details -
Add+enable service to mint/verify Equihash proofs of work
Add an abstract base class, 'ProofOfWorkService', for the existing PoW implementation 'HashCashService' and a new 'EquihashProofOfWorkService' PoW implementation based on Equihash-90-5 (which has 72 byte solutions & 5-10 MB peak memory usage). Since the current 'ProofOfWork' protobuf object only provides a 64-bit counter field to hold the puzzle solution (as that is all Hashcash requires), repurpose the 'payload' field to hold the Equihash puzzle solution bytes, with the 'challenge' field equal to the puzzle seed: the SHA256 hash of the offerId & makerAddress. Use a difficulty scale factor of 3e-5 (derived from benchmarking) to try to make the average Hashcash & Equihash puzzle solution times roughly equal for any given log-difficulty/numLeadingZeros integer chosen in the filter. NOTE: An empty enabled-version-list in the filter defaults to Hashcash (= version 0) only. The new Equihash-90-5 PoW scheme is version 1.
Configuration menu - View commit details
-
Copy full SHA for 0c94e23 - Browse repository at this point
Copy the full SHA 0c94e23View commit details
Commits on Nov 26, 2021
-
Store difficulty as floating point in Filter & PoW
Change the type of the 'difficulty' field in the Filter & ProofOfWork proto objects from int32/bytes to double and make it use a linear scale, in place of the original logarithmic scale which counts the (effective) number of required zeros. This allows fine-grained difficulty control for Equihash, though for Hashcash it simply rounds up to the nearest power of 2 internally. NOTE: This is a breaking change to PoW & filter serialisation (unlike the earlier PR commits), as the proto field version nums aren't updated.
Configuration menu - View commit details
-
Copy full SHA for e0595aa - Browse repository at this point
Copy the full SHA e0595aaView commit details
Commits on Nov 27, 2021
-
Inline predicates to simplify HashCashService & FilterManager
Remove all the 'challengeValidation', 'difficultyValidation' and 'testDifficulty' BiPredicate method params from 'HashCashService' & 'ProofOfWorkService', to simplify the API. These were originally included to aid testing, but turned out to be unnecessary. Patches committed on behalf of @chimp1984.
Configuration menu - View commit details
-
Copy full SHA for e383edc - Browse repository at this point
Copy the full SHA e383edcView commit details
Commits on Nov 30, 2021
-
Fix bug in Equihash.IntListMultimap & adjust constants
Fix a trivial bug in the iterator returned by 'IntListMultimap::get', caused by mistaken use of the iterator index in place of the key when doing lookups into the overspill map. This was causing puzzle solutions to be invalid about 3% of the time, as well as substantially reducing the average number of solutions found per nonce. As the fix increases the mean solution count per nonce to the correct value of 2.0 predicted by the paper (regardless of puzzle params k & n), inline the affected constants to simplify 'Equihash::adjustDifficulty'.
Configuration menu - View commit details
-
Copy full SHA for 0ee6175 - Browse repository at this point
Copy the full SHA 0ee6175View commit details
Commits on Dec 6, 2021
-
Code cleanup: proto fields, duplicated expr & null char separator
1. Reorder the PoW fields in the 'Filter' proto by field index, instead of contextually. 2. Deduplicate expression for 'pow' & replace if-block with boolean op to simplify 'FilterManager::isProofOfWorkValid'. 3. Avoid slightly confusing use of null char as a separator to prevent hashing collisions in 'EquihashProofOfWorkService::getChallenge'. Use comma separator and escape the 'itemId' & 'ownerId' arguments instead. (based on PR bisq-network#5858 review comments)
Configuration menu - View commit details
-
Copy full SHA for 24d2a7f - Browse repository at this point
Copy the full SHA 24d2a7fView commit details
Commits on Dec 7, 2021
-
Add separate ProofOfWork.solution proto field for Equihash
Avoid repurposing the 'ProofOfWork.payload' field for Equihash puzzle solutions, as that may be of later use in interactive PoW schemes such as P2P network DoS protection (where the challenge may be a random nonce instead of derived from the offer ID). Instead, make the payload the UTF-8 bytes of the offer ID, just as with Hashcash. Also, make the puzzle seed the SHA-256 hash of the payload concatenated with the challenge, instead of just the 256-bit challenge on its own, so that the PoW is tied to a particular payload and cannot be reused for other payloads in the case of future randomly chosen challenges.
Configuration menu - View commit details
-
Copy full SHA for db6025a - Browse repository at this point
Copy the full SHA db6025aView commit details