Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model-based Testing With Quickcheck-Dynamic #391

Merged
75 commits merged into from
Jun 28, 2022
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
b5ea26b
[wip] Write a simple QD model for HeadLogic
abailly Jun 2, 2022
99caa35
[wip] Making some progress on model, now trygin to execute test
abailly Jun 2, 2022
2a00ca1
WIP: make StateModel play nicely with IOSim
MaximilianAlgehed Jun 3, 2022
ba6136f
WIP: fixing shrinking so that it does something (not ideal at the mom…
MaximilianAlgehed Jun 3, 2022
1a796a8
WIP: unsafeCoerce hack to get arund QuickCheck's PropertyM being a ta…
MaximilianAlgehed Jun 3, 2022
754d70f
Try to get rid of some Typeable constraints but still having troubles
abailly Jun 3, 2022
6d7a16e
Fix Model so that it can run in hspec :tada:
abailly Jun 7, 2022
7718d00
Add Commits to the model
abailly Jun 7, 2022
cbc36ed
Add Commit and Abort actions
abailly Jun 7, 2022
41473c0
Wait for Init to perform Commit
abailly Jun 8, 2022
d340bec
Write first assertion on the model: assert commits consistency.
KtorZ Jun 8, 2022
b305048
Correctly compare node's state with model
abailly Jun 8, 2022
139da3a
Improve error message on failed assertion
abailly Jun 8, 2022
1cb03f8
Fix recording of server outputs in TestHydraNode
KtorZ Jun 8, 2022
a5495d5
Move state-model assertion in separate function
KtorZ Jun 8, 2022
1d71b90
Assert Head is opened with the right commits
abailly Jun 8, 2022
e741258
Try labelling transitions to enrich the model output.
KtorZ Jun 8, 2022
1c5fc8c
Add more labels for transitions and generate more tests
abailly Jun 8, 2022
e3d12fd
Simplify testing machinery for StateModel
abailly Jun 8, 2022
63a9606
Unify Model to have a single consensus state
abailly Jun 8, 2022
07f1290
Improvements to model-based test output
abailly Jun 8, 2022
6b65a48
Simplify Action type as a wrapper around ClientInput
abailly Jun 9, 2022
9dd5959
Reject transition by default
abailly Jun 9, 2022
d7a387c
Work towards generating transaction on Open
KtorZ Jun 9, 2022
a98d6ea
[wip] Implement perform on NewTx
abailly Jun 9, 2022
9ffe210
Rework model to use abstract representation of Ledger/UTxO
KtorZ Jun 9, 2022
2ad0997
[wip] Update generator and nextState to new tx model
abailly Jun 9, 2022
f3ec900
Continue plumbing mock payment model into real world.
KtorZ Jun 9, 2022
ace8fd8
Implement missing typeclasses for Payment
abailly Jun 9, 2022
c494bb2
Implement concrete perform action for 'NewTx'.
KtorZ Jun 9, 2022
e465a67
Wait for UTxO to be non-empty before performing a new transaction.
KtorZ Jun 17, 2022
4a89deb
Add safety pre-condition to 'NewTx'
KtorZ Jun 17, 2022
6321e65
Add some tracing to debug infinite loop.
KtorZ Jun 17, 2022
fed99ef
Wait for transacation to be confirmed after performing NewTx.
KtorZ Jun 17, 2022
34ce355
Add source of failure in CommandFailed output
abailly Jun 17, 2022
8708f76
Wait for head to be open before performing new tx.
KtorZ Jun 17, 2022
5e9bf31
Fix new-tx pre-condition: make sure value is correct.
KtorZ Jun 17, 2022
3a4abc3
Compare map of aggregated addr->values instead of list of txout
KtorZ Jun 17, 2022
ec7c0d9
Check equality of balances after some txs
abailly Jun 17, 2022
2465a85
Capture logs of TesthydraNode in TVar.
KtorZ Jun 17, 2022
7f3ea14
Dump node logs when ModelSpec tests errors out
abailly Jun 17, 2022
f214db5
Adapt TUI to changed output
abailly Jun 18, 2022
1207d4d
[wip] Track snapshot production to understand Model property failure
abailly Jun 18, 2022
a337fd7
[wip] Introduce NewSn event to ensure event queue is filled
abailly Jun 18, 2022
0239d36
Check there's always a leader for every snapshot number
abailly Jun 18, 2022
f4f4c4a
Filter outputs with empty values in model
abailly Jun 18, 2022
a036cd9
Make Model similar to UTxO
abailly Jun 18, 2022
b0f669f
Improve reporting of error in case no UTxO is available
abailly Jun 18, 2022
f0d8248
Do not throw error when there's no UTxO
abailly Jun 19, 2022
d2834ce
Dump logs captured in IOSim when property fails
abailly Jun 19, 2022
f001e17
Custom show instance for Payment to improve error reporting
abailly Jun 20, 2022
e8eda0c
Revert "[wip] Track snapshot production to understand Model property …
abailly Jun 21, 2022
132edcc
Start fleshing out conflict-free liveness property
abailly Jun 22, 2022
57bc30c
Ensure we don't generate 2 parties with identical keys
abailly Jun 22, 2022
2edf6e5
[wip] debugging
abailly Jun 22, 2022
cd116f7
Use type alias for CardanoSigningKey
abailly Jun 22, 2022
93fc6fb
Adjust wait time for UTxO to appear
abailly Jun 23, 2022
e265dab
Move "unit" test as an example in the ModelSpec doc
abailly Jun 23, 2022
30c2d46
Remove empty quickcheck directory artifact (former submodule?)
ch1bo Jun 23, 2022
08c5735
Fix quickcheck-dynamic package reference to remote one
abailly Jun 23, 2022
9001d23
Rename property checking balances
abailly Jun 23, 2022
4b47bf9
Rephrase liveness property
abailly Jun 23, 2022
89cf064
genPayment instead of genNewTx (an Action)
ch1bo Jun 23, 2022
74b7d2a
Draft a property using the DynamicLogic
ch1bo Jun 23, 2022
324448f
Use a branched quickcheck-dynamic to make it compile
ch1bo Jun 23, 2022
c27dcce
Make Model-based tests pass
abailly Jun 23, 2022
99d0600
Missing golden files
abailly Jun 23, 2022
360ebbf
Clean-up and document Model
abailly Jun 23, 2022
af3d6f7
Remove dynamic logic stuff
abailly Jun 23, 2022
54ff20a
Remove Ord instance for SigningKey
abailly Jun 23, 2022
4ca43bb
Update YAML for logs following change in API
abailly Jun 23, 2022
db7a68e
Rename performs -> sendsInput
abailly Jun 24, 2022
7153cc5
Make printTrace more generic
abailly Jun 24, 2022
e0e87f6
Dedup parties list when generating HeadParameters
abailly Jun 28, 2022
1161107
Fix API schema for CommandFailed server output
abailly Jun 28, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion hydra-node/json-schemas/api.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -534,15 +534,18 @@ components:
CommandFailed:
title: CommandFailed
description: |
Emitted by the server when a well-formed client input was not processable. For example, if trying to close a non opened head or, when trying to commit after having already comitted.
Emitted by the server when a well-formed client input was not processable. For example, if trying to close a non opened head or, when trying to commit after having already committed.
payload:
type: object
required:
- tag
- clientInput
properties:
tag:
type: string
enum: ["CommandFailed"]
clientInput:
$ref: "#/components/schemas/ClientInput"

########
#
Expand Down
4 changes: 0 additions & 4 deletions hydra-node/src/Hydra/Crypto.hs
Original file line number Diff line number Diff line change
Expand Up @@ -56,10 +56,6 @@ type SignAlg = Ed25519DSIGN
newtype SigningKey = HydraSigningKey (SignKeyDSIGN SignAlg)
deriving (Eq, Show)

instance Ord SigningKey where
k `compare` k' =
serialiseSigningKeyToRawBytes k `compare` serialiseSigningKeyToRawBytes k'

instance Arbitrary SigningKey where
arbitrary = generateSigningKey <$> arbitrary

Expand Down
2 changes: 1 addition & 1 deletion hydra-node/src/Hydra/ServerOutput.hs
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ instance IsTx tx => Arbitrary (ServerOutput tx) where
ReadyToFanout -> []
HeadIsFinalized u -> HeadIsFinalized <$> shrink u
HeadIsAborted u -> HeadIsAborted <$> shrink u
CommandFailed _ -> []
CommandFailed i -> CommandFailed <$> shrink i
TxSeen tx -> TxSeen <$> shrink tx
TxValid tx -> TxValid <$> shrink tx
TxInvalid u tx err -> TxInvalid <$> shrink u <*> shrink tx <*> shrink err
Expand Down
26 changes: 13 additions & 13 deletions hydra-node/test/Hydra/Model.hs
Original file line number Diff line number Diff line change
Expand Up @@ -393,17 +393,17 @@ instance
Input.NewTx{Input.transaction = tx} ->
performNewTx st party tx
Input.Init{Input.contestationPeriod = p} ->
party `performs` Input.Init{Input.contestationPeriod = p}
party `sendsInput` Input.Init{Input.contestationPeriod = p}
Input.Abort -> do
party `performs` Input.Abort
party `sendsInput` Input.Abort
Input.GetUTxO -> do
party `performs` Input.GetUTxO
party `sendsInput` Input.GetUTxO
Input.Close -> do
party `performs` Input.Close
party `sendsInput` Input.Close
Input.Contest -> do
party `performs` Input.Contest
party `sendsInput` Input.Contest
Input.Fanout -> do
party `performs` Input.Fanout
party `sendsInput` Input.Fanout

monitoring (s, s') _action _lookup _return =
case (hydraState s, hydraState s') of
Expand All @@ -418,8 +418,8 @@ deriving instance Eq (Action (WorldState m) a)

--

performs :: Monad m => Party -> ClientInput Tx -> StateT (Nodes m) m ()
performs party command = do
sendsInput :: Monad m => Party -> ClientInput Tx -> StateT (Nodes m) m ()
sendsInput party command = do
nodes <- gets nodes
case Map.lookup party nodes of
Nothing -> error $ "unexpected party " <> Hydra.Prelude.show party
Expand Down Expand Up @@ -468,7 +468,7 @@ performCommit party utxo = do
, let vk = getVerificationKey sk
, let txOut = TxOut (mkVkAddress testNetworkId vk) val TxOutDatumNone
]
party `performs` Input.Commit{Input.utxo = realUtxo}
party `sendsInput` Input.Commit{Input.utxo = realUtxo}

performNewTx ::
(MonadThrow m, MonadAsync m, MonadTimer m) =>
Expand All @@ -490,7 +490,7 @@ performNewTx st party tx = do
Just{} -> pure ()
waitForOpen

party `performs` Input.GetUTxO
party `sendsInput` Input.GetUTxO

let matchPayment p@(_, txOut) =
isOwned (from tx) p && value tx == txOutValue txOut
Expand All @@ -510,11 +510,11 @@ performNewTx st party tx = do
lift (threadDelay 1 >> waitForNext (nodes ! party)) >>= \case
GetUTxOResponse u
| u == mempty -> do
party `performs` Input.GetUTxO
party `sendsInput` Input.GetUTxO
waitForUTxO u (n -1)
| otherwise -> case find matchPayment (UTxO.pairs u) of
Nothing -> do
party `performs` Input.GetUTxO
party `sendsInput` Input.GetUTxO
waitForUTxO u (n -1)
Just p -> pure p
_ ->
Expand All @@ -528,7 +528,7 @@ performNewTx st party tx = do
id
(mkSimpleTx (i, o) (recipient, value tx) (from tx))

party `performs` Input.NewTx realTx
party `sendsInput` Input.NewTx realTx
lift $
waitUntilMatch [nodes ! party] $ \case
SnapshotConfirmed{Output.snapshot = snapshot} ->
Expand Down
8 changes: 3 additions & 5 deletions hydra-node/test/Test/Util.hs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ import Control.Monad.IOSim (
import Control.Tracer (Tracer (Tracer))
import qualified Data.Aeson as Aeson
import Data.List (isInfixOf)
import Hydra.Ledger (IsTx)
import Hydra.Ledger.Simple (SimpleTx)
import Hydra.Node (HydraNodeLog)
import Test.HUnit.Lang (FailureReason (ExpectedButGot), HUnitFailure (HUnitFailure))
Expand All @@ -38,14 +37,13 @@ shouldRunInSim action =
throwIO ex
where
tr = runSimTrace action
dumpTrace = say (toString $ printTrace (Proxy :: Proxy SimpleTx) tr)
dumpTrace = say (toString $ printTrace (Proxy :: Proxy (HydraNodeLog SimpleTx)) tr)

-- | Utility function to dump logs given a `SimTrace`.
-- TODO(SN): take a proxy instead of hard-coding HydraNodeLog
printTrace :: forall tx a. IsTx tx => Proxy tx -> SimTrace a -> Text
printTrace :: forall log a. (Typeable log, ToJSON log) => Proxy log -> SimTrace a -> Text
printTrace _ tr =
unlines . map (decodeUtf8 . Aeson.encode) $
selectTraceEventsDynamic' @_ @(HydraNodeLog tx) tr
selectTraceEventsDynamic' @_ @log tr
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍


-- | Lifted variant of Hspec's 'shouldBe'.
shouldBe :: (HasCallStack, MonadThrow m, Eq a, Show a) => a -> a -> m ()
Expand Down