Incremental commits off-chain

[noonio]

❄️ This PR implements most of the off-chain changes we need for the incremental commits to work. Since there are many changes we decided to have a series of follow up PRs to finally conclude the wanted feature.

Included:

Relevant code to be able to do full end-to-end test for depositing (and recovering) a user UTxO into a running Head.
This means:
❄️ everything needed to accept user requests at the API level
❄️ build off-chain transactions (deposit, recover, increment)
❄️ observe new transactions (deposit, recover and increment)
❄️ changes in the Snapshot type and all other relevant types and functions to accommodate this new feature
❄️ minimal needed changes in the on-chain code (validators just return True so once this is merged we will not be able to release until this gap is implemented)

Outstanding:

❄️ Observation tests are not working for the increment transaction observation since this is the first time we have a transaction that depends on another one (deposit).
❄️ tutorial and documentation
❄️ Various tests described in the original issue and which are relevant to the off-chain code
❄️ Provide info on recover tx size on each pr (the same as other txs)
❄️ Relevant for the feature overall but not to existing scope: hydra-tx library is not capable of balancing deposit/recover transactions (changes in this pr make this possible by using the hydra-node internal wallet)

---

• CHANGELOG updated or not needed
• Documentation updated or not needed
• Haddocks updated or not needed
• No new TODOs introduced or explained herafter

[github-actions]

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2024-09-25 07:32:19.872650965 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	2fac819a1f4f14e29639d1414220d2a18b6abd6b8e444d88d0dda8ff	3799
νCommit	2043a9f1a685bcf491413a5f139ee42e335157c8c6bc8d9e4018669d	1743
νHead	92eba1d473b3616388b42f80f77469c5d67a10b34b296d73c8f7bd44	10311
μHead	9b12a3f55e523a1d87a6335ef083af78a77249ea233796e518b15f61*	4555
νDeposit	e0091235d453844ca593223db7d93b67b27afff4aac13bda443e0e16	2791

• The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per head.

Init transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5143	5.71	2.25	0.44
2	5344	7.09	2.80	0.46
3	5546	8.65	3.42	0.49
5	5946	11.32	4.48	0.54
10	6951	18.30	7.24	0.66
56	16201	81.76	32.34	1.76

Commit transaction costs

This uses ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	569	10.52	4.15	0.29
2	755	13.86	5.65	0.34
3	947	17.33	7.20	0.38
5	1323	24.65	10.44	0.48
10	2254	45.22	19.36	0.75
20	4123	95.99	40.76	1.40

CollectCom transaction costs

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	560	22.18	8.68	0.42
2	114	671	33.86	13.39	0.55
3	171	782	45.14	18.08	0.68
4	226	897	59.11	23.88	0.84
5	283	1004	72.72	29.67	1.00
6	341	1116	96.64	39.38	1.26

Cost of Decrement Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	628	17.70	7.83	0.38
2	785	19.31	9.24	0.41
3	854	19.23	9.93	0.41
5	1277	25.22	13.83	0.51
10	2034	32.83	20.59	0.65
48	7711	98.99	75.73	1.84

Close transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	678	22.39	9.90	0.43
2	821	24.00	11.39	0.46
3	958	25.56	12.88	0.49
5	1280	28.95	16.13	0.56
10	1983	37.22	23.87	0.71
47	7534	99.53	82.84	1.88

Contest transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	714	28.59	12.15	0.50
2	800	30.06	13.41	0.53
3	1015	32.73	15.65	0.57
5	1256	35.97	18.47	0.63
10	2160	46.86	27.78	0.82
37	5901	98.43	71.79	1.72

Abort transaction costs

There is some variation due to the random mixture of initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5037	17.38	7.56	0.57
2	5101	27.59	12.02	0.68
3	5286	39.07	17.15	0.82
4	5488	59.88	26.60	1.07
5	5571	76.05	33.79	1.25
6	5633	90.85	40.30	1.42

FanOut transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	4981	7.53	3.18	0.45
5	1	57	5016	9.06	4.06	0.47
5	5	285	5151	13.38	6.82	0.53
5	10	571	5323	19.32	10.50	0.62
5	20	1139	5660	30.56	17.58	0.77
5	30	1704	5997	41.88	24.70	0.93
5	40	2278	6341	53.01	31.75	1.09
5	50	2846	6679	64.34	38.87	1.24
5	81	4607	7728	99.51	61.00	1.73

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2024-09-25 07:34:41.458971501 UTC

Baseline Scenario

Number of nodes	1
Number of txs	3000
Avg. Confirmation Time (ms)	4.969751595
P99	13.665000919999954ms
P95	5.637473149999998ms
P50	4.102304ms
Number of Invalid txs	0

Three local nodes

Number of nodes	3
Number of txs	9000
Avg. Confirmation Time (ms)	22.859893192
P99	112.0312904700001ms
P95	29.56238525ms
P50	20.480520499999997ms
Number of Invalid txs	0

[ch1bo]

@noonio When we continue work here, we should take a step back again given the new overall design of #199 with details from #1571. The overall off-chain story has not changed much, but is basically separated in two parts:

• Part 1: hydra-node API for deposit/cancel analogous to Implement deposit validators and off-chain code #1574
• Part 2: hydra-node detects deposits, triggers ReqSn with approval and incrementTx construction & observation (as before)

[github-actions]

Test Results

513 tests  +10   507 ✅ +10   21m 45s ⏱️ + 1m 8s
162 suites + 2     6 💤 ± 0 
  7 files   ± 0     0 ❌ ± 0 

Results for commit 900c96d. ± Comparison against base commit d5729d3.

♻️ This comment has been updated with latest results.

[ffakenz]

alphaUTxOHash is not expected, we should be able to use 𝜂′Δ only.

[ffakenz]

There is no m in the spec.

This should be a tx input: 𝜙deposit

[ffakenz]

actually, at this moment, we can also removed this type completely.

[ffakenz]

this error seems not to be used.

otoh, we cannot tell during fannout if the hash corresponds to a decrement/increment.