Incremental commits off-chain #1541

noonio · 2024-07-31T09:21:02Z

❄️ This PR implements most of the off-chain changes we need for the incremental commits to work. Since there are many changes we decided to have a series of follow up PRs to finally conclude the wanted feature.

Included:

Relevant code to be able to do full end-to-end test for depositing (and recovering) a user UTxO into a running Head.
This means:
❄️ everything needed to accept user requests at the API level
❄️ build off-chain transactions (deposit, recover, increment)
❄️ observe new transactions (deposit, recover and increment)
❄️ changes in the Snapshot type and all other relevant types and functions to accommodate this new feature
❄️ minimal needed changes in the on-chain code (validators just return True so once this is merged we will not be able to release until this gap is implemented)

Outstanding:

❄️ documentation - Write up some documentation to explain how incremental commits are working
❄️ Various tests described in the original issue and which are relevant to the off-chain code
❄️ Make the increment/recover use the observed UTxO instead of keeping those around
❄️ Remove Commit client input since it is not used
❄️ Revisit the observation/transaction posting types and make sure the field names make sense
❄️ Determine what should be good enough deadline for a deposit transaction (should we make it configurable or rely on the contestation deadline)

CHANGELOG updated or not needed
Documentation updated or not needed
Haddocks updated or not needed
No new TODOs introduced or explained herafter

github-actions · 2024-07-31T09:52:31Z

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2024-10-04 14:53:13.624567009 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	2fac819a1f4f14e29639d1414220d2a18b6abd6b8e444d88d0dda8ff	3799
νCommit	2043a9f1a685bcf491413a5f139ee42e335157c8c6bc8d9e4018669d	1743
νHead	04fa9fa7d6f2dac870291a1f2274a268b64401d91d92b4bf1423843b	10176
μHead	41936d9458823cb5b4e47aa843a2f438ff3697d10ef4227a33cbe15b*	4508
νDeposit	e0091235d453844ca593223db7d93b67b27afff4aac13bda443e0e16	2791

The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per head.

`Init` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5096	5.61	2.21	0.44
2	5298	7.18	2.84	0.46
3	5498	8.37	3.30	0.48
5	5902	11.39	4.51	0.53
10	6906	18.02	7.12	0.65
57	16355	83.00	32.84	1.78

`Commit` transaction costs

This uses ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	569	10.52	4.15	0.29
2	759	13.86	5.65	0.34
3	947	17.33	7.20	0.38
5	1314	24.65	10.44	0.48
10	2256	45.22	19.36	0.75
20	4119	95.99	40.76	1.40

`CollectCom` transaction costs

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	560	21.46	8.41	0.41
2	114	671	32.17	12.77	0.53
3	170	782	44.16	17.72	0.67
4	227	893	62.79	25.25	0.88
5	282	1004	77.71	31.50	1.05
6	338	1116	93.56	38.24	1.23

Cost of Decrement Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	646	17.73	7.85	0.38
2	779	19.31	9.24	0.41
3	898	20.67	10.54	0.43
5	1254	24.89	13.71	0.51
10	2052	32.66	20.52	0.65
50	8080	98.06	76.82	1.85

`Close` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	653	20.15	9.09	0.41
2	788	21.54	10.52	0.43
3	1034	23.92	12.68	0.48
5	1234	26.44	15.22	0.53
10	2004	34.35	23.04	0.68
49	7659	95.52	82.85	1.86

`Contest` transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	671	25.93	11.19	0.47
2	815	27.77	12.77	0.50
3	947	29.50	14.31	0.53
5	1428	34.69	18.80	0.63
10	2032	43.14	26.22	0.78
39	6464	98.62	75.05	1.77

`Abort` transaction costs

There is some variation due to the random mixture of initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4972	17.52	7.62	0.57
2	5157	29.28	12.91	0.71
3	5207	41.39	18.25	0.85
4	5331	57.16	25.26	1.03
5	5615	78.89	35.17	1.29
6	5650	97.11	43.26	1.50

`FanOut` transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	4935	7.71	3.26	0.45
5	1	57	4968	8.64	3.89	0.47
5	5	285	5104	13.76	6.99	0.54
5	10	569	5273	19.22	10.46	0.61
5	20	1140	5614	30.34	17.49	0.77
5	30	1709	5955	41.66	24.61	0.93
5	40	2277	6293	53.47	31.94	1.09
5	50	2846	6633	64.12	38.78	1.24
5	81	4605	7679	99.09	60.82	1.73

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2024-10-04 14:55:54.466747739 UTC

Baseline Scenario

Number of nodes	1
Number of txs	3000
Avg. Confirmation Time (ms)	4.946512714
P99	8.004050609999991ms
P95	6.122482349999996ms
P50	4.6671890000000005ms
Number of Invalid txs	0

Three local nodes

Number of nodes	3
Number of txs	9000
Avg. Confirmation Time (ms)	22.970043740
P99	115.73477167ms
P95	29.303824249999998ms
P50	20.578107ms
Number of Invalid txs	0

ch1bo · 2024-08-19T15:15:57Z

@noonio When we continue work here, we should take a step back again given the new overall design of #199 with details from #1571. The overall off-chain story has not changed much, but is basically separated in two parts:

Part 1: hydra-node API for deposit/cancel analogous to Implement deposit validators and off-chain code #1574
Part 2: hydra-node detects deposits, triggers ReqSn with approval and incrementTx construction & observation (as before)

This also enables re-enables observation properties of Deposit/Recover transitions. We note though, that these are not "real" head state transitions and the ctx / state return values are superficial.

This makes it easier to use in combination with observing a deposit.

Remove deposit and recover from the Head tx observation tests and have dedicated tests for deposit/recover.

Make sure to check if the deposit datum actually contains the deposit tx inputs. Remove the `utxo` from `DepositObservation` and propagate this change. Increment observation is broken now since it was depending on this utxo but we need to do it differently any way.

Added TxInType since we needed abstract type for use in head logic. Ideally I would not try to find utxo in the map but deal with txin/id only on increment/recover

If we have this type around it will serve us well for deleting observed deposits and also on recovers.

On increment the UTxO we are incrementing is not part of the head UTxO yet so we need more information in order to be able to observe the relevant transaction.

ch1bo · 2024-10-04T15:40:59Z

CHANGELOG.md

+      - Add documentation to explain the feature
+      - Implement tests scenarios outlined in the [incremental-commit] (https://github.com/cardano-scaling/hydra/issues/199) issue
+      - Remove Commit client input since it is unused
+      - Revisit types related to observations/posting transactions and make sure the fields are named appropriatelly


noonio linked an issue Aug 1, 2024 that may be closed by this pull request

Implement off-chain user journey of incremental commits #1522

Closed

noonio force-pushed the off-chain-incremental-commits branch from 235acca to 273fe91 Compare August 1, 2024 09:11

ch1bo force-pushed the master branch 2 times, most recently from 53b33e9 to 8d8d3a3 Compare August 15, 2024 17:09

v0d1ch force-pushed the off-chain-incremental-commits branch 10 times, most recently from 7ea2e9f to e63ca63 Compare September 5, 2024 10:56

locallycompact force-pushed the off-chain-incremental-commits branch from 8c0728e to a39f11a Compare September 9, 2024 08:18

v0d1ch force-pushed the off-chain-incremental-commits branch 13 times, most recently from 4501071 to 40fe8bc Compare September 12, 2024 11:41

ch1bo and others added 25 commits October 4, 2024 16:49

Define genRecoverTx in terms of genDepositTx

49fde01

This also enables re-enables observation properties of Deposit/Recover transitions. We note though, that these are not "real" head state transitions and the ctx / state return values are superficial.

Change definition of recoverTx and move observeDepositTx to hydra-tx

4e70cfa

This makes it easier to use in combination with observing a deposit.

Move recover observation to hydra-tx

56096d7

Remove deposit and recover from the Head tx observation tests and have dedicated tests for deposit/recover.

Keep Map TxIn locally so we support multiple deposits

f6791e4

Added TxInType since we needed abstract type for use in head logic. Ideally I would not try to find utxo in the map but deal with txin/id only on increment/recover

Add deposit TxIn to Increment and OnIncrementTx

0a9817a

If we have this type around it will serve us well for deleting observed deposits and also on recovers.

Add extra UTxO to make the observations pass

d8d109e

On increment the UTxO we are incrementing is not part of the head UTxO yet so we need more information in order to be able to observe the relevant transaction.

Make recover workflow go through the HeadLogic

bedd0a8

Remove alphaUTxOHash datum field and fields in IncrementRedeemer

14b7e61

Remove unused error constructor

ff6c7c9

Add a test for pending deposits

b13a8ae

Add a test that spends the script into a head

4954793

Implement GET endpoint for pending deposits

c43612a

Make everything compile

a577c9b

Add commit blueprint for deposit workflow

4ddcc56

Use TxId instead of TxIn everywhere (recover/deposit related)

524de66

Fix tests relying on empty utxo hash

5d5ac1f

Make all tests green

4e774ed

Finish up the tutorial

7f9aed0

Delete from pending deposits on successfull recover

7a4756a

Fix a bug in the pending deposit projection

5139946

Fix json schema errors

44f4da7

Make protocol.md true to implementation

a786804

Fix plutus scripts

9bd06ab

Wrap things up

efe7083

v0d1ch force-pushed the off-chain-incremental-commits branch from 60e5e3b to efe7083 Compare October 4, 2024 14:49

v0d1ch disabled auto-merge October 4, 2024 14:53

v0d1ch merged commit 4d48208 into master Oct 4, 2024
28 checks passed

v0d1ch deleted the off-chain-incremental-commits branch October 4, 2024 15:31

ch1bo reviewed Oct 4, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Incremental commits off-chain #1541

Incremental commits off-chain #1541

noonio commented Jul 31, 2024 •

edited by v0d1ch

Loading

github-actions bot commented Jul 31, 2024 •

edited

Loading

ch1bo commented Aug 19, 2024

ch1bo Oct 4, 2024

Incremental commits off-chain #1541

Incremental commits off-chain #1541

Conversation

noonio commented Jul 31, 2024 • edited by v0d1ch Loading

github-actions bot commented Jul 31, 2024 • edited Loading

Transaction costs

Script summary

Init transaction costs

Commit transaction costs

CollectCom transaction costs

Cost of Decrement Transaction

Close transaction costs

Contest transaction costs

Abort transaction costs

FanOut transaction costs

End-to-end benchmark results

Baseline Scenario

Three local nodes

ch1bo commented Aug 19, 2024

ch1bo Oct 4, 2024

Choose a reason for hiding this comment

noonio commented Jul 31, 2024 •

edited by v0d1ch

Loading

github-actions bot commented Jul 31, 2024 •

edited

Loading

`Init` transaction costs

`Commit` transaction costs

`CollectCom` transaction costs

`Close` transaction costs

`Contest` transaction costs

`Abort` transaction costs

`FanOut` transaction costs