Use `cardano-ledger` for all minimum UTxO calculations #3456

jonathanknowles · 2022-08-24T06:45:59Z

Issue Number

ADP-2144

Summary

This PR:

uses the ledger function evaluateMinLovelaceOutput to replace the use of Cardano API function calculateMinimumUTxO in all implementation code, for all eras.
repurposes the Cardano API function calculateMinimumUTxO for use as an oracle, to compare against when testing.
removes all special-casing for the Babbage era, and replaces it with a simpler implementation that works in all eras.
revises documentation for the {compute,isBelow}minimumCoinForUTxO functions to explain their intended purpose more clearly.

Context

The ledger function evaluateMinLovelaceOutput accepts an era-specific protocol parameters object and, unlike like the Cardano API, does not require that we convert to an era-agnostic protocol parameters object. Since values of the MinimumUTxO type already contain era-specific protocol parameters, this means we can avoid the extra complication of record conversion whenever we call:
- isBelowMinimumCoinForUTxO
- computeMinimumCoinForUTxO

Performance

Mainnet (Alonzo):

$ time curl -X POST http://localhost:8091/v2/wallets/042094088ed439a7b14e811e0781a00185b921c2/payment-fees -d '{"payments":[{"address":"addr1qylgm2dh3vpv07cjfcyuu6vhaqhf8998qcx6s8ucpkly6f8l0dw5r75vk42mv3ykq8vyjeaanvpytg79xqzymqy5acmqgyxuyr","amount":{"quantity":100000000,"unit":"lovelace"}}]}' -H "Content-Type: application/json"
{"deposit":{"quantity":0,"unit":"lovelace"},"estimated_max":{"quantity":187633,"unit":"lovelace"},"estimated_min":{"quantity":169021,"unit":"lovelace"},"minimum_coins":[{"quantity":999978,"unit":"lovelace"}]}
real	0m0,029s
user	0m0,006s
sys	0m0,000s

Preprod (Alonzo):

$ time curl -X POST http://localhost:8090/v2/wallets/1f82e83772b7579fc0854bd13db6a9cce21ccd95/payment-fees \
> -d '{"payments":[{"address":"addr_test1qrtez7vn0d8xp495ggypmu2kyt7tt6qyva2spm0f5a3ewn0v474mcs4q8e9g55yknx3729kyg5dl69x5596ee9tvnynq7ffety","amount":{"quantity":1000000,"unit":"lovelace"}}]}' \
> -H "Content-Type: application/json"
{"deposit":{"quantity":0,"unit":"lovelace"},"estimated_max":{"quantity":169021,"unit":"lovelace"},"estimated_min":{"quantity":169021,"unit":"lovelace"},"minimum_coins":[{"quantity":999978,"unit":"lovelace"}]}
real	0m0,029s
user	0m0,006s
sys	0m0,000s

Preview (Babbage):

$ time curl -X POST http://localhost:8090/v2/wallets/1f82e83772b7579fc0854bd13db6a9cce21ccd95/payment-fees -d '{"payments":[{"address":"addr_test1qrtez7vn0d8xp495ggypmu2kyt7tt6qyva2spm0f5a3ewn0v474mcs4q8e9g55yknx3729kyg5dl69x5596ee9tvnynq7ffety","amount":{"quantity":100000000,"unit":"lovelace"}}]}' -H "Content-Type: application/json"
{"deposit":{"quantity":0,"unit":"lovelace"},"estimated_max":{"quantity":187809,"unit":"lovelace"},"estimated_min":{"quantity":169197,"unit":"lovelace"},"minimum_coins":[{"quantity":995610,"unit":"lovelace"}]}
real	0m0,036s
user	0m0,003s
sys	0m0,003s

jonathanknowles · 2022-08-24T09:17:21Z

bors try

iohk-bors · 2022-08-24T10:20:59Z

try

Build succeeded:

Anviking · 2022-08-24T10:42:01Z

lib/shelley/test/unit/Cardano/Wallet/Shelley/MinimumUTxOSpec.hs

+        Internal.computeMinimumCoinForUTxO_CardanoLedger
+            minimumUTxO txOut
+  where
+    txOut = TxOut address (TokenBundle coin tokenMap)


Q: Why not take a TokenBundle directly like in other properties? It seems you needed to define Arbitrary Coin specially for this property.

Q: Why not take a TokenBundle directly like in other properties? It seems you needed to define Arbitrary Coin specially for this property.

Good question! I've simplified this in 9f37edc.

Anviking · 2022-08-24T10:44:59Z

lib/shelley/src/Cardano/Wallet/Shelley/MinimumUTxO.hs

+-- This function should /only/ be used to validate existing 'Coin' values that
+-- do not need to be modified in any way.
+--
+-- Increasing the 'Coin' value of an output can lead to an increase in the
+-- serialized length of that output, which can in turn lead to an increase in
+-- the minimum required 'Coin' value, since the minimum required 'Coin' value
+-- is dependent on an output's serialized length.
+--
+-- Therefore, even if this function indicates that a given value 'Coin' value
+-- 'c' satisfies the minimum UTxO rule, it should not be taken to imply that
+-- all values greater than 'c' will also satisfy the minimum UTxO rule.
+--
+-- If you need to generate a value that can always safely be increased, use
+-- the 'computeMinimumCoinForUTxO' function instead.
+--


Anviking · 2022-08-24T10:48:50Z

lib/shelley/src/Cardano/Wallet/Shelley/MinimumUTxO/Internal.hs

+                    $ toAlonzoTxOut txOut Nothing
+            Cardano.ShelleyBasedEraBabbage ->
+                evaluateMinLovelaceOutput pp
+                    $ toBabbageTxOut txOut Nothing


Possibly we could move the case era of to a toLedger :: ShelleyEra era => W.TxOut -> Ledger.TxOut era? but it doesn't matter now; we'll see how things turn out in the future

I agree. It'd be nice to factor out this pattern match as you suggest. It's something we could do in a follow-up PR.

I did give this a try, but couldn't get the types to line up in a reasonable amount of time.

The ledger code has so many re-exports that I just gave up trying to figure out which things to import...

Anviking · 2022-08-24T11:16:14Z

bors r+

3456: Use `cardano-ledger` for all minimum UTxO calculations r=Anviking a=jonathanknowles ## Issue Number ADP-2144 ## Summary This PR: - uses the ledger function [`evaluateMinLovelaceOutput`](https://github.com/input-output-hk/cardano-ledger/blob/68a535603c80b7cf53425fd34558e23f9983dfe8/eras/shelley/impl/src/Cardano/Ledger/Shelley/API/Wallet.hs#L506) to replace the use of Cardano API function [`calculateMinimumUTxO`](https://github.com/input-output-hk/cardano-node/blob/42809ad3d420f0695eb147d4c66b90573d27cafd/cardano-api/src/Cardano/Api/Fees.hs#L1226) in all implementation code, for all eras. - repurposes the Cardano API function [`calculateMinimumUTxO`](https://github.com/input-output-hk/cardano-node/blob/42809ad3d420f0695eb147d4c66b90573d27cafd/cardano-api/src/Cardano/Api/Fees.hs#L1226) for use as an oracle, to compare against when testing. - removes all special-casing for the Babbage era, and replaces it with a simpler implementation that works in all eras. - revises documentation for the `{compute,isBelow}minimumCoinForUTxO` functions to explain their intended purpose more clearly. ## Context - The ledger function [`evaluateMinLovelaceOutput`](https://github.com/input-output-hk/cardano-ledger/blob/68a535603c80b7cf53425fd34558e23f9983dfe8/eras/shelley/impl/src/Cardano/Ledger/Shelley/API/Wallet.hs#L506) accepts an _era-specific_ protocol parameters object and, unlike like the Cardano API, does _not_ require that we convert to an era-agnostic protocol parameters object. Since values of the `MinimumUTxO` type already contain era-specific protocol parameters, this means we can avoid the extra complication of record conversion whenever we call: - `isBelowMinimumCoinForUTxO` - `computeMinimumCoinForUTxO` ## Performance Mainnet (Alonzo): ```sh $ time curl -X POST http://localhost:8091/v2/wallets/042094088ed439a7b14e811e0781a00185b921c2/payment-fees -d '{"payments":[{"address":"addr1qylgm2dh3vpv07cjfcyuu6vhaqhf8998qcx6s8ucpkly6f8l0dw5r75vk42mv3ykq8vyjeaanvpytg79xqzymqy5acmqgyxuyr","amount":{"quantity":100000000,"unit":"lovelace"}}]}' -H "Content-Type: application/json" {"deposit":{"quantity":0,"unit":"lovelace"},"estimated_max":{"quantity":187633,"unit":"lovelace"},"estimated_min":{"quantity":169021,"unit":"lovelace"},"minimum_coins":[{"quantity":999978,"unit":"lovelace"}]} real 0m0,029s user 0m0,006s sys 0m0,000s ``` Preprod (Alonzo): ```sh $ time curl -X POST http://localhost:8090/v2/wallets/1f82e83772b7579fc0854bd13db6a9cce21ccd95/payment-fees \ > -d '{"payments":[{"address":"addr_test1qrtez7vn0d8xp495ggypmu2kyt7tt6qyva2spm0f5a3ewn0v474mcs4q8e9g55yknx3729kyg5dl69x5596ee9tvnynq7ffety","amount":{"quantity":1000000,"unit":"lovelace"}}]}' \ > -H "Content-Type: application/json" {"deposit":{"quantity":0,"unit":"lovelace"},"estimated_max":{"quantity":169021,"unit":"lovelace"},"estimated_min":{"quantity":169021,"unit":"lovelace"},"minimum_coins":[{"quantity":999978,"unit":"lovelace"}]} real 0m0,029s user 0m0,006s sys 0m0,000s ``` Preview (Babbage): ```sh $ time curl -X POST http://localhost:8090/v2/wallets/1f82e83772b7579fc0854bd13db6a9cce21ccd95/payment-fees -d '{"payments":[{"address":"addr_test1qrtez7vn0d8xp495ggypmu2kyt7tt6qyva2spm0f5a3ewn0v474mcs4q8e9g55yknx3729kyg5dl69x5596ee9tvnynq7ffety","amount":{"quantity":100000000,"unit":"lovelace"}}]}' -H "Content-Type: application/json" {"deposit":{"quantity":0,"unit":"lovelace"},"estimated_max":{"quantity":187809,"unit":"lovelace"},"estimated_min":{"quantity":169197,"unit":"lovelace"},"minimum_coins":[{"quantity":995610,"unit":"lovelace"}]} real 0m0,036s user 0m0,003s sys 0m0,003s ``` Co-authored-by: Jonathan Knowles <[email protected]>

iohk-bors · 2022-08-24T12:19:23Z

Build failed:

ci/hydra-build:required

Failures:

  src/Test/Integration/Scenario/API/Shelley/StakePools.hs:1226:25: 
  1) API Specifications, SHELLEY_STAKE_POOLS, STAKE_POOLS_SMASH_01 - fetching metadata from SMASH works with delisted pools
       predicate failed on: fromList []
       While verifying value:
         ( Status
             { statusCode = 200
             , statusMessage = "OK"
             }
         , Right
             [ ApiStakePool
                 { id = ApiT
                     ( PoolId
                         { getPoolId = "ì(ó=ËæÖ@
                         \x1e^3\x9bÐd|\x9sÊl\xcùÂ»æ\x83\x8dÆ" }
                     )
                 , metrics = ApiStakePoolMetrics
                     { nonMyopicMemberRewards = Quantity 165073746296518
                     , relativeStake = Quantity
                         ( Percentage
                             ( 663 % 5000 )
                         )
                     , saturation = 0.39788348046965794
                     , producedBlocks = Quantity 803
                     }
                 , metadata = Nothing
                 , cost = Quantity 0
                 , margin = Quantity
                     ( Percentage
                         ( 1 % 10 )
                     )
                 , pledge = Quantity 200000000000000
                 , retirement = Nothing
                 , flags = []
                 }
             , ApiStakePool
                 { id = ApiT
                     ( PoolId
                         { getPoolId = "´WhÁ¢ÚKÑ>¼ª\x1e¥\x14\x8í£\x1dÌ!v\ËÔ\x7Í©ò" }
                     )
                 , metrics = ApiStakePoolMetrics
                     { nonMyopicMemberRewards = Quantity 165068189605536
                     , relativeStake = Quantity
                         ( Percentage
                             ( 471 % 5000 )
                         )
                     , saturation = 0.2825072023675228
                     , producedBlocks = Quantity 549
                     }
                 , metadata = Nothing
                 , cost = Quantity 0
                 , margin = Quantity
                     ( Percentage
                         ( 1 % 10 )
                     )
                 , pledge = Quantity 100000000000000
                 , retirement = Just
                     ( ApiEpochInfo
                         { epochNumber = ApiT
                             ( EpochNo
                                 { unEpochNo = 100000 }
                             )
                         , epochStartTime = 2022-09-12 00:21:10 UTC
                         }
                     )
                 , flags = []
                 }
             , ApiStakePool
                 { id = ApiT
                     ( PoolId
                         { getPoolId = "»\x11L³}uú\x5&\x3(Â5£Úâ\x95£=\xb¦t¥ë\x1e>V\x8e" }
                     )
                 , metrics = ApiStakePoolMetrics
                     { nonMyopicMemberRewards = Quantity 165068189605536
                     , relativeStake = Quantity
                         ( Percentage
                             ( 803 % 10000 )
                         )
                     , saturation = 0.24098040481372768
                     , producedBlocks = Quantity 471
                     }
                 , metadata = Nothing
                 , cost = Quantity 0
                 , margin = Quantity
                     ( Percentage
                         ( 1 % 10 )
                     )
                 , pledge = Quantity 100000000000000
                 , retirement = Just
                     ( ApiEpochInfo
                         { epochNumber = ApiT
                             ( EpochNo
                                 { unEpochNo = 1000000 }
                             )
                         , epochStartTime = 2023-02-25 16:21:10 UTC
                         }
                     )
                 , flags = []
                 }
             ]
         )
       
       Waited longer than 300s to resolve action: "metadata is fetched".

  To rerun use: --match "/API Specifications/SHELLEY_STAKE_POOLS/STAKE_POOLS_SMASH_01 - fetching metadata from SMASH works with delisted pools/"

Randomized with seed 1613914077

#2331

jonathanknowles · 2022-08-24T12:21:52Z

bors r+

3456: Use `cardano-ledger` for all minimum UTxO calculations r=jonathanknowles a=jonathanknowles ## Issue Number ADP-2144 ## Summary This PR: - uses the ledger function [`evaluateMinLovelaceOutput`](https://github.com/input-output-hk/cardano-ledger/blob/68a535603c80b7cf53425fd34558e23f9983dfe8/eras/shelley/impl/src/Cardano/Ledger/Shelley/API/Wallet.hs#L506) to replace the use of Cardano API function [`calculateMinimumUTxO`](https://github.com/input-output-hk/cardano-node/blob/42809ad3d420f0695eb147d4c66b90573d27cafd/cardano-api/src/Cardano/Api/Fees.hs#L1226) in all implementation code, for all eras. - repurposes the Cardano API function [`calculateMinimumUTxO`](https://github.com/input-output-hk/cardano-node/blob/42809ad3d420f0695eb147d4c66b90573d27cafd/cardano-api/src/Cardano/Api/Fees.hs#L1226) for use as an oracle, to compare against when testing. - removes all special-casing for the Babbage era, and replaces it with a simpler implementation that works in all eras. - revises documentation for the `{compute,isBelow}minimumCoinForUTxO` functions to explain their intended purpose more clearly. ## Context - The ledger function [`evaluateMinLovelaceOutput`](https://github.com/input-output-hk/cardano-ledger/blob/68a535603c80b7cf53425fd34558e23f9983dfe8/eras/shelley/impl/src/Cardano/Ledger/Shelley/API/Wallet.hs#L506) accepts an _era-specific_ protocol parameters object and, unlike like the Cardano API, does _not_ require that we convert to an era-agnostic protocol parameters object. Since values of the `MinimumUTxO` type already contain era-specific protocol parameters, this means we can avoid the extra complication of record conversion whenever we call: - `isBelowMinimumCoinForUTxO` - `computeMinimumCoinForUTxO` ## Performance Mainnet (Alonzo): ```sh $ time curl -X POST http://localhost:8091/v2/wallets/042094088ed439a7b14e811e0781a00185b921c2/payment-fees -d '{"payments":[{"address":"addr1qylgm2dh3vpv07cjfcyuu6vhaqhf8998qcx6s8ucpkly6f8l0dw5r75vk42mv3ykq8vyjeaanvpytg79xqzymqy5acmqgyxuyr","amount":{"quantity":100000000,"unit":"lovelace"}}]}' -H "Content-Type: application/json" {"deposit":{"quantity":0,"unit":"lovelace"},"estimated_max":{"quantity":187633,"unit":"lovelace"},"estimated_min":{"quantity":169021,"unit":"lovelace"},"minimum_coins":[{"quantity":999978,"unit":"lovelace"}]} real 0m0,029s user 0m0,006s sys 0m0,000s ``` Preprod (Alonzo): ```sh $ time curl -X POST http://localhost:8090/v2/wallets/1f82e83772b7579fc0854bd13db6a9cce21ccd95/payment-fees \ > -d '{"payments":[{"address":"addr_test1qrtez7vn0d8xp495ggypmu2kyt7tt6qyva2spm0f5a3ewn0v474mcs4q8e9g55yknx3729kyg5dl69x5596ee9tvnynq7ffety","amount":{"quantity":1000000,"unit":"lovelace"}}]}' \ > -H "Content-Type: application/json" {"deposit":{"quantity":0,"unit":"lovelace"},"estimated_max":{"quantity":169021,"unit":"lovelace"},"estimated_min":{"quantity":169021,"unit":"lovelace"},"minimum_coins":[{"quantity":999978,"unit":"lovelace"}]} real 0m0,029s user 0m0,006s sys 0m0,000s ``` Preview (Babbage): ```sh $ time curl -X POST http://localhost:8090/v2/wallets/1f82e83772b7579fc0854bd13db6a9cce21ccd95/payment-fees -d '{"payments":[{"address":"addr_test1qrtez7vn0d8xp495ggypmu2kyt7tt6qyva2spm0f5a3ewn0v474mcs4q8e9g55yknx3729kyg5dl69x5596ee9tvnynq7ffety","amount":{"quantity":100000000,"unit":"lovelace"}}]}' -H "Content-Type: application/json" {"deposit":{"quantity":0,"unit":"lovelace"},"estimated_max":{"quantity":187809,"unit":"lovelace"},"estimated_min":{"quantity":169197,"unit":"lovelace"},"minimum_coins":[{"quantity":995610,"unit":"lovelace"}]} real 0m0,036s user 0m0,003s sys 0m0,003s ``` Co-authored-by: Jonathan Knowles <[email protected]>

iohk-bors · 2022-08-24T13:14:22Z

This PR was included in a batch that successfully built, but then failed to merge into master (it was a non-fast-forward update). It will be automatically retried.

3456: Use `cardano-ledger` for all minimum UTxO calculations r=jonathanknowles a=jonathanknowles ## Issue Number ADP-2144 ## Summary This PR: - uses the ledger function [`evaluateMinLovelaceOutput`](https://github.com/input-output-hk/cardano-ledger/blob/68a535603c80b7cf53425fd34558e23f9983dfe8/eras/shelley/impl/src/Cardano/Ledger/Shelley/API/Wallet.hs#L506) to replace the use of Cardano API function [`calculateMinimumUTxO`](https://github.com/input-output-hk/cardano-node/blob/42809ad3d420f0695eb147d4c66b90573d27cafd/cardano-api/src/Cardano/Api/Fees.hs#L1226) in all implementation code, for all eras. - repurposes the Cardano API function [`calculateMinimumUTxO`](https://github.com/input-output-hk/cardano-node/blob/42809ad3d420f0695eb147d4c66b90573d27cafd/cardano-api/src/Cardano/Api/Fees.hs#L1226) for use as an oracle, to compare against when testing. - removes all special-casing for the Babbage era, and replaces it with a simpler implementation that works in all eras. - revises documentation for the `{compute,isBelow}minimumCoinForUTxO` functions to explain their intended purpose more clearly. ## Context - The ledger function [`evaluateMinLovelaceOutput`](https://github.com/input-output-hk/cardano-ledger/blob/68a535603c80b7cf53425fd34558e23f9983dfe8/eras/shelley/impl/src/Cardano/Ledger/Shelley/API/Wallet.hs#L506) accepts an _era-specific_ protocol parameters object and, unlike like the Cardano API, does _not_ require that we convert to an era-agnostic protocol parameters object. Since values of the `MinimumUTxO` type already contain era-specific protocol parameters, this means we can avoid the extra complication of record conversion whenever we call: - `isBelowMinimumCoinForUTxO` - `computeMinimumCoinForUTxO` ## Performance Mainnet (Alonzo): ```sh $ time curl -X POST http://localhost:8091/v2/wallets/042094088ed439a7b14e811e0781a00185b921c2/payment-fees -d '{"payments":[{"address":"addr1qylgm2dh3vpv07cjfcyuu6vhaqhf8998qcx6s8ucpkly6f8l0dw5r75vk42mv3ykq8vyjeaanvpytg79xqzymqy5acmqgyxuyr","amount":{"quantity":100000000,"unit":"lovelace"}}]}' -H "Content-Type: application/json" {"deposit":{"quantity":0,"unit":"lovelace"},"estimated_max":{"quantity":187633,"unit":"lovelace"},"estimated_min":{"quantity":169021,"unit":"lovelace"},"minimum_coins":[{"quantity":999978,"unit":"lovelace"}]} real 0m0,029s user 0m0,006s sys 0m0,000s ``` Preprod (Alonzo): ```sh $ time curl -X POST http://localhost:8090/v2/wallets/1f82e83772b7579fc0854bd13db6a9cce21ccd95/payment-fees \ > -d '{"payments":[{"address":"addr_test1qrtez7vn0d8xp495ggypmu2kyt7tt6qyva2spm0f5a3ewn0v474mcs4q8e9g55yknx3729kyg5dl69x5596ee9tvnynq7ffety","amount":{"quantity":1000000,"unit":"lovelace"}}]}' \ > -H "Content-Type: application/json" {"deposit":{"quantity":0,"unit":"lovelace"},"estimated_max":{"quantity":169021,"unit":"lovelace"},"estimated_min":{"quantity":169021,"unit":"lovelace"},"minimum_coins":[{"quantity":999978,"unit":"lovelace"}]} real 0m0,029s user 0m0,006s sys 0m0,000s ``` Preview (Babbage): ```sh $ time curl -X POST http://localhost:8090/v2/wallets/1f82e83772b7579fc0854bd13db6a9cce21ccd95/payment-fees -d '{"payments":[{"address":"addr_test1qrtez7vn0d8xp495ggypmu2kyt7tt6qyva2spm0f5a3ewn0v474mcs4q8e9g55yknx3729kyg5dl69x5596ee9tvnynq7ffety","amount":{"quantity":100000000,"unit":"lovelace"}}]}' -H "Content-Type: application/json" {"deposit":{"quantity":0,"unit":"lovelace"},"estimated_max":{"quantity":187809,"unit":"lovelace"},"estimated_min":{"quantity":169197,"unit":"lovelace"},"minimum_coins":[{"quantity":995610,"unit":"lovelace"}]} real 0m0,036s user 0m0,003s sys 0m0,003s ``` Co-authored-by: Jonathan Knowles <[email protected]>

iohk-bors · 2022-08-24T16:28:43Z

Build failed:

ci/hydra-build:required

Failures:

  src/Test/Integration/Scenario/API/Shelley/Addresses.hs:241:18: 
  1) API Specifications, SHELLEY_ADDRESSES, ADDRESS_LIST_03 - Generates new address pool gap
       Waited longer than 90s to resolve action: "Wallet balance = initPoolGap * minUTxOValue".
       expected: Quantity 9783700
        but got: Quantity 0

  To rerun use: --match "/API Specifications/SHELLEY_ADDRESSES/ADDRESS_LIST_03 - Generates new address pool gap/"

  src/Test/Integration/Scenario/API/Shelley/Addresses.hs:334:42: 
  2) API Specifications, SHELLEY_ADDRESSES, ADDRESS_LIST_06 - Used change addresses are listed after a transaction is no longer pending
       Waited longer than 90s to resolve action: "Transaction from A -> B is discovered on B".
       expected: ApiT InLedger
        but got: ApiT Pending

  To rerun use: --match "/API Specifications/SHELLEY_ADDRESSES/ADDRESS_LIST_06 - Used change addresses are listed after a transaction is no longer pending/"

  src/Test/Integration/Framework/DSL.hs:1056:51: 
  3) API Specifications, SHELLEY_COIN_SELECTION, WALLETS_COIN_SELECTION_06a - can redeem rewards from self
       not yet
       Waited longer than 90s to resolve action: "waitForNextEpoch: goes to next epoch".

  To rerun use: --match "/API Specifications/SHELLEY_COIN_SELECTION/WALLETS_COIN_SELECTION_06a - can redeem rewards from self/"
  
...


  src/Test/Integration/Scenario/CLI/Shelley/Wallets.hs:761:5: 
  246) CLI Specifications, SHELLEY_CLI_WALLETS, WALLETS_UTXO_02 - Utxo statistics works properly
       uncaught exception: IOException of type UserError
       user error (No more faucet wallets of type shelley! You may need to manually add entries to the relevant list in lib/core-integration/src/Test/Integration/Faucet.hs)

  To rerun use: --match "/CLI Specifications/SHELLEY_CLI_WALLETS/WALLETS_UTXO_02 - Utxo statistics works properly/"

  src/Test/Integration/Scenario/CLI/Shelley/HWWallets.hs:100:5: 
  247) CLI Specifications, SHELLEY_CLI_HW_WALLETS, HW_WALLETS_01x - Restoration from account public key preserves funds
       uncaught exception: IOException of type UserError
       user error (No more faucet wallets of type shelley! You may need to manually add entries to the relevant list in lib/core-integration/src/Test/Integration/Faucet.hs)

  To rerun use: --match "/CLI Specifications/SHELLEY_CLI_HW_WALLETS/HW_WALLETS_01x - Restoration from account public key preserves funds/"

  src/Test/Integration/Scenario/CLI/Shelley/HWWallets.hs:158:9: 
  248) CLI Specifications, SHELLEY_CLI_HW_WALLETS, HW_WALLETS_03 - Cannot do operations requiring private key, Cannot send tx
       uncaught exception: IOException of type UserError
       user error (No more faucet wallets of type shelley! You may need to manually add entries to the relevant list in lib/core-integration/src/Test/Integration/Faucet.hs)

  To rerun use: --match "/CLI Specifications/SHELLEY_CLI_HW_WALLETS/HW_WALLETS_03 - Cannot do operations requiring private key/Cannot send tx/"

  src/Test/Integration/Scenario/CLI/Shelley/HWWallets.hs:223:9: 
  249) CLI Specifications, SHELLEY_CLI_HW_WALLETS, HW_WALLETS_04 - Can manage HW wallet the same way as others, Can get tx fee
       uncaught exception: IOException of type UserError
       user error (No more faucet wallets of type shelley! You may need to manually add entries to the relevant list in lib/core-integration/src/Test/Integration/Faucet.hs)

  To rerun use: --match "/CLI Specifications/SHELLEY_CLI_HW_WALLETS/HW_WALLETS_04 - Can manage HW wallet the same way as others/Can get tx fee/"

  src/Test/Integration/Scenario/CLI/Network.hs:60:18: 
  250) CLI Specifications, COMMON_CLI_NETWORK, CLI_NETWORK - cardano-wallet network information
       uncaught exception: ErrorCall
       Maybe.fromJust: Nothing
       CallStack (from HasCallStack):
         error, called at libraries/base/Data/Maybe.hs:148:21 in base:Data.Maybe
         fromJust, called at src/Test/Integration/Scenario/CLI/Network.hs:60:18 in cardano-wallet-core-integration-2022.8.16-B9GxqSrgs4zKVx9CxR3hks:Test.Integration.Scenario.CLI.Network

  To rerun use: --match "/CLI Specifications/COMMON_CLI_NETWORK/CLI_NETWORK - cardano-wallet network information/"

Randomized with seed 1397860413

#2456

jonathanknowles · 2022-08-24T16:42:52Z

bors r+

3456: Use `cardano-ledger` for all minimum UTxO calculations r=jonathanknowles a=jonathanknowles ## Issue Number ADP-2144 ## Summary This PR: - uses the ledger function [`evaluateMinLovelaceOutput`](https://github.com/input-output-hk/cardano-ledger/blob/68a535603c80b7cf53425fd34558e23f9983dfe8/eras/shelley/impl/src/Cardano/Ledger/Shelley/API/Wallet.hs#L506) to replace the use of Cardano API function [`calculateMinimumUTxO`](https://github.com/input-output-hk/cardano-node/blob/42809ad3d420f0695eb147d4c66b90573d27cafd/cardano-api/src/Cardano/Api/Fees.hs#L1226) in all implementation code, for all eras. - repurposes the Cardano API function [`calculateMinimumUTxO`](https://github.com/input-output-hk/cardano-node/blob/42809ad3d420f0695eb147d4c66b90573d27cafd/cardano-api/src/Cardano/Api/Fees.hs#L1226) for use as an oracle, to compare against when testing. - removes all special-casing for the Babbage era, and replaces it with a simpler implementation that works in all eras. - revises documentation for the `{compute,isBelow}minimumCoinForUTxO` functions to explain their intended purpose more clearly. ## Context - The ledger function [`evaluateMinLovelaceOutput`](https://github.com/input-output-hk/cardano-ledger/blob/68a535603c80b7cf53425fd34558e23f9983dfe8/eras/shelley/impl/src/Cardano/Ledger/Shelley/API/Wallet.hs#L506) accepts an _era-specific_ protocol parameters object and, unlike like the Cardano API, does _not_ require that we convert to an era-agnostic protocol parameters object. Since values of the `MinimumUTxO` type already contain era-specific protocol parameters, this means we can avoid the extra complication of record conversion whenever we call: - `isBelowMinimumCoinForUTxO` - `computeMinimumCoinForUTxO` ## Performance Mainnet (Alonzo): ```sh $ time curl -X POST http://localhost:8091/v2/wallets/042094088ed439a7b14e811e0781a00185b921c2/payment-fees -d '{"payments":[{"address":"addr1qylgm2dh3vpv07cjfcyuu6vhaqhf8998qcx6s8ucpkly6f8l0dw5r75vk42mv3ykq8vyjeaanvpytg79xqzymqy5acmqgyxuyr","amount":{"quantity":100000000,"unit":"lovelace"}}]}' -H "Content-Type: application/json" {"deposit":{"quantity":0,"unit":"lovelace"},"estimated_max":{"quantity":187633,"unit":"lovelace"},"estimated_min":{"quantity":169021,"unit":"lovelace"},"minimum_coins":[{"quantity":999978,"unit":"lovelace"}]} real 0m0,029s user 0m0,006s sys 0m0,000s ``` Preprod (Alonzo): ```sh $ time curl -X POST http://localhost:8090/v2/wallets/1f82e83772b7579fc0854bd13db6a9cce21ccd95/payment-fees \ > -d '{"payments":[{"address":"addr_test1qrtez7vn0d8xp495ggypmu2kyt7tt6qyva2spm0f5a3ewn0v474mcs4q8e9g55yknx3729kyg5dl69x5596ee9tvnynq7ffety","amount":{"quantity":1000000,"unit":"lovelace"}}]}' \ > -H "Content-Type: application/json" {"deposit":{"quantity":0,"unit":"lovelace"},"estimated_max":{"quantity":169021,"unit":"lovelace"},"estimated_min":{"quantity":169021,"unit":"lovelace"},"minimum_coins":[{"quantity":999978,"unit":"lovelace"}]} real 0m0,029s user 0m0,006s sys 0m0,000s ``` Preview (Babbage): ```sh $ time curl -X POST http://localhost:8090/v2/wallets/1f82e83772b7579fc0854bd13db6a9cce21ccd95/payment-fees -d '{"payments":[{"address":"addr_test1qrtez7vn0d8xp495ggypmu2kyt7tt6qyva2spm0f5a3ewn0v474mcs4q8e9g55yknx3729kyg5dl69x5596ee9tvnynq7ffety","amount":{"quantity":100000000,"unit":"lovelace"}}]}' -H "Content-Type: application/json" {"deposit":{"quantity":0,"unit":"lovelace"},"estimated_max":{"quantity":187809,"unit":"lovelace"},"estimated_min":{"quantity":169197,"unit":"lovelace"},"minimum_coins":[{"quantity":995610,"unit":"lovelace"}]} real 0m0,036s user 0m0,003s sys 0m0,003s ``` Co-authored-by: Jonathan Knowles <[email protected]>

iohk-bors · 2022-08-24T20:29:09Z

Build failed:

ci/hydra-build:required


  src/Test/Integration/Scenario/API/Shelley/Addresses.hs:241:18: 
  1) API Specifications, SHELLEY_ADDRESSES, ADDRESS_LIST_03 - Generates new address pool gap
       Waited longer than 90s to resolve action: "Wallet balance = initPoolGap * minUTxOValue".
       expected: Quantity 9783700
        but got: Quantity 0

  To rerun use: --match "/API Specifications/SHELLEY_ADDRESSES/ADDRESS_LIST_03 - Generates new address pool gap/"

  src/Test/Integration/Scenario/API/Shelley/Addresses.hs:334:42: 
  2) API Specifications, SHELLEY_ADDRESSES, ADDRESS_LIST_06 - Used change addresses are listed after a transaction is no longer pending
       Waited longer than 90s to resolve action: "Transaction from A -> B is discovered on B".
       expected: ApiT InLedger
        but got: ApiT Pending

  To rerun use: --match "/API Specifications/SHELLEY_ADDRESSES/ADDRESS_LIST_06 - Used change addresses are listed after a transaction is no longer pending/"

  src/Test/Integration/Framework/DSL.hs:1056:51: 
  3) API Specifications, SHELLEY_COIN_SELECTION, WALLETS_COIN_SELECTION_06a - can redeem rewards from self
       not yet
       Waited longer than 90s to resolve action: "waitForNextEpoch: goes to next epoch".

  To rerun use: --match "/API Specifications/SHELLEY_COIN_SELECTION/WALLETS_COIN_SELECTION_06a - can redeem rewards from self/"

...

  246) CLI Specifications, SHELLEY_CLI_WALLETS, WALLETS_UTXO_02 - Utxo statistics works properly
       uncaught exception: IOException of type UserError
       user error (No more faucet wallets of type shelley! You may need to manually add entries to the relevant list in lib/core-integration/src/Test/Integration/Faucet.hs)

  To rerun use: --match "/CLI Specifications/SHELLEY_CLI_WALLETS/WALLETS_UTXO_02 - Utxo statistics works properly/"

  src/Test/Integration/Scenario/CLI/Shelley/HWWallets.hs:100:5: 
  247) CLI Specifications, SHELLEY_CLI_HW_WALLETS, HW_WALLETS_01x - Restoration from account public key preserves funds
       uncaught exception: IOException of type UserError
       user error (No more faucet wallets of type shelley! You may need to manually add entries to the relevant list in lib/core-integration/src/Test/Integration/Faucet.hs)

  To rerun use: --match "/CLI Specifications/SHELLEY_CLI_HW_WALLETS/HW_WALLETS_01x - Restoration from account public key preserves funds/"

  src/Test/Integration/Scenario/CLI/Shelley/HWWallets.hs:158:9: 
  248) CLI Specifications, SHELLEY_CLI_HW_WALLETS, HW_WALLETS_03 - Cannot do operations requiring private key, Cannot send tx
       uncaught exception: IOException of type UserError
       user error (No more faucet wallets of type shelley! You may need to manually add entries to the relevant list in lib/core-integration/src/Test/Integration/Faucet.hs)

  To rerun use: --match "/CLI Specifications/SHELLEY_CLI_HW_WALLETS/HW_WALLETS_03 - Cannot do operations requiring private key/Cannot send tx/"

  src/Test/Integration/Scenario/CLI/Shelley/HWWallets.hs:223:9: 
  249) CLI Specifications, SHELLEY_CLI_HW_WALLETS, HW_WALLETS_04 - Can manage HW wallet the same way as others, Can get tx fee
       uncaught exception: IOException of type UserError
       user error (No more faucet wallets of type shelley! You may need to manually add entries to the relevant list in lib/core-integration/src/Test/Integration/Faucet.hs)

  To rerun use: --match "/CLI Specifications/SHELLEY_CLI_HW_WALLETS/HW_WALLETS_04 - Can manage HW wallet the same way as others/Can get tx fee/"

  src/Test/Integration/Scenario/CLI/Network.hs:60:18: 
  250) CLI Specifications, COMMON_CLI_NETWORK, CLI_NETWORK - cardano-wallet network information
       uncaught exception: ErrorCall
       Maybe.fromJust: Nothing
       CallStack (from HasCallStack):
         error, called at libraries/base/Data/Maybe.hs:148:21 in base:Data.Maybe
         fromJust, called at src/Test/Integration/Scenario/CLI/Network.hs:60:18 in cardano-wallet-core-integration-2022.8.16-B9GxqSrgs4zKVx9CxR3hks:Test.Integration.Scenario.CLI.Network

  To rerun use: --match "/CLI Specifications/COMMON_CLI_NETWORK/CLI_NETWORK - cardano-wallet network information/"

#2456

… module.

This function simply encapsulates the Cardano API minimum UTxO function in a wallet-friendly interface, acceping and returning wallet-friendly types.

…nction. This function is now only used by `computeMinimumCoinForUTxOCardanoApi`. We can safely make it an inner function.

This function simply encapsulates the Cardano Ledger minimum UTxO function in a wallet-friendly interface, acceping and returning wallet-friendly types.

…quivalent. We eventually wish to switch from the Cardano API implementation of the minimum UTXO calculaion to the Cardano Ledger implementation. But before we do that, we test that the implementations are equivalent.

This commit switches from the Cardano API implementation of the minimum UTxO calculation to the Cardano Ledger implementation.

Now that we are using the Cardano Ledger implementation of the minimum UTxO calculation, we no longer need to treat the Babbage era specially in our public implementation.

These functions are no longer used.

Since our minimum UTxO calculation now relies on Cardano Ledger, we no longer need to convert from era-specific protocol parameters to the era-agnostic protocol parameters required by the Cardano API. Therefore, this property test (which checked that conversions would never cause run-time errors) is no longer necessary.

The juxtaposition of "UTxO" followed by another capitalised word is rather hard to read. This commit introduces underscores to aid readability.

…itions.

…oLedger`. It's simpler for this property to accept a `TokenBundle` instead of a `TokenMap` and `Coin`. The `Arbitrary` instance for `TokenBundle` uses the `genTxOutTokenBundle` generator, which already has good coverage of minimum and maximum values specifically in the context of a transaction output. This allows us to get rid of the `Arbitrary` instance for `Coin` (which in hindsight would have been better to implement with the `genTxOutCoin` generator). In response to review feedback: #3456 (comment)

We can use the `Standard{Alonzo,Babbage}` synonyms for greater concision.

jonathanknowles · 2022-08-25T03:53:20Z

I'm going to merge this manually, as:

We've now made 4 attempts to merge with bors
Each attempt has failed (on hydra) with test failures unrelated to this PR
We've categorised every test failure with a related issue number
Each hydra job failure, when re-run, has eventually succeeded:

…dp-2144/use-ledger-minimum-utxo-function

jonathanknowles self-assigned this Aug 24, 2022

jonathanknowles force-pushed the jonathanknowles/adp-2144/use-ledger-minimum-utxo-function branch 3 times, most recently from 9d84e68 to 5d8f504 Compare August 24, 2022 07:51

jonathanknowles marked this pull request as ready for review August 24, 2022 07:54

jonathanknowles requested a review from Anviking August 24, 2022 08:02

jonathanknowles force-pushed the jonathanknowles/adp-2144/use-ledger-minimum-utxo-function branch 2 times, most recently from 9566e7a to acbfaad Compare August 24, 2022 08:26

iohk-bors bot added a commit that referenced this pull request Aug 24, 2022

Try #3456:

ed2f5d5

jonathanknowles force-pushed the jonathanknowles/adp-2144/use-ledger-minimum-utxo-function branch from acbfaad to 43a604a Compare August 24, 2022 09:22

Anviking approved these changes Aug 24, 2022

View reviewed changes

jonathanknowles added 6 commits August 25, 2022 03:46

Miscellaneous formatting fixes.

a846156

Move unsafeCoinFromCardanoApiCalculateMinimumUTxOResult to internal…

a345f43

… module.

Add function computeMinimumCoinForUTxOCardanoApi.

d014ad1

This function simply encapsulates the Cardano API minimum UTxO function in a wallet-friendly interface, acceping and returning wallet-friendly types.

Use computeMinimumCoinForUTxOCardanoApi to simplify implementation.

58893a7

Make unsafeCoinFromCardanoApiCalculateMinimumUTxOResult an inner fu…

73bcc85

…nction. This function is now only used by `computeMinimumCoinForUTxOCardanoApi`. We can safely make it an inner function.

Add era-specific TxOut conversion functions to Compatibility.Ledger.

8450f1d

jonathanknowles added 14 commits August 25, 2022 03:46

Add function computeMinimumCoinForUTxOCardanoLedger.

eef2937

This function simply encapsulates the Cardano Ledger minimum UTxO function in a wallet-friendly interface, acceping and returning wallet-friendly types.

Use computeMinimumCoinForUTxOCardanoLedger in implementation.

070fa91

This commit switches from the Cardano API implementation of the minimum UTxO calculation to the Cardano Ledger implementation.

Do not treat the Babbage era specially in the public implementation.

b54ab99

Now that we are using the Cardano Ledger implementation of the minimum UTxO calculation, we no longer need to treat the Babbage era specially in our public implementation.

Simplify public function computeMinimumCoinForUTxO.

3b8469c

Simplify public function isBelowMinimumCoinForUTxO.

85d4577

Delete redundant code from public MinimumUTxO module.

7d04155

These functions are no longer used.

Use underscores to make function names clearer.

3b009d2

The juxtaposition of "UTxO" followed by another capitalised word is rather hard to read. This commit introduces underscores to aid readability.

Use computeMinimumCoinForUTxO_CardanoApi to simplify property defin…

bd1f8e3

…itions.

Simplify property names for computeMinimumCoinForUTxO.

3b8f585

Revise documentation of computeMinimumCoinForUTxO functions.

93b4405

Simplify type signatures of to{Alonzo,Babbage}TxOut.

3866692

We can use the `Standard{Alonzo,Babbage}` synonyms for greater concision.

jonathanknowles force-pushed the jonathanknowles/adp-2144/use-ledger-minimum-utxo-function branch from 43a604a to 3866692 Compare August 25, 2022 03:48

jonathanknowles merged commit 6262c89 into master Aug 25, 2022

jonathanknowles deleted the jonathanknowles/adp-2144/use-ledger-minimum-utxo-function branch August 25, 2022 04:23

WilliamKingNoel-Bot pushed a commit that referenced this pull request Aug 25, 2022

docs: Merge pull request #3456 from input-output-hk/jonathanknowles/a…

0927e75

…dp-2144/use-ledger-minimum-utxo-function

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use `cardano-ledger` for all minimum UTxO calculations #3456

Use `cardano-ledger` for all minimum UTxO calculations #3456

jonathanknowles commented Aug 24, 2022 •

edited

Loading

jonathanknowles commented Aug 24, 2022

iohk-bors bot commented Aug 24, 2022

Anviking Aug 24, 2022

jonathanknowles Aug 25, 2022

Anviking Aug 24, 2022

Anviking Aug 24, 2022

Anviking Aug 24, 2022

jonathanknowles Aug 25, 2022

jonathanknowles Aug 25, 2022

Anviking commented Aug 24, 2022

iohk-bors bot commented Aug 24, 2022 •

edited by Anviking

Loading

jonathanknowles commented Aug 24, 2022

iohk-bors bot commented Aug 24, 2022

iohk-bors bot commented Aug 24, 2022 •

edited by jonathanknowles

Loading

jonathanknowles commented Aug 24, 2022

iohk-bors bot commented Aug 24, 2022 •

edited by jonathanknowles

Loading

jonathanknowles commented Aug 25, 2022

Use cardano-ledger for all minimum UTxO calculations #3456

Use cardano-ledger for all minimum UTxO calculations #3456

Conversation

jonathanknowles commented Aug 24, 2022 • edited Loading

Issue Number

Summary

Context

Performance

jonathanknowles commented Aug 24, 2022

iohk-bors bot commented Aug 24, 2022

try

Anviking Aug 24, 2022

Choose a reason for hiding this comment

jonathanknowles Aug 25, 2022

Choose a reason for hiding this comment

Anviking Aug 24, 2022

Choose a reason for hiding this comment

Anviking Aug 24, 2022

Choose a reason for hiding this comment

Anviking Aug 24, 2022

Choose a reason for hiding this comment

jonathanknowles Aug 25, 2022

Choose a reason for hiding this comment

jonathanknowles Aug 25, 2022

Choose a reason for hiding this comment

Anviking commented Aug 24, 2022

iohk-bors bot commented Aug 24, 2022 • edited by Anviking Loading

jonathanknowles commented Aug 24, 2022

iohk-bors bot commented Aug 24, 2022

iohk-bors bot commented Aug 24, 2022 • edited by jonathanknowles Loading

jonathanknowles commented Aug 24, 2022

iohk-bors bot commented Aug 24, 2022 • edited by jonathanknowles Loading

jonathanknowles commented Aug 25, 2022

Use `cardano-ledger` for all minimum UTxO calculations #3456

Use `cardano-ledger` for all minimum UTxO calculations #3456

jonathanknowles commented Aug 24, 2022 •

edited

Loading

iohk-bors bot commented Aug 24, 2022 •

edited by Anviking

Loading

iohk-bors bot commented Aug 24, 2022 •

edited by jonathanknowles

Loading

iohk-bors bot commented Aug 24, 2022 •

edited by jonathanknowles

Loading