Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimise query stake snapshot command #4179

Merged
merged 2 commits into from
Jan 7, 2023

Conversation

newhoggy
Copy link
Contributor

@newhoggy newhoggy commented Jul 13, 2022

@newhoggy newhoggy force-pushed the newhoggy/optimise-query-stake-snapshot-command branch from 3460124 to 197680d Compare July 20, 2022 00:38
@newhoggy
Copy link
Contributor Author

Before:

$ time CARDANO_NODE_SOCKET_PATH=/Users/jky/wrk/iohk/mainnet/cardano-node.socket cardano-cli query stake-snapshot --cardano-mode --mainnet --stake-pool-id fc79a939d5986c6565fc5e21bb5dd292261ac5da564c30385b3bb8d4

{
    "activeStakeGo": 24456415918186222,
    "activeStakeMark": 24453012636354912,
    "activeStakeSet": 24441961174746539,
    "poolStakeGo": 8595693051,
    "poolStakeMark": 8595693051,
    "poolStakeSet": 8595693051
}
CARDANO_NODE_SOCKET_PATH=/Users/jky/wrk/iohk/mainnet/cardano-node.socket       54.74s user 4.08s system 63% cpu 1:32.24 total

After:

$ time CARDANO_NODE_SOCKET_PATH=/Users/jky/wrk/iohk/mainnet/cardano-node.socket $(./scripts/bin-path.sh cardano-cli) query stake-snapshot --cardano-mode --mainnet --stake-pool-id fc79a939d5986c6565fc5e21bb5dd292261ac5da564c30385b3bb8d4

{
    "activeStakeGo": 24456415918186222,
    "activeStakeMark": 24453012636354912,
    "activeStakeSet": 24441961174746539,
    "poolStakeGo": 8595693051,
    "poolStakeMark": 8595693051,
    "poolStakeSet": 8595693051
}
CARDANO_NODE_SOCKET_PATH=/Users/jky/wrk/iohk/mainnet/cardano-node.socket       0.01s user 0.01s system 1% cpu 1.014 total

@newhoggy newhoggy force-pushed the newhoggy/optimise-query-stake-snapshot-command branch from 197680d to 1287358 Compare July 20, 2022 01:48
@newhoggy newhoggy marked this pull request as ready for review July 20, 2022 02:05
@newhoggy newhoggy force-pushed the newhoggy/optimise-query-stake-snapshot-command branch 2 times, most recently from fe68cd5 to 76e3bf6 Compare July 20, 2022 02:25
Copy link
Contributor

@Jimbo4350 Jimbo4350 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of minor changes

cardano-api/src/Cardano/Api/Orphans.hs Outdated Show resolved Hide resolved
cardano-api/src/Cardano/Api/Orphans.hs Outdated Show resolved Hide resolved
@newhoggy newhoggy force-pushed the newhoggy/optimise-query-stake-snapshot-command branch 2 times, most recently from 29b6b80 to 8143b24 Compare July 27, 2022 12:38
@newhoggy newhoggy requested a review from Jimbo4350 July 27, 2022 12:40
@newhoggy newhoggy dismissed Jimbo4350’s stale review July 27, 2022 12:40

comments addressed

Copy link
Contributor

@Jimbo4350 Jimbo4350 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Please squash the commits and you have some build errors

@newhoggy newhoggy force-pushed the newhoggy/optimise-query-stake-snapshot-command branch 3 times, most recently from 561072a to eafc49a Compare August 3, 2022 11:47
@newhoggy newhoggy force-pushed the newhoggy/optimise-query-stake-snapshot-command branch 2 times, most recently from 3f2f2e4 to 68b658c Compare August 5, 2022 18:01
@Jimbo4350 Jimbo4350 self-requested a review August 5, 2022 19:17
Copy link
Contributor

@Jimbo4350 Jimbo4350 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please squash the commits

@newhoggy newhoggy force-pushed the newhoggy/optimise-query-stake-snapshot-command branch from 68b658c to f3d7b64 Compare August 6, 2022 00:11
@newhoggy newhoggy dismissed Jimbo4350’s stale review August 6, 2022 00:11

Squashed commits

@newhoggy newhoggy requested a review from Jimbo4350 August 6, 2022 00:11
@newhoggy newhoggy force-pushed the newhoggy/optimise-query-stake-snapshot-command branch 2 times, most recently from 2936b7d to 535f703 Compare August 7, 2022 01:52
Copy link
Contributor

@Jimbo4350 Jimbo4350 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we implement something similar to:

data QueryUTxOFilter =
     -- | /O(n) time and space/ for utxo size n
     QueryUTxOWhole

     -- | /O(n) time, O(m) space/ for utxo size n, and address set size m
   | QueryUTxOByAddress (Set AddressAny)

     -- | /O(m log n) time, O(m) space/ for utxo size n, and address set size m
   | QueryUTxOByTxIn (Set TxIn)
  deriving (Eq, Show)

We can then account for the case where we are interested in single or multiple pool stake snapshots.

e.g

data QueryStakeSnapshotFilter = 
     QuerySinglePoolStakeSnapshot PoolId
   | QueryMutiplePoolStakeSnapshots (Set PoolId) 

And then we can pipe that through in a similar fashion to QueryUTxOFilter

cardano-api/src/Cardano/Api/KeysShelley.hs Outdated Show resolved Hide resolved
toJSON = object . stakeSnapshotsToPair
toEncoding = pairs . mconcat . stakeSnapshotsToPair

stakeSnapshotsToPair :: Aeson.KeyValue a => Consensus.StakeSnapshots crypto -> [a]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this logic shouldn't be in the JSON instance but at the point of rendering. The JSON instance should only be concerned with rendering JSON. In the cli we should determine how we render depending on if we get a Map of length 1 or more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's not a good way to do the fact there is a Map at all breaks backwards compatibility.

This PR is meant to preserve backwards compatibility.

A follow up PR simplifies this code but breaks backwards compatibility. #4279

It would be good to get an optimised version of this command merged first before merging the change that breaks compatibility.

Copy link
Contributor

@Jimbo4350 Jimbo4350 Jan 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just re-discovered data Stakes. This is what we were rendering. What we should do to maintain backwards compatibility is create a function with type signature Consensus.StakeSnapshots crypto -> Stakes. Imposing our cli logic on a JSON instance seems wrong, which should just be concerned with rendering JSON and should not be dictated by the needs of our query.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The data doesn't fit into data Stakes either because the map can be empty and I would not have anything to fill the pool fields in that case.

I've rendered the JSON manually instead.

@newhoggy newhoggy force-pushed the newhoggy/optimise-query-stake-snapshot-command branch from 82aef90 to 028d0dc Compare December 12, 2022 04:20
@newhoggy newhoggy dismissed Jimbo4350’s stale review December 12, 2022 04:20

Comments addressed.

@newhoggy
Copy link
Contributor Author

data QueryStakeSnapshotFilter =
     QuerySinglePoolStakeSnapshot PoolId
   | QueryMutiplePoolStakeSnapshots (Set PoolId)

I feel that QueryMutiplePoolStakeSnapshots subsumes QuerySinglePoolStakeSnapshot.

What's the case for adding an extra single pool id constructor?

@newhoggy newhoggy force-pushed the newhoggy/optimise-query-stake-snapshot-command branch from 028d0dc to ed897dc Compare December 22, 2022 03:43
@Jimbo4350 Jimbo4350 self-requested a review January 3, 2023 14:07
@newhoggy newhoggy force-pushed the newhoggy/optimise-query-stake-snapshot-command branch 2 times, most recently from 2b02875 to 3f95f83 Compare January 6, 2023 06:54
@newhoggy newhoggy force-pushed the newhoggy/optimise-query-stake-snapshot-command branch from 3f95f83 to 8cd88e5 Compare January 6, 2023 14:49
@IntersectMBO IntersectMBO deleted a comment Jan 6, 2023
@newhoggy newhoggy merged commit ee0890e into master Jan 7, 2023
@iohk-bors iohk-bors bot deleted the newhoggy/optimise-query-stake-snapshot-command branch January 7, 2023 02:10
@newhoggy newhoggy restored the newhoggy/optimise-query-stake-snapshot-command branch January 7, 2023 02:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

stake-snapshot crashes cardano-node
2 participants