Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

create-testnet-data: add option to create constitutional committee members #961

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions cardano-cli/src/Cardano/CLI/EraBased/Commands/Genesis.hs
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,8 @@ data GenesisCreateTestNetDataCmdArgs era = GenesisCreateTestNetDataCmdArgs
, numPools :: !Word
-- ^ The number of stake pools credentials to create and write to disk.
, stakeDelegators :: !StakeDelegators
-- ^ The number of members of the constitutional committee
, numCommitteeKeys :: !Word
-- ^ The number of delegators to pools and DReps to create.
, numDRepKeys :: !DRepCredentials
-- ^ The number of DRep keys to create. They are registered and get delegated to by stake delegators
Expand Down
10 changes: 10 additions & 0 deletions cardano-cli/src/Cardano/CLI/EraBased/Options/Genesis.hs
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,7 @@ pGenesisCreateTestNetData sbe envCli =
<*> pNumGenesisKeys
<*> pNumPools
<*> pNumStakeDelegs
<*> pNumCommittee
<*> pNumDReps
<*> pNumStuffedUtxoCount
<*> pNumUtxoKeys
Expand Down Expand Up @@ -258,6 +259,15 @@ pGenesisCreateTestNetData sbe envCli =
, Opt.help "The number of stake pool credential sets to make (default is 0)."
, Opt.value 0
]
pNumCommittee :: Parser Word
pNumCommittee =
Opt.option integralReader $
mconcat
[ Opt.long "committee-keys"
, Opt.metavar "INT"
, Opt.help "The number of constitutional committee credentials to make (default is 0)."
, Opt.value 0
]
pNumDReps :: Parser DRepCredentials
pNumDReps =
pDReps OnDisk "drep-keys" "Credentials are written to disk."
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,7 @@ where
import Cardano.Api hiding (ConwayEra)
import Cardano.Api.Ledger (StrictMaybe (SNothing))
import qualified Cardano.Api.Ledger as L
import Cardano.Api.Shelley
(Hash (DRepKeyHash, GenesisDelegateKeyHash, GenesisKeyHash, StakeKeyHash, VrfKeyHash),
KESPeriod (KESPeriod),
import Cardano.Api.Shelley (Hash (..), KESPeriod (KESPeriod),
OperationalCertificateIssueCounter (OperationalCertificateIssueCounter),
ShelleyGenesis (ShelleyGenesis, sgGenDelegs, sgInitialFunds, sgMaxLovelaceSupply, sgNetworkMagic, sgProtocolParams, sgStaking, sgSystemStart),
StakeCredential (StakeCredentialByKey), VerificationKey (VrfVerificationKey),
Expand All @@ -35,8 +33,10 @@ import Cardano.Api.Shelley

import qualified Cardano.CLI.Commands.Node as Cmd
import Cardano.CLI.EraBased.Commands.Genesis as Cmd
import qualified Cardano.CLI.EraBased.Commands.Governance.Committee as CC
import qualified Cardano.CLI.EraBased.Commands.Governance.DRep as DRep
import Cardano.CLI.EraBased.Run.Genesis.Common
import qualified Cardano.CLI.EraBased.Run.Governance.Committee as CC
import qualified Cardano.CLI.EraBased.Run.Governance.DRep as DRep
import Cardano.CLI.EraBased.Run.StakeAddress (runStakeAddressKeyGenCmd)
import qualified Cardano.CLI.IO.Lazy as Lazy
Expand All @@ -56,6 +56,7 @@ import Control.Monad (forM, forM_, unless, void, when)
import qualified Data.Aeson as Aeson
import Data.Bifunctor (Bifunctor (..))
import qualified Data.ByteString.Lazy.Char8 as LBS
import Data.Function ((&))
import Data.ListMap (ListMap (..))
import Data.Map.Strict (Map)
import qualified Data.Map.Strict as Map
Expand Down Expand Up @@ -181,6 +182,7 @@ runGenesisCreateTestNetDataCmd
{ stakeDelegatorsGenerationMode
, numOfStakeDelegators
}
, numCommitteeKeys
, numDRepKeys =
DRepCredentials
{ dRepCredentialGenerationMode
Expand Down Expand Up @@ -253,6 +255,26 @@ runGenesisCreateTestNetDataCmd

when (0 < numPools) $ writeREADME poolsDir poolsREADME

-- CC members
ccColdKeys <- forM [1 .. numCommitteeKeys] $ \index -> do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we not missing a transient vs on disk distinction here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Jimbo4350> Yes we do miss the distinction. I thought the transient case was not worth the extra effort for the constitutional committee, because I think having a committee with millions of members is not a case we likely want to spend resource testing.

WDYT? Do you want me to add a on-disk/transient distinction?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would check with @mgmeier

Copy link
Contributor

@mgmeier mgmeier Nov 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the optional nature of creating constitutional committee keys, and the expected number of keys being vastly lower than for instance for DReps, there's no need for a transient mode from Performance & Tracing.

let committeeDir = committeesDir </> "cc" <> show index
vkeyHotFile = File @(VerificationKey ()) $ committeeDir </> "cc.hot.vkey"
skeyHotFile = File @(SigningKey ()) $ committeeDir </> "cc.hot.skey"
vkeyColdFile = File @(VerificationKey ()) $ committeeDir </> "cc.cold.vkey"
skeyColdFile = File @(SigningKey ()) $ committeeDir </> "cc.cold.skey"
hotArgs = CC.GovernanceCommitteeKeyGenHotCmdArgs ConwayEraOnwardsConway vkeyHotFile skeyHotFile
coldArgs = CC.GovernanceCommitteeKeyGenColdCmdArgs ConwayEraOnwardsConway vkeyColdFile skeyColdFile
Comment on lines +265 to +266
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we not hardcode the era here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we can if we make create-testnet-data only Conway onwards. Which makes sense, because it was introduced when we were developing conway. Right now we are incorrect, because nothing in create-testnet-data is parameterized on the era (everything is Shelley onwards, including for example DReps).

@Jimbo4350> what do you think about making create-testnet-data only Conway onwards?

Copy link
Contributor

@Jimbo4350 Jimbo4350 Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would parameterize it on Era era. We actually need to deprecate the BabbageEra constructor of Era era.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created a follow-up issue to avoid mixing this change in this PR: #962

liftIO $ createDirectoryIfMissing True committeeDir
void $
withExceptT GenesisCmdGovernanceCommitteeError $
CC.runGovernanceCommitteeKeyGenHot hotArgs
(vColdKey, _) <-
withExceptT GenesisCmdGovernanceCommitteeError $
CC.runGovernanceCommitteeKeyGenCold coldArgs
return vColdKey

when (0 < numCommitteeKeys) $ writeREADME committeesDir committeeREADME

-- DReps
g <- Random.getStdGen

Expand Down Expand Up @@ -302,7 +324,9 @@ runGenesisCreateTestNetDataCmd
stuffedUtxoAddrs <-
liftIO $ Lazy.replicateM (fromIntegral numStuffedUtxo) $ genStuffedAddress network

let conwayGenesis' = addDRepsToConwayGenesis dRepKeys (map snd delegatorKeys) conwayGenesis
let conwayGenesis' =
addDRepsToConwayGenesis dRepKeys (map snd delegatorKeys) conwayGenesis
& addCommitteeToConwayGenesis ccColdKeys

let stake = second L.ppId . mkDelegationMapEntry <$> delegations
stakePools = [(L.ppId poolParams', poolParams') | poolParams' <- snd . mkDelegationMapEntry <$> delegations]
Expand All @@ -328,6 +352,7 @@ runGenesisCreateTestNetDataCmd
where
genesisDir = outputDir </> "genesis-keys"
delegateDir = outputDir </> "delegate-keys"
committeesDir = outputDir </> "cc-keys"
drepsDir = outputDir </> "drep-keys"
utxoKeysDir = outputDir </> "utxo-keys"
poolsDir = outputDir </> "pools-keys"
Expand All @@ -336,6 +361,28 @@ runGenesisCreateTestNetDataCmd
:: Delegation -> (L.KeyHash L.Staking L.StandardCrypto, L.PoolParams L.StandardCrypto)
mkDelegationMapEntry d = (dDelegStaking d, dPoolParams d)

addCommitteeToConwayGenesis
:: [VerificationKey CommitteeColdKey]
-> L.ConwayGenesis L.StandardCrypto
-> L.ConwayGenesis L.StandardCrypto
addCommitteeToConwayGenesis ccColdKeys conwayGenesis =
conwayGenesis
{ L.cgCommittee =
L.Committee
{ L.committeeMembers =
Map.fromList $ map ((,EpochNo maxBound) . toCommitteeColdCredential) ccColdKeys
, L.committeeThreshold = zeroUnitInterval -- Taken from cardano-testnet at the time of writing. Change to 0.5 in the future?
}
}
where
zeroUnitInterval = unsafeBoundedRational @L.UnitInterval 0
toCommitteeColdCredential
:: VerificationKey CommitteeColdKey -> L.Credential L.ColdCommitteeRole L.StandardCrypto
toCommitteeColdCredential vk = toCredential (verificationKeyHash vk)
where
toCredential :: Hash CommitteeColdKey -> L.Credential L.ColdCommitteeRole L.StandardCrypto
toCredential (CommitteeColdKeyHash v) = L.KeyHashObj v

addDRepsToConwayGenesis
:: [VerificationKey DRepKey]
-> [VerificationKey StakeKey]
Expand Down Expand Up @@ -434,6 +481,14 @@ genesisREADME =
, "With the introduction of Conway, these keys should become useless"
]

committeeREADME :: Text.Text
committeeREADME =
Text.intercalate
"\n"
[ "Keys generated by the --committee-keys flag. These keys are used to run the constitutional committee."
, "A pair of both cold keys and hot keys are generated for each committee member."
]

delegatesREADME :: Text.Text
delegatesREADME =
Text.intercalate
Expand Down
32 changes: 15 additions & 17 deletions cardano-cli/src/Cardano/CLI/EraBased/Run/Governance/Committee.hs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@

module Cardano.CLI.EraBased.Run.Governance.Committee
( runGovernanceCommitteeCmds
, runGovernanceCommitteeKeyGenCold
, runGovernanceCommitteeKeyGenHot
, GovernanceCommitteeError (..)
)
where
Expand All @@ -21,6 +23,7 @@ import Cardano.CLI.Types.Common (PotentiallyCheckedAnchor (..))
import Cardano.CLI.Types.Errors.GovernanceCommitteeError
import Cardano.CLI.Types.Key.VerificationKey

import Control.Monad (void)
import Data.ByteString (ByteString)
import qualified Data.ByteString.Char8 as BS
import Data.Function
Expand All @@ -31,9 +34,9 @@ runGovernanceCommitteeCmds
-> ExceptT GovernanceCommitteeError IO ()
runGovernanceCommitteeCmds = \case
GovernanceCommitteeKeyGenColdCmd cmd ->
runGovernanceCommitteeKeyGenCold cmd
void $ runGovernanceCommitteeKeyGenCold cmd
GovernanceCommitteeKeyGenHotCmd cmd ->
runGovernanceCommitteeKeyGenHot cmd
void $ runGovernanceCommitteeKeyGenHot cmd
GovernanceCommitteeKeyHashCmd cmd ->
runGovernanceCommitteeKeyHash cmd
GovernanceCommitteeCreateHotKeyAuthorizationCertificateCmd cmd ->
Expand All @@ -44,26 +47,25 @@ runGovernanceCommitteeCmds = \case
runGovernanceCommitteeKeyGenCold
:: ()
=> Cmd.GovernanceCommitteeKeyGenColdCmdArgs era
-> ExceptT GovernanceCommitteeError IO ()
-> ExceptT GovernanceCommitteeError IO (VerificationKey CommitteeColdKey, SigningKey CommitteeColdKey)
runGovernanceCommitteeKeyGenCold
Cmd.GovernanceCommitteeKeyGenColdCmdArgs
{ Cmd.vkeyOutFile = vkeyPath
, Cmd.skeyOutFile = skeyPath
} = do
skey <- generateSigningKey AsCommitteeColdKey

let vkey = getVerificationKey skey

writeLazyByteStringFile skeyPath (textEnvelopeToJSON (Just Key.ccColdSkeyDesc) skey)
& onLeft (left . GovernanceCommitteeCmdWriteFileError)
void $ firstExceptT GovernanceCommitteeCmdWriteFileError $ do
void $ writeLazyByteStringFile skeyPath (textEnvelopeToJSON (Just Key.ccColdSkeyDesc) skey)
writeLazyByteStringFile vkeyPath (textEnvelopeToJSON (Just Key.ccColdVkeyDesc) vkey)

writeLazyByteStringFile vkeyPath (textEnvelopeToJSON (Just Key.ccColdVkeyDesc) vkey)
& onLeft (left . GovernanceCommitteeCmdWriteFileError)
return (vkey, skey)

runGovernanceCommitteeKeyGenHot
:: ()
=> Cmd.GovernanceCommitteeKeyGenHotCmdArgs era
-> ExceptT GovernanceCommitteeError IO ()
-> ExceptT GovernanceCommitteeError IO (VerificationKey CommitteeHotKey, SigningKey CommitteeHotKey)
runGovernanceCommitteeKeyGenHot
Cmd.GovernanceCommitteeKeyGenHotCmdArgs
{ Cmd.eon = _eon
Expand All @@ -74,15 +76,11 @@ runGovernanceCommitteeKeyGenHot

let vkey = getVerificationKey skey

firstExceptT GovernanceCommitteeCmdWriteFileError
. newExceptT
$ writeLazyByteStringFile skeyPath
$ textEnvelopeToJSON (Just Key.ccHotSkeyDesc) skey
void $ firstExceptT GovernanceCommitteeCmdWriteFileError $ do
void $ writeLazyByteStringFile skeyPath $ textEnvelopeToJSON (Just Key.ccHotSkeyDesc) skey
writeLazyByteStringFile vkeyPath $ textEnvelopeToJSON (Just Key.ccHotVkeyDesc) vkey

firstExceptT GovernanceCommitteeCmdWriteFileError
. newExceptT
$ writeLazyByteStringFile vkeyPath
$ textEnvelopeToJSON (Just Key.ccHotVkeyDesc) vkey
return (vkey, skey)

data SomeCommitteeKey
= ACommitteeHotKey (VerificationKey CommitteeHotKey)
Expand Down
5 changes: 5 additions & 0 deletions cardano-cli/src/Cardano/CLI/Types/Errors/GenesisCmdError.hs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ where
import Cardano.Api

import Cardano.CLI.Byron.Genesis as Byron
import Cardano.CLI.EraBased.Run.Governance.Committee (GovernanceCommitteeError)
import Cardano.CLI.Types.Common
import Cardano.CLI.Types.Errors.AddressCmdError
import Cardano.CLI.Types.Errors.NodeCmdError
Expand All @@ -30,6 +31,7 @@ data GenesisCmdError
| GenesisCmdFilesNoIndex [FilePath]
| GenesisCmdGenesisFileDecodeError !FilePath !Text
| GenesisCmdGenesisFileError !(FileError ())
| GenesisCmdGovernanceCommitteeError !GovernanceCommitteeError
| GenesisCmdMismatchedGenesisKeyFiles [Int] [Int] [Int]
| GenesisCmdNodeCmdError !NodeCmdError
| GenesisCmdStakeAddressCmdError !StakeAddressCmdError
Expand Down Expand Up @@ -133,3 +135,6 @@ instance Error GenesisCmdError where
<> "."
<> "This is incorrect: the delegated supply should be less or equal to the total supply."
<> " Note that the total supply can either come from --total-supply or from the default template. Please fix what you use."
GenesisCmdGovernanceCommitteeError govCommitteeError ->
"Error during committee creation: "
<> prettyError govCommitteeError
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,9 @@ import qualified Hedgehog.Extras.Test.Golden as H
networkMagic :: Word32
networkMagic = 623

numCommitteeKeys :: Int
numCommitteeKeys = 7

numDReps :: Int
numDReps = 5

Expand Down Expand Up @@ -62,6 +65,8 @@ mkArguments outputDir =
, show numPools
, "--drep-keys"
, show numDReps
, "--committee-keys"
, show numCommitteeKeys
, -- Relays file specifies two relays, like the number of SPOs
"--relays"
, "test/cardano-cli-golden/files/input/shelley/genesis/relays.json"
Expand Down Expand Up @@ -137,6 +142,9 @@ golden_create_testnet_data mShelleyTemplate =
forM_ (L.sgsPools $ sgStaking shelleyGenesis) $ \pool ->
Seq.length (L.ppRelays pool) H.=== 1

actualNumCCs <- liftIO $ listDirectories $ outputDir </> "cc-keys"
length actualNumCCs H.=== numCommitteeKeys

actualNumDReps <- liftIO $ listDirectories $ outputDir </> "drep-keys"
length actualNumDReps H.=== numDReps

Expand All @@ -146,6 +154,8 @@ golden_create_testnet_data mShelleyTemplate =
conwayGenesis :: ConwayGenesis StandardCrypto <-
H.readJsonFileOk $ outputDir </> "conway-genesis.json"

length (L.committeeMembers $ cgCommittee conwayGenesis) H.=== numCommitteeKeys

length (cgInitialDReps conwayGenesis) H.=== numDReps

length (cgDelegs conwayGenesis) H.=== numStakeDelegs
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,33 @@
alonzo-genesis.json
cc-keys/README.md
cc-keys/cc1/cc.cold.skey
cc-keys/cc1/cc.cold.vkey
cc-keys/cc1/cc.hot.skey
cc-keys/cc1/cc.hot.vkey
cc-keys/cc2/cc.cold.skey
cc-keys/cc2/cc.cold.vkey
cc-keys/cc2/cc.hot.skey
cc-keys/cc2/cc.hot.vkey
cc-keys/cc3/cc.cold.skey
cc-keys/cc3/cc.cold.vkey
cc-keys/cc3/cc.hot.skey
cc-keys/cc3/cc.hot.vkey
cc-keys/cc4/cc.cold.skey
cc-keys/cc4/cc.cold.vkey
cc-keys/cc4/cc.hot.skey
cc-keys/cc4/cc.hot.vkey
cc-keys/cc5/cc.cold.skey
cc-keys/cc5/cc.cold.vkey
cc-keys/cc5/cc.hot.skey
cc-keys/cc5/cc.hot.vkey
cc-keys/cc6/cc.cold.skey
cc-keys/cc6/cc.cold.vkey
cc-keys/cc6/cc.hot.skey
cc-keys/cc6/cc.hot.vkey
cc-keys/cc7/cc.cold.skey
cc-keys/cc7/cc.cold.vkey
cc-keys/cc7/cc.hot.skey
cc-keys/cc7/cc.hot.vkey
conway-genesis.json
delegate-keys/README.md
delegate-keys/delegate1/kes.skey
Expand Down
7 changes: 7 additions & 0 deletions cardano-cli/test/cardano-cli-golden/files/golden/help.cli
Original file line number Diff line number Diff line change
Expand Up @@ -1202,6 +1202,7 @@ Usage: cardano-cli shelley genesis create-testnet-data [--spec-shelley FILEPATH]
[ --stake-delegators INT
| --transient-stake-delegators INT
]
[--committee-keys INT]
[ --drep-keys INT
| --transient-drep-keys INT
]
Expand Down Expand Up @@ -2274,6 +2275,7 @@ Usage: cardano-cli allegra genesis create-testnet-data [--spec-shelley FILEPATH]
[ --stake-delegators INT
| --transient-stake-delegators INT
]
[--committee-keys INT]
[ --drep-keys INT
| --transient-drep-keys INT
]
Expand Down Expand Up @@ -3344,6 +3346,7 @@ Usage: cardano-cli mary genesis create-testnet-data [--spec-shelley FILEPATH]
[ --stake-delegators INT
| --transient-stake-delegators INT
]
[--committee-keys INT]
[ --drep-keys INT
| --transient-drep-keys INT
]
Expand Down Expand Up @@ -4406,6 +4409,7 @@ Usage: cardano-cli alonzo genesis create-testnet-data [--spec-shelley FILEPATH]
[ --stake-delegators INT
| --transient-stake-delegators INT
]
[--committee-keys INT]
[ --drep-keys INT
| --transient-drep-keys INT
]
Expand Down Expand Up @@ -5485,6 +5489,7 @@ Usage: cardano-cli babbage genesis create-testnet-data [--spec-shelley FILEPATH]
[ --stake-delegators INT
| --transient-stake-delegators INT
]
[--committee-keys INT]
[ --drep-keys INT
| --transient-drep-keys INT
]
Expand Down Expand Up @@ -6840,6 +6845,7 @@ Usage: cardano-cli conway genesis create-testnet-data [--spec-shelley FILEPATH]
[ --stake-delegators INT
| --transient-stake-delegators INT
]
[--committee-keys INT]
[ --drep-keys INT
| --transient-drep-keys INT
]
Expand Down Expand Up @@ -8839,6 +8845,7 @@ Usage: cardano-cli latest genesis create-testnet-data [--spec-shelley FILEPATH]
[ --stake-delegators INT
| --transient-stake-delegators INT
]
[--committee-keys INT]
[ --drep-keys INT
| --transient-drep-keys INT
]
Expand Down
Loading
Loading