Fix leadership schedule for current on babbage #4106

newhoggy · 2022-06-28T10:24:53Z

No description provided.

newhoggy · 2022-06-30T02:06:45Z

$ cabal test cardano-testnet --enable-tests --test-show-details=direct --test-options '-p Babbage.leadership-schedule'
...
Test suite cardano-testnet-tests: RUNNING...
test/Spec.hs
  Spec
    Babbage
      leadership-schedule: OK (259.17s)
          ✓ leadership-schedule passed 1 test.

All 1 tests passed (259.17s)
Test suite cardano-testnet-tests: PASS
Test suite logged to:
/Users/jky/wrk/iohk/cardano-node/dist-newstyle/build/aarch64-osx/ghc-8.10.7/cardano-testnet-1.35.0/t/cardano-testnet-tests/test/cardano-testnet-1.35.0-cardano-testnet-tests.log
1 of 1 test suites (1 of 1 test cases) passed.

newhoggy · 2022-06-30T02:13:47Z

Resolves #3883

cardano-testnet/cardano-testnet.cabal

Addressed concerns

Jimbo4350

Nice 👍 A few comments. Can you also tidy up the commit history? It's a fairly big PR and it was difficult to follow the changes.

cardano-testnet/src/Testnet/Babbage.hs

Jimbo4350 · 2022-07-01T13:50:49Z

cardano-testnet/src/Testnet/Babbage.hs

+{- HLINT ignore "Use let" -}
+
+
+data TestnetOptions = TestnetOptions deriving (Eq, Show)


This is duplicated 3 more times in cardano-testnet. We should have a single TestnetOptions that is a sum type. And if there are no testnet options, we can use a constructor of that sum type to indicate this e.g NoTestnetOptions

Currently there are different functions for setting up different testnets and they take different types because they configurations they take are different. Unifying them is desirable, but it would take a lot of work. I think this PR is the wrong place to do it.

Ok fair enough. Can you create an issue to track this?

cardano-testnet/testnet/Testnet/Commands/Babbage.hs

cardano-testnet/src/Test/Runtime.hs

cardano-testnet/src/Test/Process.hs

cardano-testnet/test/Spec/Cli/Alonzo/LeadershipSchedule.hs

cardano-testnet/cardano-testnet.cabal

cardano-testnet/src/Testnet/Babbage.hs

Jimbo4350 · 2022-07-01T14:11:35Z

cardano-testnet/test/Spec/Cli/Babbage/LeadershipSchedule.hs

@@ -0,0 +1,202 @@
+{-# LANGUAGE BlockArguments           #-}


I discovered it was an interesting way to use layout in expressions instead of parenthesis.

For example the expression:

H.rewriteYamlFile (tempAbsPath </> "configuration.yaml") . J.rewriteObject $ HM.delete "GenesisFile" . HM.insert "Protocol" (J.toJSON @String "Cardano") ... . HM.insert "TestEnableDevelopmentNetworkProtocols" (J.toJSON True) . flip HM.alter "setupScribes" ( fmap ( J.rewriteArrayElements ( J.rewriteObject ( HM.insert "scFormat" ( case nodeLoggingFormat testnetOptions of NodeLoggingFormatAsJson -> "ScJson" NodeLoggingFormatAsText -> "ScText")))))

Can be written as:

H.rewriteYamlFile (tempAbsPath </> "configuration.yaml") . J.rewriteObject $ HM.delete "GenesisFile" . HM.insert "Protocol" (J.toJSON @String "Cardano") ... . HM.insert "TestEnableDevelopmentNetworkProtocols" (J.toJSON True) . flip HM.alter "setupScribes" do fmap do J.rewriteArrayElements do J.rewriteObject do HM.insert "scFormat" case nodeLoggingFormat testnetOptions of NodeLoggingFormatAsJson -> "ScJson" NodeLoggingFormatAsText -> "ScText"

This works because do x y z is the same as x y z. This is true even without BlockArguments.

What BlockArguments gives us is that in the expression f $ x y z can be rewritten as f do x y z.

This gives us two nice things:

We can use do to use layout to group expressions, which means we don't have to match parens anymore. When we write Haskell, we usually indent sub-expressions to help the reader read the code anyway. The parentheses are for the compiler because layout is not significant to it inside an expression. BlockArguments allows code to be written such that the mechanism humans use to group expressions and the compiler uses to group expressions can be one and the same.

We normally think of $ and swappable with a pair of (). This is not always true. The ability to interchange them in this $ is unreliable because it is an operator and operators have a precedence and sometimes it is the wrong precedence.

For example if you tried to write the large expression from before using $ instead it is actually wrong and doesn't compile:

H.rewriteYamlFile (tempAbsPath </> "configuration.yaml") . J.rewriteObject $ HM.delete "GenesisFile" . HM.insert "Protocol" (J.toJSON @String "Cardano") ... . HM.insert "TestEnableDevelopmentNetworkProtocols" (J.toJSON True) . flip HM.alter "setupScribes" $ fmap $ J.rewriteArrayElements $ J.rewriteObject $ HM.insert "scFormat" $ case nodeLoggingFormat testnetOptions of NodeLoggingFormatAsJson -> "ScJson" NodeLoggingFormatAsText -> "ScText"

Similarly, f $ case { x -> y } can be rewritten without the $ as f case {x -> y}.

And f $ \x -> y can be rewritten without the $ as f \x -> y.

Basically there are a number of cases where when we previously used $, it can simply be dropped.

You could also write it as follows:

(fmap . J.rewriteArrayElements . J.rewriteObject . HM.insert "scFormat" $ case nodeLoggingFormat testnetOptions of NodeLoggingFormatAsJson -> "ScJson" NodeLoggingFormatAsText -> "ScText")

The problem I have with BlockArguments is that it makes it more difficult to discern between monadic actions and function composition. For this reason I avoid using it because I value that more than slightly more convenient code formatting.

I've taken your suggestion, as it's really only two places.

Mainly I thought that BlockArguments has some really interesting properties and that it is a better $ and wanted to shared the idea.

At the place I worked previously, we used block argument do for error handling in monadic contexts, so perhaps we can revisit this.

we used block argument do for error handling in monadic contexts

Do you have an example of this I can look at?

I do, but I'll figure out an appropriate time to share it. It can wait until some urgent features are done.

cardano-api/src/Cardano/Api/LedgerState.hs

.stylish-haskell.yaml

Review comments

nfrisby

I'm in a call, but I skimmed through 62082a4, which I think is the only Consensus-related commit.

It looks great, from what I can see. 👏 One minor question below, but low priority, I think. (I'm not Approving, because I only looked at so little of this PR, but I this code looks as expected. 👍 )

PS - The commit 3ffe9ee is Fix leadership schedule for current on babbage, which sounded like what I was looking for, but I think it's just maybe getting a test to build again? I'd suggest a commit title that clarifies that eg.

nfrisby · 2022-07-07T14:38:55Z

cardano-api/src/Cardano/Api/LedgerState.hs

-   Crypto.certifiedOutput
-   $ Crypto.evalCertified () (TPraos.mkSeed TPraos.seedL slot epochNonce) vrfSkey'
+  -> Either LeadershipError (Set SlotNo)
+isLeadingSlotsTPraos slotRangeOfInterest poolid snapshotPoolDistr eNonce vrfSkey activeSlotCoeff' = do


Nice simplifications! Thanks for discerning which parts of the Consensus API you could use here 👍

nfrisby · 2022-07-07T14:40:31Z

cardano-api/src/Cardano/Api/LedgerState.hs

-                         (Right . ShelleyAPI.individualPoolStake)
-                         (Map.lookup poolHash setSnapshotPoolDistr)
+  let slotRangeOfInterest = Set.filter
+        (not . Ledger.isOverlaySlot firstSlotOfEpoch (getField @"_d" (toLedgerPParams sbe pParams)))


I think we (eventually?) won't have d at all for Babbage/Praos, right? Will this require an update then?

There is no longer an overlay schedule in Praos, so this line could be simplified to just return the set of slots in the given epoch.

(As for removing d, it is not in the Babbage era protocol parameters, but the HasField instance still exists, sadly, since d has its tendrils all throughout the reward calculation and we chose not to redo all that code.)

Since we need to keep it for TPraos anyway, I'm happy to keep this code for the moment and be guided by the type errors if/when the d parameters get deleted for Babbage onwards.

…rdano testnet

newhoggy · 2022-07-08T00:48:03Z

bors r+

iohk-bors · 2022-07-08T01:15:11Z

Build succeeded:

newhoggy force-pushed the newhoggy/fix-leadership-schedule-for-babbage branch from 645287f to 8dcf6f5 Compare June 28, 2022 11:37

newhoggy changed the title ~~Newhoggy/fix leadership schedule for babbage~~ Fix leadership schedule for current on babbage Jun 28, 2022

newhoggy marked this pull request as ready for review June 28, 2022 11:37

newhoggy requested review from dcoutts, Jimbo4350, erikd and JaredCorduan as code owners June 28, 2022 11:37

newhoggy force-pushed the newhoggy/fix-leadership-schedule-for-babbage branch 3 times, most recently from cfe0475 to 823fddb Compare June 30, 2022 01:56

This was referenced Jun 30, 2022

[BUG] - vasil-testnet-v1 - CLI queries "kes-period-info", "protocol-state" and "leadership-schedule" are broken #3883

Closed

[BUG] - query leadership-chedule reports wrong slots on Babbage #4017

Closed

Jimbo4350 previously requested changes Jun 30, 2022

View reviewed changes

cardano-testnet/cardano-testnet.cabal Outdated Show resolved Hide resolved

newhoggy requested a review from Jimbo4350 June 30, 2022 20:47

newhoggy force-pushed the newhoggy/fix-leadership-schedule-for-babbage branch 5 times, most recently from 736666c to 3e4f8ca Compare June 30, 2022 22:38

Jimbo4350 previously requested changes Jul 1, 2022

View reviewed changes

newhoggy commented Jul 2, 2022

View reviewed changes

.stylish-haskell.yaml Show resolved Hide resolved

newhoggy force-pushed the newhoggy/fix-leadership-schedule-for-babbage branch 2 times, most recently from 7acc550 to bcec4d4 Compare July 2, 2022 07:27

newhoggy requested a review from Jimbo4350 July 3, 2022 04:59

newhoggy force-pushed the newhoggy/fix-leadership-schedule-for-babbage branch 2 times, most recently from 9fdd4fb to 85068a6 Compare July 3, 2022 06:26

newhoggy force-pushed the newhoggy/fix-leadership-schedule-for-babbage branch from b1e0a82 to 9265bcd Compare July 6, 2022 23:34

nfrisby reviewed Jul 7, 2022

View reviewed changes

newhoggy added 18 commits July 8, 2022 10:16

Add isLinux to Test.Base

ea5b739

Remove configurability of TestEnableDevelopmentNetworkProtocols in ca…

86930db

…rdano testnet

Remove NoImplicitPrelude from .stylish-haskell.yml

1dc1d7b

Babbage testnet

45ddadd

Export wallets

b07f0e2

Add command line options for babbage testnet

e6a052c

Common runtime types for cardan-testnet

7baebc7

New assertByDeadlineMCustom function

97fe74c

Add key pairs to testnet runtime.

48b3f75

Add type signatures

1b8246d

Node logging format configuration for babbage

7473877

New LeadershipSlot type for tests

0ed37bf

Add support for JSON logging to cardano testnet

030f843

New Alonzo leadership schedule test

de31768

New Babbage leadership schedule test

c2a48fa

Fix query leadership-schedule for current on Babbage.

e1ff7db

Remove lens

e16d65e

Disable kes-period-info test with code rather than comments

207cb9f

newhoggy force-pushed the newhoggy/fix-leadership-schedule-for-babbage branch from 9265bcd to 207cb9f Compare July 8, 2022 00:17

iohk-bors bot merged commit 1b91216 into master Jul 8, 2022

iohk-bors bot deleted the newhoggy/fix-leadership-schedule-for-babbage branch July 8, 2022 01:15

CarlosLopezDeLara mentioned this pull request Jul 22, 2022

[BUG] - VASIL cardano-cli query leadership-schedule Error (Next Epoch) #4131

Closed

CarlosLopezDeLara mentioned this pull request Aug 2, 2022

1.35.3 SCOPE #4257

Open

12 tasks

CarlosLopezDeLara mentioned this pull request Oct 6, 2022

[BUG] - leaderlog error with hardfork transition epoch #4478

Closed

CarlosLopezDeLara mentioned this pull request Dec 6, 2022

[BUG] - leadership-schedule not match with real situation #4583

Closed

amesgen mentioned this pull request Jan 11, 2023

Block fits onto some fork IntersectMBO/ouroboros-network#4253

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix leadership schedule for current on babbage #4106

Fix leadership schedule for current on babbage #4106

newhoggy commented Jun 28, 2022

newhoggy commented Jun 30, 2022

newhoggy commented Jun 30, 2022

Jimbo4350 left a comment

Jimbo4350 Jul 1, 2022

newhoggy Jul 2, 2022

Jimbo4350 Jul 4, 2022

newhoggy Jul 5, 2022

Jimbo4350 Jul 1, 2022

newhoggy Jul 3, 2022 •

edited

Loading

Jimbo4350 Jul 4, 2022 •

edited

Loading

newhoggy Jul 5, 2022

Jimbo4350 Jul 5, 2022

newhoggy Jul 6, 2022

nfrisby left a comment

nfrisby Jul 7, 2022

nfrisby Jul 7, 2022

JaredCorduan Jul 7, 2022

newhoggy Jul 8, 2022

newhoggy commented Jul 8, 2022

iohk-bors bot commented Jul 8, 2022

		{- HLINT ignore "Use let" -}


		data TestnetOptions = TestnetOptions deriving (Eq, Show)

Fix leadership schedule for current on babbage #4106

Fix leadership schedule for current on babbage #4106

Conversation

newhoggy commented Jun 28, 2022

newhoggy commented Jun 30, 2022

newhoggy commented Jun 30, 2022

Jimbo4350 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

newhoggy Jul 3, 2022 • edited Loading

Choose a reason for hiding this comment

Jimbo4350 Jul 4, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nfrisby left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

newhoggy commented Jul 8, 2022

iohk-bors bot commented Jul 8, 2022

newhoggy Jul 3, 2022 •

edited

Loading

Jimbo4350 Jul 4, 2022 •

edited

Loading