-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
write a simulation for POA to get off SDK fork #170
Comments
May not get us off a fork anyways since andrew needs Either way simulation is a good thing to add |
Hey, I started writing simulation support in Manifest and encountered I'm happy to help! |
This is a POA error in the ante. You can not directly use the StakingKeeper as flow is expected through POA. May not be needed i have a POA refactor idea I am working on to remove some of the pitfalls with SDK stuff I ran into |
Sounds good. You might already know this, but this repo's ( |
Yes. I did not re-wire any of the sims due to not using it. The new design will be many times easier to sim as well (hopefully) |
Can I help with anything @Reecepbcups? |
Hey @Reecepbcups, I just wanted to follow up on this issue quickly. Is there a workaround that I could use to write simulator tests on our POA chain? |
I've made some progress on this. I was able to run the simulator by disabling
This approach will set the weight of the Note: The |
I made some progress on the PR. I'm currently implementing msg := poatypes.MsgSetPower{
Sender: admin,
ValidatorAddress: validator.OperatorAddress,
Power: 1_000_000,
Unsafe: false,
} The panic log is :42PM ERR panic recovered in runTx err="recovered: calculated final stake for delegator cosmos1mf03f7gqya7u98fhmfh4kmj6qhsafefjdesges greater than current stake\n\tfinal stake:\t161933052469.000000000000000000\n\tcurrent stake:\t1000000.000000000000000000\nstack:\ngoroutine 13 [running]:\nruntime/debug.Stack()\n\t/home/fmorency/go1.22.2/src/runtime/debug/stack.go:24 +0x5e\ngithub.com/cosmos/cosmos-sdk/baseapp.NewBaseApp.newDefaultRecoveryMiddleware.func5({0x26cb360, 0xc0017fe420})\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/recovery.go:74 +0x25\ngithub.com/cosmos/cosmos-sdk/baseapp.NewBaseApp.newDefaultRecoveryMiddleware.newRecoveryMiddleware.func7({0x26cb360?, 0xc0017fe420?})\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/recovery.go:42 +0x2d\ngithub.com/cosmos/cosmos-sdk/baseapp.processRecovery({0x26cb360, 0xc0017fe420}, 0x0?)\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/recovery.go:31 +0x2f\ngithub.com/cosmos/cosmos-sdk/baseapp.processRecovery({0x26cb360, 0xc0017fe420}, 0x0?)\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/recovery.go:36 +0x53\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx.func1()\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/baseapp.go:837 +0x156\npanic({0x26cb360?, 0xc0017fe420?})\n\t/home/fmorency/go1.22.2/src/runtime/panic.go:770 +0x132\ngithub.com/cosmos/cosmos-sdk/x/distribution/keeper.Keeper.CalculateDelegationRewards({{0x3a80b80, 0xc000cbaf30}, {0x3acee90, 0xc001029aa0}, {0x3ab9660, 0xc000cc8f00}, {0x7fa490023ba0, 0xc000f5c508}, {0x3acd288, 0xc001027e00}, ...}, ...)\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/distribution/keeper/delegation.go:177 +0x894\ngithub.com/cosmos/cosmos-sdk/x/distribution/keeper.Keeper.withdrawDelegationRewards({{0x3a80b80, 0xc000cbaf30}, {0x3acee90, 0xc001029aa0}, {0x3ab9660, 0xc000cc8f00}, {0x7fa490023ba0, 0xc000f5c508}, {0x3acd288, 0xc001027e00}, ...}, ...)\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/distribution/keeper/delegation.go:222 +0x26b\ngithub.com/cosmos/cosmos-sdk/x/distribution/keeper.Keeper.WithdrawDelegationRewards({{0x3a80b80, 0xc000cbaf30}, {0x3acee90, 0xc001029aa0}, {0x3ab9660, 0xc000cc8f00}, {0x7fa490023ba0, 0xc000f5c508}, {0x3acd288, 0xc001027e00}, ...}, ...)\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/distribution/keeper/keeper.go:127 +0x15e\ngithub.com/cosmos/cosmos-sdk/x/distribution/keeper.msgServer.WithdrawDelegatorReward({{{0x3a80b80, 0xc000cbaf30}, {0x3acee90, 0xc001029aa0}, {0x3ab9660, 0xc000cc8f00}, {0x7fa490023ba0, 0xc000f5c508}, {0x3acd288, 0xc001027e00}, ...}}, ...)\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/distribution/keeper/msg_server.go:59 +0x269\ngithub.com/cosmos/cosmos-sdk/x/distribution/types._Msg_WithdrawDelegatorReward_Handler.func1({0x3ab57f8?, 0xc002b65020?}, {0x2af3540?, 0xc002b547c0?})\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/distribution/types/tx.pb.go:1173 +0xcb\ngithub.com/cosmos/cosmos-sdk/baseapp.(*MsgServiceRouter).registerMsgServiceHandler.func2.1({0x3ab56a8, 0xc002b6ae08}, {0xc001a6de00?, 0x411265?}, 0x358?, 0xc000cc52f0)\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/msg_service_router.go:175 +0x8a\ngithub.com/cosmos/cosmos-sdk/x/distribution/types._Msg_WithdrawDelegatorReward_Handler({0x2c33720, 0xc0006c4a00}, {0x3ab56a8, 0xc002b6ae08}, 0x3429498, 0xc002b549c0)\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/distribution/types/tx.pb.go:1175 +0x143\ngithub.com/cosmos/cosmos-sdk/baseapp.(*MsgServiceRouter).registerMsgServiceHandler.func2({{0x3ab5600, 0x553c6c0}, {0x3acf750, 0xc0018973c0}, {{0x0, 0x0}, {0x2c5b432, 0xe}, 0x7, {0x0, ...}, ...}, ...}, ...)\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/msg_service_router.go:198 +0x37f\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runMsgs(_, {{0x3ab5600, 0x553c6c0}, {0x3acf750, 0xc0018973c0}, {{0x0, 0x0}, {0x2c5b432, 0xe}, 0x7, ...}, ...}, ...)\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/baseapp.go:1010 +0x1e6\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc001016908, 0x7, {0xc00404bb80, 0x149, 0x149})\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/baseapp.go:948 +0x114c\ngithub.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).SimDeliver(0xc001016908, 0x2c5b432?, {0x3a926e8?, 0xc00156b680?})\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/test_helpers.go:38 +0xa5\ngithub.com/cosmos/cosmos-sdk/x/simulation.GenAndDeliverTx({0xc002796ab0, 0xc001016908, {0x3acd648, 0xc001977c40}, 0x0, {0x3a9e470, 0xc002b54640}, {0xc000cc4b70, 0x1, 0x1}, ...}, ...)\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/simulation/util.go:117 +0x2f2\ngithub.com/cosmos/cosmos-sdk/x/simulation.GenAndDeliverTxWithRandFees({0xc002796ab0, 0xc001016908, {0x3acd648, 0xc001977c40}, 0x0, {0x3a9e470, 0xc002b54640}, {0xc000cc4b70, 0x1, 0x1}, ...})\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/simulation/util.go:96 +0x2d1\ngithub.com/cosmos/cosmos-sdk/x/distribution/simulation.WeightedOperations.SimulateMsgWithdrawDelegatorReward.func6(_, _, {{0x3ab5600, 0x553c6c0}, {0x3acf750, 0xc0019292c0}, {{0x0, 0x0}, {0x2c5b432, 0xe}, ...}, ...}, ...)\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/distribution/simulation/operations.go:171 +0xad0\ngithub.com/cosmos/cosmos-sdk/x/simulation.createBlockSimulator.func1(_, _, {{0x3ab5600, 0x553c6c0}, {0x3acf750, 0xc0019292c0}, {{0x0, 0x0}, {0x2c5b432, 0xe}, ...}, ...}, ...)\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/simulation/simulate.go:326 +0x512\ngithub.com/cosmos/cosmos-sdk/x/simulation.SimulateFromSeed({0x3af52b0, 0xc00103cb60}, {_, _}, _, _, _, {0xc001bf6008, 0x34, 0x3f}, ...)\n\t/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/simulation/simulate.go:210 +0x19e4\ngithub.com/strangelove-ventures/poa/simapp.TestFullAppSimulation(0xc00103cb60)\n\t/home/fmorency/dev/poa/simapp/sim_test.go:82 +0x816\ntesting.tRunner(0xc00103cb60, 0x34289b8)\n\t/home/fmorency/go1.22.2/src/testing/testing.go:1689 +0xfb\ncreated by testing.(*T).Run in goroutine 1\n\t/home/fmorency/go1.22.2/src/testing/testing.go:1742 +0x390\n: panic"
Logs to writing to /home/fmorency/.simapp/simulations/1718394173108.log
simulate.go:335: error on block 7/2000, operation (47/273) from x/distribution:
recovered: calculated final stake for delegator cosmos1mf03f7gqya7u98fhmfh4kmj6qhsafefjdesges greater than current stake
final stake: 161933052469.000000000000000000
current stake: 1000000.000000000000000000
stack:
goroutine 13 [running]:
runtime/debug.Stack()
/home/fmorency/go1.22.2/src/runtime/debug/stack.go:24 +0x5e
github.com/cosmos/cosmos-sdk/baseapp.NewBaseApp.newDefaultRecoveryMiddleware.func5({0x26cb360, 0xc0017fe420})
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/recovery.go:74 +0x25
github.com/cosmos/cosmos-sdk/baseapp.NewBaseApp.newDefaultRecoveryMiddleware.newRecoveryMiddleware.func7({0x26cb360?, 0xc0017fe420?})
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/recovery.go:42 +0x2d
github.com/cosmos/cosmos-sdk/baseapp.processRecovery({0x26cb360, 0xc0017fe420}, 0x0?)
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/recovery.go:31 +0x2f
github.com/cosmos/cosmos-sdk/baseapp.processRecovery({0x26cb360, 0xc0017fe420}, 0x0?)
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/recovery.go:36 +0x53
github.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx.func1()
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/baseapp.go:837 +0x156
panic({0x26cb360?, 0xc0017fe420?})
/home/fmorency/go1.22.2/src/runtime/panic.go:770 +0x132
github.com/cosmos/cosmos-sdk/x/distribution/keeper.Keeper.CalculateDelegationRewards({{0x3a80b80, 0xc000cbaf30}, {0x3acee90, 0xc001029aa0}, {0x3ab9660, 0xc000cc8f00}, {0x7fa490023ba0, 0xc000f5c508}, {0x3acd288, 0xc001027e00}, ...}, ...)
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/distribution/keeper/delegation.go:177 +0x894
github.com/cosmos/cosmos-sdk/x/distribution/keeper.Keeper.withdrawDelegationRewards({{0x3a80b80, 0xc000cbaf30}, {0x3acee90, 0xc001029aa0}, {0x3ab9660, 0xc000cc8f00}, {0x7fa490023ba0, 0xc000f5c508}, {0x3acd288, 0xc001027e00}, ...}, ...)
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/distribution/keeper/delegation.go:222 +0x26b
github.com/cosmos/cosmos-sdk/x/distribution/keeper.Keeper.WithdrawDelegationRewards({{0x3a80b80, 0xc000cbaf30}, {0x3acee90, 0xc001029aa0}, {0x3ab9660, 0xc000cc8f00}, {0x7fa490023ba0, 0xc000f5c508}, {0x3acd288, 0xc001027e00}, ...}, ...)
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/distribution/keeper/keeper.go:127 +0x15e
github.com/cosmos/cosmos-sdk/x/distribution/keeper.msgServer.WithdrawDelegatorReward({{{0x3a80b80, 0xc000cbaf30}, {0x3acee90, 0xc001029aa0}, {0x3ab9660, 0xc000cc8f00}, {0x7fa490023ba0, 0xc000f5c508}, {0x3acd288, 0xc001027e00}, ...}}, ...)
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/distribution/keeper/msg_server.go:59 +0x269
github.com/cosmos/cosmos-sdk/x/distribution/types._Msg_WithdrawDelegatorReward_Handler.func1({0x3ab57f8?, 0xc002b65020?}, {0x2af3540?, 0xc002b547c0?})
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/distribution/types/tx.pb.go:1173 +0xcb
github.com/cosmos/cosmos-sdk/baseapp.(*MsgServiceRouter).registerMsgServiceHandler.func2.1({0x3ab56a8, 0xc002b6ae08}, {0xc001a6de00?, 0x411265?}, 0x358?, 0xc000cc52f0)
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/msg_service_router.go:175 +0x8a
github.com/cosmos/cosmos-sdk/x/distribution/types._Msg_WithdrawDelegatorReward_Handler({0x2c33720, 0xc0006c4a00}, {0x3ab56a8, 0xc002b6ae08}, 0x3429498, 0xc002b549c0)
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/distribution/types/tx.pb.go:1175 +0x143
github.com/cosmos/cosmos-sdk/baseapp.(*MsgServiceRouter).registerMsgServiceHandler.func2({{0x3ab5600, 0x553c6c0}, {0x3acf750, 0xc0018973c0}, {{0x0, 0x0}, {0x2c5b432, 0xe}, 0x7, {0x0, ...}, ...}, ...}, ...)
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/msg_service_router.go:198 +0x37f
github.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runMsgs(_, {{0x3ab5600, 0x553c6c0}, {0x3acf750, 0xc0018973c0}, {{0x0, 0x0}, {0x2c5b432, 0xe}, 0x7, ...}, ...}, ...)
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/baseapp.go:1010 +0x1e6
github.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).runTx(0xc001016908, 0x7, {0xc00404bb80, 0x149, 0x149})
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/baseapp.go:948 +0x114c
github.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).SimDeliver(0xc001016908, 0x2c5b432?, {0x3a926e8?, 0xc00156b680?})
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/baseapp/test_helpers.go:38 +0xa5
github.com/cosmos/cosmos-sdk/x/simulation.GenAndDeliverTx({0xc002796ab0, 0xc001016908, {0x3acd648, 0xc001977c40}, 0x0, {0x3a9e470, 0xc002b54640}, {0xc000cc4b70, 0x1, 0x1}, ...}, ...)
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/simulation/util.go:117 +0x2f2
github.com/cosmos/cosmos-sdk/x/simulation.GenAndDeliverTxWithRandFees({0xc002796ab0, 0xc001016908, {0x3acd648, 0xc001977c40}, 0x0, {0x3a9e470, 0xc002b54640}, {0xc000cc4b70, 0x1, 0x1}, ...})
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/simulation/util.go:96 +0x2d1
github.com/cosmos/cosmos-sdk/x/distribution/simulation.WeightedOperations.SimulateMsgWithdrawDelegatorReward.func6(_, _, {{0x3ab5600, 0x553c6c0}, {0x3acf750, 0xc0019292c0}, {{0x0, 0x0}, {0x2c5b432, 0xe}, ...}, ...}, ...)
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/distribution/simulation/operations.go:171 +0xad0
github.com/cosmos/cosmos-sdk/x/simulation.createBlockSimulator.func1(_, _, {{0x3ab5600, 0x553c6c0}, {0x3acf750, 0xc0019292c0}, {{0x0, 0x0}, {0x2c5b432, 0xe}, ...}, ...}, ...)
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/simulation/simulate.go:326 +0x512
github.com/cosmos/cosmos-sdk/x/simulation.SimulateFromSeed({0x3af52b0, 0xc00103cb60}, {_, _}, _, _, _, {0xc001bf6008, 0x34, 0x3f}, ...)
/home/fmorency/go/pkg/mod/github.com/rollchains/[email protected]/x/simulation/simulate.go:210 +0x19e4
github.com/strangelove-ventures/poa/simapp.TestFullAppSimulation(0xc00103cb60)
/home/fmorency/dev/poa/simapp/sim_test.go:82 +0x816
testing.tRunner(0xc00103cb60, 0x34289b8)
/home/fmorency/go1.22.2/src/testing/testing.go:1689 +0xfb
created by testing.(*T).Run in goroutine 1
/home/fmorency/go1.22.2/src/testing/testing.go:1742 +0x390
: panic [rollchains/[email protected]/baseapp/recovery.go:72]
Comment: unable to deliver tx You can reproduce by running go test ./simapp -run TestFullAppSimulation \
-Enabled=true \
-NumBlocks=2000 \
-Genesis= \
-Verbose=true \
-Commit=true \
-Period=5 \
-Params=/path/to/sim_params.json \
-Seed=12 \
-v \
-timeout 24h against my PR at commit 5558615. Any idea what's going on? Do I need to craft a non-random genesis file to make it work? |
@fmorency just got back from travel so I do not have much time to look into this yet. Just throwing down quick ideas: 1)) rewards auto claim in cosmos (changing delegations auto claims rewards) so it may be changing the amounts since I have Distribution & Mint enabled in the app (doubt it but yea it can get you)
|
@Reecepbcups Thank you for your response. I disabled I don't fully understand the problem but I'm concerned that we could get such a crash in production. WDYT? |
Hey @Reecepbcups, I'm currently debugging The function My feeling is that I need to use Thanks! |
Hey @Reecepbcups, PR #186 is ready for a first round of comments.
Thanks! |
I've been debugging this thing. Here's an update.
I'll continue debugging tomorrow. Edit: Oh, it's the |
I think I fixed all the crashes. 4x simulator had been running in a loop for a while now. I removed the Here's the {
"op_weight_msg_delegate": 0,
"op_weight_msg_undelegate": 0,
"op_weight_msg_begin_redelegate": 0,
"op_weight_msg_cancel_unbonding_delegation": 0,
"op_weight_msg_create_validator": 0,
"op_weight_msg_edit_validator": 0,
"inflation": "0.000000000000000000",
"annual_provisions": "0.000000000000000000",
"inflation_rate_change": "0.000000000000000000",
"inflation_max": "0.000000000000000000",
"inflation_min": "0.000000000000000000",
"goal_bonded": "0.670000000000000000",
"blocks_per_year": "6311520",
"slash_fraction_double_sign": "0.000000000000000000",
"slash_fraction_downtime": "0.000000000000000000",
"community_tax": "0.0"
} |
I caught one crash caused by cosmos/cosmos-sdk#18196 which should be fixed in the next CosmosSDK version. |
I caught a couple of |
Backported to 0.50.x in cosmos/cosmos-sdk#20897 |
@fmorency I'm not sure yet but I assume there is some unentended behavior yea. Since It’s a core sdk module, there is some logic that probably expects it at height 1. There may also be use cases where someone wants to distribute rewards to validators on POA network. Not sure if the SDK team will consider it fully sim-ed or not with a core distribution module being removed (which staking uses in their upstream sims no problem) |
Thanks for your feedback.
I agree. However, the current state of the POA module is that I believe it's unsafe to use if the |
The where the stakes are truncated when multiplied by slash fractions. The sanity check makes sure that the stakes (minus the slash fraction) agree with the tokens obtained from the delegation's shares. When a POA validator power is updated, e.g., using Lines 19 to 46 in da5cc79
When calculating delegation rewards // x/distribution/keeper/delegation.go
startingInfo, err := k.GetDelegatorStartingInfo(ctx, sdk.ValAddress(valAddr), sdk.AccAddress(delAddr))
...
stake := startingInfo.Stake
...
currentStake := val.TokensFromShares(del.GetShares())
... the The error will trigger if |
Backported to 0.50.x in cosmos/cosmos-sdk#20909 |
I ran simulations in a loop overnight using the PR with the I'm afraid I can't continue this work until I get some help from a POA/CosmosSDK dev. |
So if a val has tokens delegated, and they delegate more, they are auto claimed. So another option here is reduce minting to 0%, then distribution is effectively disabled without removing it (ideal) since this is for simulation anyways. this way stake should remain the same and never auto claimed as values are updated and set etc |
I believe minting is already set to 0% in the simulation parameter file, or are you talking about something else? |
|
As discussed on Discord, we decided to implement an AnteHandler to prevent This fixes the issue while we work on a better POA design. |
Sim written & in tagged v0.50.2 version. |
write simulator for POA before he merges cosmos/cosmos-sdk#20059
another idea is to remove app.SlashingKeeper.Hooks(), from StakingKeeper hooks?
other idea is
POA on new val, set signing info -1 block. then see if sdk patch required
The text was updated successfully, but these errors were encountered: