-
Notifications
You must be signed in to change notification settings - Fork 294
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tracking issue: 🤝 mock consensus engine for App
tests
#3588
Comments
update 2024-01-09: in #3592 and in some direct conversations, we discussed the desire to address #3588 using two crates: (1) a core library that provides the ability to generate block headers, and (2) an in-memory consensus engine with facilities to manipulate time. after some further reading, i believe it makes sense to keep #3597 focused on (1), rather than attempting to completely address #3588 in one PR. this core library should be able to generate valid block headers that can be successfully parsed by a real light client library. ( today i spent time reading through the broad strokes of how the protobuf types and at this point i'm still getting a lay of the land, but i feel like i am gaining a much clearer picture of what work is to come. tomorrow i'll meet up with @avahowell to answer questions i've accrued concerning IBC, and spend some time working on stubs to generate a valid tendermint block header. 🍞 🔗 helpful links for future reference
|
update 2024-01-22:
- me, #status channel in discord with my draft pr (#3597) further along, i am going to pivot focus to work on #3627 this week. i'll resume work on this issue afterwards. |
today @hdevalence linked me to #1664. this is a great issue outlining the larger goals and direction of our testing facilities. linking here, for future reference. |
I think it would be good to focus this effort on the concrete goal of writing
We should start by writing a second version of each of those tests, but instead of driving each component's code manually in the Let's look at the first, simpler test, in #[tokio::test]
async fn spend_happy_path() -> anyhow::Result<()> {
let mut rng = rand_chacha::ChaChaRng::seed_from_u64(1312);
let storage = TempStorage::new().await?.apply_default_genesis().await?;
let mut state = Arc::new(StateDelta::new(storage.latest_snapshot()));
let height = 1; This is initializing a new #[tokio::test]
async fn spend_happy_path_2() -> anyhow::Result<()> {
let mut rng = rand_chacha::ChaChaRng::seed_from_u64(1312);
let storage = TempStorage::new().await?;
let app = App::new(storage.latest_snapshot());
let engine = MockComet::builder()
.single_validator() // builder-style methods for configuring validators
.app_state(genesis::AppState::default()) // N.B. will later need an ext trait
.init_chain(app) // "finish" method of builder, saves app and calls InitChain
.await?; As before, we create a The note about the extension trait is that we'll later want to have an extension trait for the builder API that does Penumbra-specific behavior, because the Penumbra application needs to know about its validators at genesis, so we'll eventually want to have a method that can "reconcile" a default genesis state with whatever validators the Let's keep going. // ORIGINAL spend_happy_path()
// Precondition: This test uses the default genesis which has existing notes for the test keys.
let mut client = MockClient::new(test_keys::FULL_VIEWING_KEY.clone());
let sk = test_keys::SPEND_KEY.clone();
client.sync_to(0, state.deref()).await?;
let note = client.notes.values().next().unwrap().clone();
let note_commitment = note.commit();
let proof = client.sct.witness(note_commitment).unwrap();
let root = client.sct.root();
let tct_position = proof.position(); This section uses the existing The This code would remain unchanged. If the complexity of the actions we wanted to perform in tests grew, we could either hook up the complete view server implementation, or add adapters to the // ORIGINAL spend_happy_path()
// 1. Simulate BeginBlock
let mut state_tx = state.try_begin_transaction().unwrap();
state_tx.put_block_height(height);
state_tx.put_epoch_by_height(
height,
Epoch {
index: 0,
start_height: 0,
},
);
state_tx.apply();
// 2. Create a Spend action
let spend_plan = SpendPlan::new(&mut rng, note, tct_position);
let dummy_effect_hash = [0u8; 64];
let rsk = sk.spend_auth_key().randomize(&spend_plan.randomizer);
let auth_sig = rsk.sign(&mut rng, dummy_effect_hash.as_ref());
let spend = spend_plan.spend(&test_keys::FULL_VIEWING_KEY, auth_sig, proof, root);
let transaction_context = TransactionContext {
anchor: root,
effect_hash: EffectHash(dummy_effect_hash),
};
// 3. Simulate execution of the Spend action
spend.check_stateless(transaction_context).await?;
spend.check_stateful(state.clone()).await?;
let mut state_tx = state.try_begin_transaction().unwrap();
state_tx.put_mock_source(1u8);
spend.execute(&mut state_tx).await?;
state_tx.apply();
// 4. Execute EndBlock
let height = 1;
let end_block = abci::request::EndBlock {
height: height.try_into().unwrap(),
};
ShieldedPool::end_block(&mut state, &end_block).await;
let mut state_tx = state.try_begin_transaction().unwrap();
// ... and for the App, call `finish_block` to correctly write out the SCT with the data we'll use next.
state_tx.finish_block(false).await.unwrap();
state_tx.apply();
Ok(()) In the rest of the test, the current code:
Instead, this should use // NEW spend_happy_path_2()
// 1. Create a `TransactionPlan` with a 1-spend, 1-output transaction
let plan = TransactionPlan {
// self-contained struct literal with a 1-spend, 1-output transaction
// using the data obtained from the MockClient as in the snippet above
};
let auth_data = AuthorizationData {
// we can either create a struct literal with the signatures we need,
// or we can initialize a SoftKms instance with the test keys and pass the plan
};
let witness_data = WitnessData {
// self-contained struct literal using the inclusion proof from the MockClient
};
// 2. Use the `TransactionPlan`, `AuthorizationData`, and `WitnessData` to build the transaction
let tx = plan.build_concurrent(...).await?;
// 3. Now use the `MockComet` to create a new block with that transaction
engine.block_builder()
//.timestamp(...) // methods for controlling simulated execution
.add_tx(tx.encode_to_vec()) // adds to list in block builder
.execute() // triggers app execution
.await?; This is much simpler and less error-prone. We need to change from simulating a single We manually plan the transaction using the data from the Then, we use a
Now, in this example, we can go even further with // NEW spend_happy_path_2()
// 4. Use the MockClient to check that we detect our output note
client.sync_to(...)
// .... check that the note we expect is now seen in the MockClient
// ... or we could make a second transaction, checking we can spend notes that weren't created at genesis
// ... or we could check that a double-spend fails, or, .... By contrast, what would happen if we tried to add this to the existing test? The entire test would explode, because the current test leaves the application in an invalid state, it doesn't write out the compact block, it doesn't close the SCT, etc., because it doesn't execute the full application logic. Note also what's not required: anything other than actually driving the application and feeding it a header as part of |
App
tests
first: i've renamed this issue, to be clearer about the fact that this is not intended for intgegration tests, and is intended for |
fixes #3933. this introduces a `fast_forward` method to a test node, allowing tests using the mock consensus engine (#3588) to fast forward a certain number of blocks. the existing `mock_consensus_can_send_a_sequence_of_empty_blocks` test is rewritten to make use of this method. this is done in service of #3995. --- * #3588 * #3933 * #3995
fixes #3933. this introduces a `fast_forward` method to a test node, allowing tests using the mock consensus engine (#3588) to fast forward a certain number of blocks. the existing `mock_consensus_can_send_a_sequence_of_empty_blocks` test is rewritten to make use of this method. this is done in service of #3995. --- * #3588 * #3933 * #3995
see #3913, #3973 and #3588. this is a second attempt, following up on #3980. #### 🔭 background NB: the difference between this and #3679 is that the latter (_which ran afoul of a regression_) would have `penumbra-app` create a `Routes`, that we would [add](https://github.com/penumbra-zone/penumbra/pull/3679/files#diff-fbc4204ceb976c8cb30ed06168e2476700bae21bfd803e26281b2d026194d430R204) to the builder (_which stays in `pd`_). here, i'm not trying to make that cut between `Router` and `Routes`, and am attempting to hoist the whole thing out of `pd`, without making changes to how we interact with `tonic`. my aim is for us to be able to move this, without running into that bug (#3697) again. NB: after running into problems in #3980, i found a way to easily reproduce the issue locally. my belief was that something related to our dependencies' cargo features was at play. rather than isolate the issue, it was easier to rewrite this (_it's just code motion, after all_) while running some of the network integration tests in a loop. unlike #3980, this moves the rpc server into `penumbra-app`, per #3980 (comment) #### 👁️ overview we would like to use the rust view server in mock consensus tests. in order to run the `penumbra_view::ViewServer` however, we need to spin up the corresponding grpc endpoint for it to connect to. this branch performs a bit of code motion, moving the `grpc_server` out of `pd` and into `penumbra-app`. there will likely be other functional changes to the code in question before we can use it in those tests, but this PR is interested in moving that code into a place where our tests can rely upon it.
fixes #3933. this introduces a `fast_forward` method to a test node, allowing tests using the mock consensus engine (#3588) to fast forward a certain number of blocks. the existing `mock_consensus_can_send_a_sequence_of_empty_blocks` test is rewritten to make use of this method. this is done in service of #3995. --- * #3588 * #3933 * #3995
) fixes #3936. based upon #4002. see #3588. this changes the type of the block builder's `data`, allowing it to be implicitly kept empty. this is useful when e.g. fast forwarding a number of blocks _(see #4002)_. this spares us the need to call `with_data(vec![])`, and additionally means that the block builder is never in an uninitialized state _(making #4003 needless)._ * #3936 * #4002 * #3588
fixes #3966. fixes #3908. fixes _part of_ #3995. this branch introduces the first steps towards mock consensus (#3588) testing of the staking component (#3845). this defines a validator after genesis, and then shows that it does _not_ enter the consensus set. #3966 is addressed in this branch so that the existing genesis validator correctly enters the consensus set, and so that we can successfully progress to the second epoch. subsequent changes will exercise delegating to this validator in the `mock_consensus_can_define_and_delegate_to_a_validator`. #### ✨ changes * alters `with_penumbra_auto_app_state` so that it adds an allocation of delegation tokens to the shielded pool component's genesis content. * extends `generate_penumbra_validator` so that it generates a real spend key, and returns an `Allocation` for the generated validator. _(see #3966)_ * adds a new `mock_consensus_can_define_and_delegate_to_a_validator` test that defines a post-genesis validator. _(see #3908)_ * defines a new `ConsensusIndexRead::get_consensus_set()` method, which collects all of the identity keys returned by `consensus_set_stream`. * lowers the events in `penumbra_mock_consensus::block::Builder::execute()` to trace-level events. * `penumbra_mock_consensus::builder::Builder` will now log a warning if values may be errantly rewritten by the builder methods. * `TestNode::fast_forward` sets its `i` span field to `1..n`, rather than `0..n-1`. --- #### :link: related * #4009 * #4010 * #4011 * #4017 * #4027 * #4028 * #4029 * #3966 * #3908 * #3588 --------- Co-authored-by: Henry de Valence <[email protected]>
fixes #4181. see #3588. this makes a pass through the `penumbra-mock-consensus` library and further documents various interfaces. one other small tweak, logging a warning if a caller may be inadvertently discarding transactions, is made while we are here. these docs may be rendered by running: `cargo doc --package penumbra-mock-consensus --open` --------- Co-authored-by: Conor Schaefer <[email protected]>
fixes #4050. see also #3588. this introduces a test case that demonstrates that transactions' validator definition actions must have a valid authentication signature, or the validator will not be added to the set of known validators. - [x] If this code contains consensus-breaking changes, I have added the "consensus-breaking" label. Otherwise, I declare my belief that there are not consensus-breaking changes, for the following reason: > this is a test case.
see #3588. follows #4184 and #4181. this takes a pass through the shared, Penumbra-specific test infrastructure for mock consensus tests. notably, this decomposes `init_chain.rs`, which has now become somewhat redundant with the existence of other more involved tests of e.g. validator uptime tracking. this also cleans up some unused imports, guards against future occurrences of that issue (_sharing code in `tests/` files is awkward_), and decomposes the `common/mod.rs` file into some distinct standalone components. this also belatedly removes the `common::start_test_node()` helper. at some point (_i was unable to find the link_) it was suggested that we refrain from a shared setup helper like that. this branch removes that helper, and updates its call-sites. this branch is largely code motion, and is intended to be a last bit of cleanup as we prepare for #3588 to wind down. ❤️ --------- Co-authored-by: Henry de Valence <[email protected]>
💭 background and motivation
currently, writing complete end-to-end integration tests for Penumbra is
difficult. moreover, we don't have a better alternative strategy for testing the core
App
type.currently, the
smoke-test.sh
script performs the steps neededto run integration tests of the system, including:
cometbft
processpd
processbecause these integration tests assume a test network including
pd
andcometbft
is running locally, these tests must run serially, and must spendtime waiting for new blocks to be generated.
additionally, this means we must under-handedly make use of the
#[ignore]
attribute to prevent these tests from running (and failing) alongside unit
tests when commands like
cargo test --workspace
are run.see CometMock for a mock implementation of CometBFT implemented
in Go. see #2771 for previous work in this space integrating with
CometMock
.📐 overview
this state of affairs would be improved if we had a mock implementation of the
consensus engine (CometBFT) that would allow
#[test]
functions togenerate ABCI messages. this mock implementation should function as a drop-in
replacement for CometBFT from the perspective of the
pd
daemon.this mocking library should additionally expose some "meta" interfaces. to allow us
to write integration tests including things like:
involving e.g. expiry may be exercised.
this will allow us to run integration tests faster, write them more easily,
and generally improve our ability to make assurances about the correctness
of our software.
📐 requirements & design guidelines
(this will go into crate-level documentation at some point)
the mock consensus engine should be application agnostic. that means it should not depend on
penumbra-*
crates. (#3810)the mock consensus engine is built to drive a consensus service
C: Service<ConsensusRequest, Response = ConsensusResponse, Error = BoxError>
🔗 related work
app
library #3695pd::cli
#3672Consensus
andMempool
intopenumbra-app
#3789penumbra-app
#3794App
can be instantiated with mock consensus #3787listen_channel
for in-memory connections tower-abci#41InitChain
with a single validator #3816SpendKeyBytes: From<[u8; SPENDKEY_LEN_BYTES]>
#3922evidence::List::default
#3848fast_forward
#4002add_tx
to append tx's #3936mock_consensus_staking
todo #4047ValidatorUptimeTracker
extension trait #4099TestNode<C>
#3934auto_app_state
assigns delegation tokens #3966PrivateKey
andPublicKey
haveFrom
impls fored25519-consensus
types informalsystems/tendermint-rs#1401InitChain
with two validators #3937two_validators
adds two keys #4174penumbra-mock-consensus
library #4181ViewClient
(work in progress) #3958penumbra-app
#3996🔜 future work
this is a large project, and the extension and maintenance of this library will be an ongoing project. here are things that we ought to do someday, but won't be considered requirements for this issue to be closed.
proptest
#3741❌ out-of-scope
things we will not be doing, and other closed tickets:
Components
#3740👓 further reading
The text was updated successfully, but these errors were encountered: