Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip mandatory commitment when beefy authorities not changed #1215

Open
wants to merge 41 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
a686dfa
Permit non-consecutive increases in validator set
Lederstrumpf Jan 31, 2024
a223f97
adjust submitInitial too
Lederstrumpf Jan 31, 2024
ea2f146
Improve filtering
vgeddes May 27, 2024
ebb5794
Skip mandatory commitment
yrong Feb 4, 2024
0faba1f
Merge remote-tracking branch 'Lederstrumpf/permit-nonconsecutive-incr…
yrong May 28, 2024
5eba01b
Fix updating with current beefy state
yrong May 28, 2024
482e8a2
Ignore submit when authorities not change
yrong May 28, 2024
40f9592
Cleanup
yrong May 29, 2024
8853005
Improve BEEFY relayer
vgeddes May 30, 2024
629a478
Sync beefy commitment on demand
yrong May 31, 2024
67c52e8
Minor fix
yrong May 31, 2024
dba8cfc
More comments
yrong May 31, 2024
a886879
More refactoring
yrong May 31, 2024
fd7db5b
review feedback
vgeddes May 31, 2024
58d83bc
review feedback #2
vgeddes May 31, 2024
80b4dbd
unused code
vgeddes May 31, 2024
c11ed7d
Fix for boundary update
yrong Jun 1, 2024
10f38d4
Fix for skip mandatory commitment
yrong Jun 1, 2024
98e3e74
Some refactoring
yrong Jun 3, 2024
fc29329
Fix ci breaking
yrong Jun 3, 2024
6efd6c1
Merge branch 'vincent/beefy-relay-improvements' into ron/beefy-relay-…
yrong Jun 3, 2024
715ac5a
Find for next beefy block
yrong Jun 3, 2024
7d06d62
Merge branch 'ron/beefy-relay-improvements' into ron/skip-mandatory-c…
yrong Jun 3, 2024
16c5012
Merge branch 'ron/skip-mandatory-commitment' of https://github.com/Sn…
yrong Jun 3, 2024
913960e
Improve log
yrong Jun 3, 2024
edd9f21
Remove check unrelated
yrong Jun 3, 2024
47b0530
Merge branch 'ron/beefy-relay-improvements' into ron/skip-mandatory-c…
yrong Jun 3, 2024
08c70f1
Check commitment in CurrentValidatorSetID
yrong Jun 3, 2024
95845a3
Sync beefy commitment on demand (#1217)
yrong Jun 3, 2024
bb0c52f
Resolve conflicts
yrong Jun 3, 2024
6d26d04
unused context parameter
vgeddes Jun 3, 2024
39aac20
Merge branch 'main' into vincent/beefy-relay-improvements
vgeddes Jun 3, 2024
ba37da4
Merge branch 'vincent/beefy-relay-improvements' into ron/skip-mandato…
yrong Jun 4, 2024
3d8daa6
Test happy path
yrong Jun 4, 2024
47cceeb
More tests
yrong Jun 4, 2024
0f221e5
More tests
yrong Jun 4, 2024
ae9c1ad
Merge branch 'main' into ron/skip-mandatory-commitment
yrong Jun 4, 2024
585619e
For rococo compatibility
yrong Jun 4, 2024
4b80965
Merge branch 'main' into ron/skip-mandatory-commitment
yrong Jun 5, 2024
2b30202
Revert changes
yrong Jun 5, 2024
c99028d
Allow commitment in future
yrong Jun 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions contracts/src/BeefyClient.sol
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,7 @@ contract BeefyClient {
signatureUsageCount = currentValidatorSet.usageCounters.get(proof.index);
currentValidatorSet.usageCounters.set(proof.index, signatureUsageCount.saturatingAdd(1));
vset = currentValidatorSet;
} else if (commitment.validatorSetID == nextValidatorSet.id) {
} else if (commitment.validatorSetID >= nextValidatorSet.id) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need some tests in the solidity that skips a mandatory commitment.

Example scenario:
We are in validator set id 6 (next validator set 7), we skip 7 and 8 and then import 9. After importing 9 the current validator set will be 7 and the next validator set will be 10.

See here: https://github.com/Snowfork/snowbridge/pull/1215/files#diff-473fead6cd500a4a05f06d0cea4d9ef166c9bdbe2a2e1a06a33d2e1aedcc428eR379-R382

We also need to make sure that a future commit does not commit to a block number in the past and a test that covers it.

Context:
#1137 (comment)
#1137 (comment)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch Alistair, yeah, we need more unit tests for this work.

Copy link
Contributor Author

@yrong yrong Jun 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, so in this case the current&next validatorId pair will be (7,10) and latestBeefyBlock is in session 9.

I would expect a minor change here

if (commitment.validatorSetID == currentValidatorSet.id) {

from if (commitment.validatorSetID == currentValidatorSet.id) to if (commitment.validatorSetID == currentValidatorSet.id) || commitment.validatorSetID == nextValidatorSet.id - 1 will just work but I'll definetely do more tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

signatureUsageCount = nextValidatorSet.usageCounters.get(proof.index);
nextValidatorSet.usageCounters.set(proof.index, signatureUsageCount.saturatingAdd(1));
vset = nextValidatorSet;
Expand Down Expand Up @@ -354,7 +354,7 @@ contract BeefyClient {

bool is_next_session = false;
ValidatorSetState storage vset;
if (commitment.validatorSetID == nextValidatorSet.id) {
if (commitment.validatorSetID > currentValidatorSet.id) {
is_next_session = true;
vset = nextValidatorSet;
} else if (commitment.validatorSetID == currentValidatorSet.id) {
Expand All @@ -368,7 +368,7 @@ contract BeefyClient {
bytes32 newMMRRoot = ensureProvidesMMRRoot(commitment);

if (is_next_session) {
if (leaf.nextAuthoritySetID != nextValidatorSet.id + 1) {
if (leaf.nextAuthoritySetID <= nextValidatorSet.id) {
revert InvalidMMRLeaf();
}
bool leafIsValid =
Expand Down
6 changes: 5 additions & 1 deletion relayer/cmd/scan_beefy.go
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,11 @@ func ScanBeefyFn(cmd *cobra.Command, _ []string) error {
"validator-set-id": validatorSetID,
}).Info("Connected to relaychain.")

commitments, err := polkadotListener.Start(ctx, eg, beefyBlock, validatorSetID)
var currentState beefy.BeefyState
currentState.CurrentValidatorSetId = validatorSetID
currentState.LatestBeefyBlock = beefyBlock

commitments, err := polkadotListener.Start(ctx, eg, currentState)
if err != nil {
logrus.WithError(err).Fatalf("could not start")
}
Expand Down
13 changes: 12 additions & 1 deletion relayer/relays/beefy/ethereum-writer.go
Original file line number Diff line number Diff line change
Expand Up @@ -89,9 +89,13 @@ func (wr *EthereumWriter) submit(ctx context.Context, task Request) error {
return err
}
if uint32(latestBeefyBlock) >= task.SignedCommitment.Commitment.BlockNumber {
log.WithFields(logrus.Fields{
"blockNumber": task.SignedCommitment.Commitment.BlockNumber,
"IsHandover": task.IsHandover,
"ValidatorSetID": task.SignedCommitment.Commitment.ValidatorSetID,
}).Warn("Ignore outdated commitment")
return nil
}

currentValidatorSet, err := wr.contract.CurrentValidatorSet(&callOpts)
if err != nil {
return err
Expand All @@ -103,6 +107,13 @@ func (wr *EthereumWriter) submit(ctx context.Context, task Request) error {
task.ValidatorsRoot = currentValidatorSet.Root
if task.IsHandover {
task.ValidatorsRoot = nextValidatorSet.Root
if task.nextAuthoritiesRoot == task.ValidatorsRoot {
log.WithFields(logrus.Fields{
"blockNumber": task.SignedCommitment.Commitment.BlockNumber,
"ValidatorSetID": task.SignedCommitment.Commitment.ValidatorSetID,
}).Warn("Ignore mandatory commitment authorities not change")
return nil
}
}

// Initial submission
Expand Down
37 changes: 26 additions & 11 deletions relayer/relays/beefy/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,15 @@ func NewRelay(config *Config, ethereumKeypair *secp256k1.Keypair) (*Relay, error

log.Info("Beefy relay created")

return &Relay{
relayer := Relay{
config: config,
relaychainConn: relaychainConn,
ethereumConn: ethereumConn,
polkadotListener: polkadotListener,
ethereumWriter: ethereumWriter,
}, nil
}
polkadotListener.relayer = &relayer
return &relayer, nil
}

func (relay *Relay) Start(ctx context.Context, eg *errgroup.Group) error {
Expand All @@ -57,16 +59,15 @@ func (relay *Relay) Start(ctx context.Context, eg *errgroup.Group) error {
return fmt.Errorf("create ethereum connection: %w", err)
}

initialBeefyBlock, initialValidatorSetID, err := relay.getInitialState(ctx)
currentState, err := relay.CurrentState(ctx)
if err != nil {
return fmt.Errorf("fetch BeefyClient current state: %w", err)
}
log.WithFields(log.Fields{
"beefyBlock": initialBeefyBlock,
"validatorSetID": initialValidatorSetID,
"currentState": currentState,
}).Info("Retrieved current BeefyClient state")

requests, err := relay.polkadotListener.Start(ctx, eg, initialBeefyBlock, initialValidatorSetID)
requests, err := relay.polkadotListener.Start(ctx, eg, currentState)
if err != nil {
return fmt.Errorf("initialize polkadot listener: %w", err)
}
Expand All @@ -79,11 +80,12 @@ func (relay *Relay) Start(ctx context.Context, eg *errgroup.Group) error {
return nil
}

func (relay *Relay) getInitialState(ctx context.Context) (uint64, uint64, error) {
func (relay *Relay) CurrentState(ctx context.Context) (BeefyState, error) {
var currentState BeefyState
address := common.HexToAddress(relay.config.Sink.Contracts.BeefyClient)
beefyClient, err := contracts.NewBeefyClient(address, relay.ethereumConn.Client())
if err != nil {
return 0, 0, err
return currentState, err
}

callOpts := bind.CallOpts{
Expand All @@ -92,13 +94,26 @@ func (relay *Relay) getInitialState(ctx context.Context) (uint64, uint64, error)

latestBeefyBlock, err := beefyClient.LatestBeefyBlock(&callOpts)
if err != nil {
return 0, 0, err
return currentState, err
}

currentValidatorSet, err := beefyClient.CurrentValidatorSet(&callOpts)
if err != nil {
return 0, 0, err
return currentState, err
}

nextValidatorSet, err := beefyClient.NextValidatorSet(&callOpts)
if err != nil {
return currentState, err
}

currentState = BeefyState{
LatestBeefyBlock: latestBeefyBlock,
CurrentValidatorSetId: currentValidatorSet.Id.Uint64(),
CurrentValidatorSetRoot: currentValidatorSet.Root,
NextValidatorSetId: nextValidatorSet.Id.Uint64(),
NextValidatorSetRoot: nextValidatorSet.Root,
}

return latestBeefyBlock, currentValidatorSet.Id.Uint64(), nil
return currentState, nil
}
8 changes: 8 additions & 0 deletions relayer/relays/beefy/parameters.go
Original file line number Diff line number Diff line change
Expand Up @@ -245,3 +245,11 @@ func proofToLog(proof contracts.BeefyClientValidatorProof) logrus.Fields {
"Proof": hexProof,
}
}

type BeefyState struct {
LatestBeefyBlock uint64
CurrentValidatorSetId uint64
CurrentValidatorSetRoot [32]byte
NextValidatorSetId uint64
NextValidatorSetRoot [32]byte
}
61 changes: 41 additions & 20 deletions relayer/relays/beefy/polkadot-listener.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ type PolkadotListener struct {
config *SourceConfig
conn *relaychain.Connection
beefyAuthoritiesKey types.StorageKey
relayer *Relay
}

func NewPolkadotListener(
Expand All @@ -32,8 +33,7 @@ func NewPolkadotListener(
func (li *PolkadotListener) Start(
ctx context.Context,
eg *errgroup.Group,
currentBeefyBlock uint64,
currentValidatorSetID uint64,
currentState BeefyState,
) (<-chan Request, error) {
storageKey, err := types.CreateStorageKey(li.conn.Metadata(), "Beefy", "Authorities", nil, nil)
if err != nil {
Expand All @@ -45,7 +45,7 @@ func (li *PolkadotListener) Start(

eg.Go(func() error {
defer close(requests)
err := li.scanCommitments(ctx, currentBeefyBlock, currentValidatorSetID, requests)
err := li.scanCommitments(ctx, currentState, requests)
if err != nil {
return err
}
Expand All @@ -57,16 +57,15 @@ func (li *PolkadotListener) Start(

func (li *PolkadotListener) scanCommitments(
ctx context.Context,
currentBeefyBlock uint64,
currentValidatorSet uint64,
currentState BeefyState,
requests chan<- Request,
) error {
in, err := ScanSafeCommitments(ctx, li.conn.Metadata(), li.conn.API(), currentBeefyBlock+1)
lastSyncedBeefyBlock := currentState.LatestBeefyBlock
currentValidatorSet := currentState.CurrentValidatorSetId
in, err := ScanSafeCommitments(ctx, li.conn.Metadata(), li.conn.API(), lastSyncedBeefyBlock+1)
if err != nil {
return fmt.Errorf("scan commitments: %w", err)
}
lastSyncedBeefyBlock := currentBeefyBlock

for {
select {
case <-ctx.Done():
Expand All @@ -83,43 +82,49 @@ func (li *PolkadotListener) scanCommitments(
validatorSetID := result.SignedCommitment.Commitment.ValidatorSetID
nextValidatorSetID := uint64(result.MMRProof.Leaf.BeefyNextAuthoritySet.ID)

if validatorSetID != currentValidatorSet && validatorSetID != currentValidatorSet+1 {
return fmt.Errorf("commitment has unexpected validatorSetID: blockNumber=%v validatorSetID=%v expectedValidatorSetID=%v",
committedBeefyBlock,
validatorSetID,
currentValidatorSet,
)
}

logEntry := log.WithFields(log.Fields{
"commitment": log.Fields{
"blockNumber": committedBeefyBlock,
"validatorSetID": validatorSetID,
"nextValidatorSetID": nextValidatorSetID,
},
"validatorSetID": currentValidatorSet,
"IsHandover": validatorSetID == currentValidatorSet+1,
"IsHandover": validatorSetID > currentValidatorSet,
"lastSyncedBeefyBlock": lastSyncedBeefyBlock,
})

validators, err := li.queryBeefyAuthorities(result.BlockHash)
if err != nil {
return fmt.Errorf("fetch beefy authorities at block %v: %w", result.BlockHash, err)
}
nextAuthoritySet, err := li.queryBeefyNextAuthoritySet(result.BlockHash)
if err != nil {
return fmt.Errorf("fetch beefy authorities set at block %v: %w", result.BlockHash, err)
}
task := Request{
Validators: validators,
SignedCommitment: result.SignedCommitment,
Proof: result.MMRProof,
}

if validatorSetID == currentValidatorSet+1 && validatorSetID == nextValidatorSetID-1 {
currentState, err = li.relayer.CurrentState(ctx)
if err != nil {
return fmt.Errorf("get current state from beefy LC %w", err)
}

if validatorSetID > currentValidatorSet {
if nextAuthoritySet.Root == currentState.NextValidatorSetRoot && committedBeefyBlock < lastSyncedBeefyBlock+li.config.UpdatePeriod {
logEntry.Info("Discarded commitment with beefy authorities not change")
continue
}
task.IsHandover = true
task.nextAuthoritiesRoot = nextAuthoritySet.Root
select {
case <-ctx.Done():
return ctx.Err()
case requests <- task:
logEntry.Info("New commitment with handover added to channel")
currentValidatorSet++
currentValidatorSet = validatorSetID
lastSyncedBeefyBlock = committedBeefyBlock
}
} else if validatorSetID == currentValidatorSet {
Expand Down Expand Up @@ -164,7 +169,7 @@ func (li *PolkadotListener) queryBeefyAuthorities(blockHash types.Hash) ([]subst

func (li *PolkadotListener) queryBeefyNextAuthoritySet(blockHash types.Hash) (types.BeefyNextAuthoritySet, error) {
var nextAuthoritySet types.BeefyNextAuthoritySet
storageKey, err := types.CreateStorageKey(li.conn.Metadata(), "MmrLeaf", "BeefyNextAuthorities", nil, nil)
storageKey, err := types.CreateStorageKey(li.conn.Metadata(), "BeefyMmrLeaf", "BeefyNextAuthorities", nil, nil)
ok, err := li.conn.API().RPC.State.GetStorage(storageKey, &nextAuthoritySet, blockHash)
if err != nil {
return nextAuthoritySet, err
Expand All @@ -175,3 +180,19 @@ func (li *PolkadotListener) queryBeefyNextAuthoritySet(blockHash types.Hash) (ty

return nextAuthoritySet, nil
}

type BeefyAuthoritySet = types.BeefyNextAuthoritySet

func (li *PolkadotListener) queryBeefyAuthoritySet(blockHash types.Hash) (BeefyAuthoritySet, error) {
var authoritySet BeefyAuthoritySet
storageKey, err := types.CreateStorageKey(li.conn.Metadata(), "BeefyMmrLeaf", "BeefyAuthorities", nil, nil)
ok, err := li.conn.API().RPC.State.GetStorage(storageKey, &authoritySet, blockHash)
if err != nil {
return authoritySet, err
}
if !ok {
return authoritySet, fmt.Errorf("beefy AuthoritySet not found")
}

return authoritySet, nil
}
9 changes: 0 additions & 9 deletions relayer/relays/beefy/scanner.go
Original file line number Diff line number Diff line change
Expand Up @@ -128,10 +128,6 @@ func scanCommitments(ctx context.Context, api *gsrpc.SubstrateAPI, startBlock ui
sendError(fmt.Errorf("fetch block: %w", err))
return
}
log.WithFields(log.Fields{
"blockNumber": result.BlockNumber,
"depth": result.Depth,
}).Info("fetch block")

var commitment *types.SignedCommitment
for j := range block.Justifications {
Expand Down Expand Up @@ -210,11 +206,6 @@ func scanSafeCommitments(ctx context.Context, meta *types.Metadata, api *gsrpc.S
sendError(result.Error)
return
}
log.WithFields(log.Fields{
"blockNumber": result.BlockNumber,
"depth": result.Depth,
"commitment": result.SignedCommitment.Commitment,
}).Info("fetch commitment")

blockNumber := result.SignedCommitment.Commitment.BlockNumber
blockHash, err := api.RPC.Chain.GetBlockHash(uint64(blockNumber))
Expand Down
11 changes: 6 additions & 5 deletions relayer/relays/beefy/task.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,10 @@ import (
)

type Request struct {
Validators []substrate.Authority
ValidatorsRoot [32]byte
SignedCommitment types.SignedCommitment
Proof merkle.SimplifiedMMRProof
IsHandover bool
Validators []substrate.Authority
ValidatorsRoot [32]byte
nextAuthoritiesRoot [32]byte
SignedCommitment types.SignedCommitment
Proof merkle.SimplifiedMMRProof
IsHandover bool
}
Loading