Skip to content

Commit

Permalink
kvserver: handle context cancellation when receiving snapshots
Browse files Browse the repository at this point in the history
In a3fd4fb, context errors are now propagated back up the stack
when receiving snapshots. However, a `maybeFatalOnRaftReadyErr`
assertion further up the stack was not prepared to handle these new
errors and crash the node.

This patch explicitly propagates the context cancellation errors on the
snapshot receival code path, returning it up the stack. It does not
modify `maybeFatalOnRaftReadyErr` to allow context cancellation, since
the main Raft `processReady` path appears to use a background context
and improved context handling here likely requires more care.

Release note: None
  • Loading branch information
erikgrinaker committed Dec 5, 2021
1 parent 506d129 commit ea68173
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion pkg/kv/kvserver/store_raft.go
Original file line number Diff line number Diff line change
Expand Up @@ -338,14 +338,17 @@ func (s *Store) processRaftSnapshotRequest(
var expl string
var err error
stats, expl, err = r.handleRaftReadyRaftMuLocked(ctx, inSnap)
if errors.IsAny(err, context.Canceled, context.DeadlineExceeded) {
return roachpb.NewError(err)
}
maybeFatalOnRaftReadyErr(ctx, expl, err)
if !stats.snap.applied {
// This line would be hit if a snapshot was sent when it isn't necessary
// (i.e. follower was able to catch up via the log in the interim) or when
// multiple snapshots raced (as is possible when raft leadership changes
// and both the old and new leaders send snapshots).
log.Infof(ctx, "ignored stale snapshot at index %d", snapHeader.RaftMessageRequest.Message.Snapshot.Metadata.Index)
}
maybeFatalOnRaftReadyErr(ctx, expl, err)
return nil
})
}
Expand Down

0 comments on commit ea68173

Please sign in to comment.