-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
44458: storage/engine: MVCC Metamorphic test suite, first phase r=itsbilal a=itsbilal This PR adds a new test-only sub-package to engine, metamorphic, which has one test, TestMeta, that generates and runs random MVCC operations on rocksdb and pebble instances with default settings. Future additions to this test suite could include: - [x] A "check" mode that takes an output file as input, parses it, runs the operations in that sequence, and compares output strings. - [ ] Diffing test output between rocksdb and pebble and failing if there's a difference - [ ] Adding support for more operations - [ ] Adding a "restart" operation that closes the engine and restarts a different kind of engine in the store directory, then confirming operations after that point generate the same output. First-but-biggest part of #43762 . Release note: None 44730: storage: fix some tests that were fooling themselves r=andreimatei a=andreimatei A couple of tests wanted multiple write too old errors, but they were running at the wrong timestamp and so they were really getting a single one. Also add a test showing a funky scenario where 1PC batches behave differently from non-1PC. Release note: None 44786: jobs: allow blocking job adoption via sentinel file r=dt a=dt This change adds a check for a sentinel file, DISABLE_STARTING_BACKGROUND_JOBS, in the first on-disk store directory to the job adoption loop. If it is found, jobs will not be adopted by that node and current job executions are cancelled. Operators can thus touch this file if a misbehaving job is causing problems and otherwise preventing traditional job control that requires being able to write to the database. Release note (general change): background job execution is disabled on nodes where the file DISABLE_STARTING_BACKGROUND_JOBS exists in the first store directory providing an emergency tool for disabling job execution. Co-authored-by: Bilal Akhtar <[email protected]> Co-authored-by: Andrei Matei <[email protected]> Co-authored-by: David Taylor <[email protected]>
- Loading branch information
Showing
10 changed files
with
1,699 additions
and
18 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| // Copyright 2020 The Cockroach Authors. | ||
| // | ||
| // Use of this software is governed by the Business Source License | ||
| // included in the file licenses/BSL.txt. | ||
| // | ||
| // As of the Change Date specified in that file, in accordance with | ||
| // the Business Source License, use of this software will be governed | ||
| // by the Apache License, Version 2.0, included in the file | ||
| // licenses/APL.txt. | ||
|
|
||
| package metamorphic | ||
|
|
||
| import ( | ||
| "math/rand" | ||
|
|
||
| "github.com/cockroachdb/cockroach/pkg/util/syncutil" | ||
| ) | ||
|
|
||
| // deck is a random number generator that generates numbers in the range | ||
| // [0,len(weights)-1] where the probability of i is | ||
| // weights(i)/sum(weights). Unlike Weighted, the weights are specified as | ||
| // integers and used in a deck-of-cards style random number selection which | ||
| // ensures that each element is returned with a desired frequency within the | ||
| // size of the deck. | ||
| type deck struct { | ||
| rng *rand.Rand | ||
| mu struct { | ||
| syncutil.Mutex | ||
| index int | ||
| deck []int | ||
| } | ||
| } | ||
|
|
||
| // newDeck returns a new deck random number generator. | ||
| func newDeck(rng *rand.Rand, weights ...int) *deck { | ||
| var sum int | ||
| for i := range weights { | ||
| sum += weights[i] | ||
| } | ||
| expandedDeck := make([]int, 0, sum) | ||
| for i := range weights { | ||
| for j := 0; j < weights[i]; j++ { | ||
| expandedDeck = append(expandedDeck, i) | ||
| } | ||
| } | ||
| d := &deck{ | ||
| rng: rng, | ||
| } | ||
| d.mu.index = len(expandedDeck) | ||
| d.mu.deck = expandedDeck | ||
| return d | ||
| } | ||
|
|
||
| // Int returns a random number in the range [0,len(weights)-1] where the | ||
| // probability of i is weights(i)/sum(weights). | ||
| func (d *deck) Int() int { | ||
| d.mu.Lock() | ||
| if d.mu.index == len(d.mu.deck) { | ||
| d.rng.Shuffle(len(d.mu.deck), func(i, j int) { | ||
| d.mu.deck[i], d.mu.deck[j] = d.mu.deck[j], d.mu.deck[i] | ||
| }) | ||
| d.mu.index = 0 | ||
| } | ||
| result := d.mu.deck[d.mu.index] | ||
| d.mu.index++ | ||
| d.mu.Unlock() | ||
| return result | ||
| } |
Oops, something went wrong.