cli/start: enhance and better comment the startup process #21344

knz · 2018-01-09T14:58:51Z

Arjun and folk have expressed interest in better understanding
"what happens during CockroachDB start up". While studying the code
to prepare an explanation, I found out that the code was somewhat
under-documented, and the ordering of steps wasn't following
a reasonable order.

This patch enhances this situation by adding necessary comments
and tweaking the start up process:

logging is set up earlier.
configuration tweaks are grouped into a single packet
of logic / statements.
InitNode() is called in the main goroutine, earlier, so
that configuration reporting and env var reporting can
be done together.

Release note: None

cc @bdarnell

cockroach-teamcity · 2018-01-09T14:58:56Z

This change is

bdarnell · 2018-01-09T15:59:54Z

Review status: 0 of 1 files reviewed at latest revision, all discussions resolved, all commit checks successful.

Comments from Reviewable

Arjun and folk have expressed interest in better understanding "what happens during CockroachDB start up". While studying the code to prepare an explanation, I found out that the code was somewhat under-documented, and the ordering of steps wasn't following a reasonable order. This patch enhances this situation by adding necessary comments and tweaking the start up process: - logging is set up earlier. - configuration tweaks are grouped into a single packet of logic / statements. - InitNode() is called in the main goroutine, earlier, so that configuration reporting and env var reporting can be done together. Release note: None

knz · 2018-01-09T16:24:32Z

TFYR!

Prior to this patch, a warning was reported if the user was running `cockroach` with the `--cache` parameter left to its default value. Such a warning is also useful for `--max-sql-memory`. This patch adds it. Release note: None

Prior to this patch, the code in `runStartSQL` was using a different sequence of steps than in `runStart`, even for those steps that are beneficial to run in both cases. This commit fixes that. In particular it adds the following missing bits to `cockroach mt start-sql`: - it fixes support for the (test-only) `COCKROACH_EXPERIMENTAL_LINEARIZABLE` env var. (from cockroachdb#4754) - it adds a tracing span for the startup code. (from cockroachdb#8712) - it properly supports `--listening-url-file`. (from cockroachdb#15468) - it properly does sanitization of `--external-io-dir`. (from cockroachdb#19725) - it sets the proper log severity level for gRPC. (from cockroachdb#20308) - it reports the command-line and env config to logs. (from cockroachdb#21344) - it prevents startup if there is a `_CRITICAL_ALERT.txt` file in the store directory. (from cockroachdb#42401) - sets the umask for newly created file to remove "other" permission bits. This was a security team request originally. (from cockroachdb#44043) - it sets `GOMAXPROCS` from current cgroup limits. (from cockroachdb#57390) - it stops the server early if the storage disk is full. (from cockroachdb#66893) - it fixes support for the `COCKROACH_DISABLE_SPAN_CONFIGS` config env var. (from cockroachdb#73876) To review this commit, it is useful to open the files `start.go` and `mt_start_sql.go` side-by-side, which clarifies how much closer they have become to each other. Looking at the `mt_start_sql.go` changes without that context likely makes the review more difficult. A later commit will actually merge the two code paths together so there is just one thing to maintain. Release note: None

90176: cli/start: unify code between `cockroach start` and `cockroach mt start-sql` r=andreimatei a=knz Fixes #89974. Fixes #90831. Fixes #90524. This PR merges the server initialization code between `cockroach start` and `cockroach mt start-sql`. In doing so, it brings `cockroach mt start-sql` closer to what we expect from proper CockroachDB server processes: - it fixes support for the (test-only) `COCKROACH_EXPERIMENTAL_LINEARIZABLE` env var. (from #4754) - it adds a tracing span for the startup code. (from #8712!!) - it properly supports `--listening-url-file`. (from #15468) - it properly does sanitization of `--external-io-dir`. (from #19725) - it sets the proper log severity level for gRPC. (from #20308) - it reports the command-line and env config to logs. (from #21344) - it prevents startup if there is a `_CRITICAL_ALERT.txt` file in the store directory. (from #42401) - sets the umask for newly created file to remove "other" permission bits. This was a security team request originally. (from #44043) - it recovers support for `DISABLE_STARTING_BACKGROUND_JOBS`. (from #44786) - it sets `GOMAXPROCS` from current cgroup limits. (from #57390) - it stops the server early if the storage disk is full. (from #66893) - it fixes support for the `COCKROACH_DISABLE_SPAN_CONFIGS` config env var. (from #73876) See the individual commit for details. 90660: sql: add contention_events to cluster_execution_insights r=j82w a=j82w The original contention column will remain to make query operations faster. The events are being put into a json column because it's possible there could be multiple blocking events for a single statement. The json column avoids the complexity of adding another table and keeping it in sync with the insights table. The table can be joined with index_columns and tables to get the database name, table name, and index name that contention occurred on. This does not contain the blocking statement information, and the blocking fingerprint id. closes: #88561 Release note (sql change): Adds contention_events to cluster_execution_insights. This is used to see which transaction is blocking the specific statement. 90719: opgen: added a bool field in struct opgen.transition r=Xiang-Gu a=Xiang-Gu This PR adds a bool field in struct opgen.transition that indicates whether it results from a `equiv(xx)` transition spec in the opgen file. It will be useful for a test where we need to find the inital status on a adding/dropping path. Without such a change, it can be problematic if we have a `equiv(xx)` spec as the first transition. E.g. ``` ToAbsent( PUBLIC, equiv(VALIDATED), to(WRITE_ONLY), to(ABSENT), ) ``` Without this change, the inital status will confusingly be `VALIDATED`, and the next status will be `PUBLIC`. With this change, the initial status will be `PUBLIC`, and the next status will be `WRITE_ONLY`. We also added some comments when we make transitions from the specs. Epic: None Release note: None 90865: sql: use bare name string of new pk to compare with pk name when altering primary key r=chengxiong-ruan a=chengxiong-ruan Fixes #90836 Release note (sql change): previously, the `DROP CONSTRAINT, ADD CONSTRAINT` in one trick to have a new primary key without moving old primary key to be a secondary index didn't work if the primary key name is a reserved SQL keyword. A `constraint already exists` error was returned. This patch fixed the bug, the trick now also works with primary key named as reserved keywords. Co-authored-by: Raphael 'kena' Poss <[email protected]> Co-authored-by: j82w <[email protected]> Co-authored-by: Xiang Gu <[email protected]> Co-authored-by: Chengxiong Ruan <[email protected]>

knz requested a review from a-robinson January 9, 2018 14:58

knz requested a review from a team as a code owner January 9, 2018 14:58

knz force-pushed the 20180109-doc-start branch from 9bf2b9d to b658aa7 Compare January 9, 2018 15:03

knz merged commit 4accee8 into cockroachdb:master Jan 9, 2018

knz deleted the 20180109-doc-start branch January 9, 2018 16:24

cli/start: also warn when --max-sql-memory is not set

115b477

Prior to this patch, a warning was reported if the user was running `cockroach` with the `--cache` parameter left to its default value. Such a warning is also useful for `--max-sql-memory`. This patch adds it. Release note: None

knz mentioned this pull request Oct 18, 2022

cli/start: unify code between cockroach start and cockroach mt start-sql #90176

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cli/start: enhance and better comment the startup process #21344

cli/start: enhance and better comment the startup process #21344

knz commented Jan 9, 2018

cockroach-teamcity commented Jan 9, 2018

bdarnell commented Jan 9, 2018

knz commented Jan 9, 2018

cli/start: enhance and better comment the startup process #21344

cli/start: enhance and better comment the startup process #21344

Conversation

knz commented Jan 9, 2018

cockroach-teamcity commented Jan 9, 2018

bdarnell commented Jan 9, 2018

knz commented Jan 9, 2018