ydb_app_ensures_isolation provides initial setting for global variables where application design ensures Isolation #218
Comments
nars1
changed the title
nars1
added a commit
to nars1/YottaDB
that referenced
this issue
Apr 30, 2018
…y list of global variable names where application design ensures isolation
nars1
added a commit
to nars1/YottaDB
that referenced
this issue
Apr 30, 2018
…list of global variable names where application design ensures isolation
nars1
added a commit
to nars1/YottaDB
that referenced
this issue
Apr 30, 2018
…nv_init() to gtm_startup() (a lot of other setup like stp_init() are done only after gtm_env_init() and can cause SIG-11 in case the NOISOLATION setup requires stringpool expansion etc.)
nars1
added a commit
to nars1/YottaDBtest
that referenced
this issue
Apr 30, 2018
…/YDB#218) Two test framework scripts that were already VIEW "NOISOLATION" were identified. 1) imptp.m 2) tpntpupd.m And they were enhanced to check for the new env var and if present use that and if not do VIEW "NOISOLATION" commands like they previously did. This way all tests that use imptp.m (many dozens) and tpntpupd.m (2 tests) will automatically test the new env var. Took this opportunity to do some "usesimpleapi"-related cleanup in imptp.csh.
nars1
added a commit
that referenced
this issue
Apr 30, 2018
… global variable names where application design ensures isolation The ydb_app_ensures_isolation env var check is not done in gtm_env_init() but instead in gtm_startup() since a lot of other setup like stp_init() is done only after gtm_env_init() and can cause SIG-11 in case the NOISOLATION setup requires stringpool expansion etc.
nars1
added a commit
to YottaDB/YDBTest
that referenced
this issue
Apr 30, 2018
Two test framework scripts that were already VIEW "NOISOLATION" were identified. 1) imptp.m 2) tpntpupd.m And they were enhanced to check for the new env var and if present use that and if not do VIEW "NOISOLATION" commands like they previously did. This way all tests that use imptp.m (many dozens) and tpntpupd.m (2 tests) will automatically test the new env var. Took this opportunity to do some "usesimpleapi"-related cleanup in imptp.csh.
nars1
added a commit
to nars1/YottaDBtest
that referenced
this issue
May 1, 2018
…0002703 subtest (due to env var introduced by YottaDB/YDB#218)
nars1
added a commit
to YottaDB/YDBTest
that referenced
this issue
May 2, 2018
…0002703 subtest (due to env var introduced by YottaDB/YDB#218)
nars1
added a commit
that referenced
this issue
Dec 11, 2023
…TRIGGER Background ---------- The below is pasted from https://gitlab.com/YottaDB/DB/YDB/-/issues/1047#description * This is a YDB issue that I noticed while developing an automated test case in YDBTest!1770. **Test case (tcsh commands)** ```sh unsetenv ydb_app_ensures_isolation ydb_statsdir ydb_statshare setenv ydb_gbldir mumps.gld rm -f $ydb_gbldir $ydb_dist/mumps -run GDE exit rm -f mumps.dat $ydb_dist/mupip create setenv ydb_app_ensures_isolation "^x" setenv ydb_statsdir . setenv ydb_statshare 1 echo '+^ax(1,2,2) -commands=K -xecute="set x=1"' | $ydb_dist/mupip trigger -stdin $ydb_dist/mumps -run %XCMD 'view "statshare" zsystem "mupip integ -reg DEFAULT -stats"' ``` **mupip trigger output in Debug build** ```sh $ echo '+^ax(1,2,2) -commands=K -xecute="set x=1"' | $ydb_dist/mupip trigger -stdin . . %YDB-F-ASSERT, Assert failed in sr_port/gvnh_spanreg_init.c line 75 for expression ((reg >= reg_start) && (reg < &addr->regions[addr->n_regions])) ``` **mupip integ output in Release build** ```sh $ $ydb_dist/mumps -run %XCMD 'view "statshare" zsystem "mupip integ -reg DEFAULT -stats"' . . Integ of region default %YDB-F-KILLBYSIGSINFO1, MUPIP process 7447 has been killed by a signal 11 at address 0x00007FF0BCE40C95 (vaddr 0x0000000000000018) %YDB-F-SIGMAPERR, Signal was caused by an address not mapped to an object ``` Issue ----- * The global variable `TREF(ok_to_see_statsdb_regs)` is set to `TRUE` in `sr_port/common_startup_init.c` for most binaries (e.g. yottadb, dse, lke etc.). There are a few exceptions though and they are - sr_port/mupip_create.c : MUPIP CREATE - sr_unix/mupip_cvtgbl.c : MUPIP LOAD - sr_unix/mupip_rundown.c : MUPIP RUNDOWN - sr_unix/mupip_trigger.c : MUPIP TRIGGER - sr_unix/mupip_integ.c : MUPIP INTEG - sr_port/updproc.c : Update process (in the receiver side of a replicated instance) In these cases, this set happens a little later as we don't have enough context about that particular command invocation until a little later. * But as part of commit 684be17 (for #218), support was added for the `ydb_app_ensures_isolation` env var. This specifies a list of global names for which a `VIEW "NOISOLATION"` is done at process startup time. Processing this environment variables requires the global directory to be opened and since this processing was done at process startup as part of commit 684be17, there were a few situations where this processing could happen BEFORE `TREF(ok_to_see_statsdb_regs)` was set to TRUE for the exceptional cases (e.g. mupip rundown or mupip integ). * The logic that opens the global directory does it using `gd_load()` and that relies on the global variable `TREF(ok_to_see_statsdb_regs)` to be appropriately set. * Because the `ydb_app_ensures_isolation` env var processing happened before `TREF(ok_to_see_statsdb_regs)` got set in some cases, the `gd_load()` invocation done as part of the env var processing assumed the default value of `TREF(ok_to_see_statsdb_regs)` to be FALSE and therefore removed all STATSDB regions from the open global directory even though `ydb_statshare` env var was set to 1. * This meant that later when `TREF(ok_to_see_statsdb_regs)` gets set to TRUE, following logic assumes that the STATSDB regions do exist in the global directory but they have already been removed. * This caused assert failures in some cases and SIG-11 in some cases. Fix --- * The fix is to move the `ydb_app_ensures_isolation` env var processing logic from process startup (in `sr_unix/gtm_startup.c`) to when the default global directory is opened (in `sr_port/gvinit.c`). It is okay to defer the processing until the latter stage as we don't need to do any processing on those NOISOLATION globals until we open the global directory.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Final Release Note
A string value of a comma-separated list of global variable names in the environment variable
ydb_app_ensures_isolationinforms YottaDB that application design ensures the transaction property of Isolation for global variables in that list and YottaDB need not do so, potentially increasing application throughput by reducing random, accidental TP restarts. The effect is functionally equivalent to executing the M language commandview "noisolation":"<list>"where"<list>"is the value of the environment variable. For example,export ydb_app_ensures_isolation="^TMP,^XREF"is equivalent to a process executingview "noisolation":"^TMP,^XREF"before it makes any database accesses. Note that using the environment variableydb_app_ensures_isolationrequires the environment variableydb_gbldirto be set to a valid global directory. The setting applies to the global variables mapped by that global directory. (#218)Description
Without this enhancement, informing the database engine where application design ensures Isolation, thus allowing the engine to skip certain restarts that are accidental because of unrelated nodes residing in the same database block requires application code changes. The change provides a mechanism to accomplish this with an environment variable, thus removing the need for application code changs.
Draft Release Note
A string value of a comma-separated list of global variable names in the environment variable
ydb_app_ensures_isolationinforms YottaDB that application design ensures the transaction property of Isolation for global variables in that list and YottaDB need not do so, potentially increasing application throughput by reducing random, accidental TP restarts. The effect is functionally equivalent to executing the M language commandview "noisolation":"<list>"where"<list>"is the value of the environment variable. For example,export ydb_app_ensures_isolation="^TMP,^XREF"is equivalent to a process executingview "noisolation":"^TMP,^XREF"before it makes any database accesses. Note that using the environment variableydb_app_ensures_isolationrequires the environment variableydb_gbldirto be set to a valid global directory. The setting applies to the global variables mapped by that global directory.The text was updated successfully, but these errors were encountered: