ydb script to run YottaDB, and files ydb_env_set & ydb_env_unset to source to set & clear environment

[ksbhaskar]

Final Release Note

For "out of the box" use of YottaDB, ydb_env_set is a file you can source with a POSIX or compatible shell to configure an environment with a default structure and required environment variables, creating a default environment if one does not exist.

To set up the environment for a YottaDB quick start execute:

source /usr/local/lib/yottadb/r#.##/ydb_env_set

where r#.## is the YottaDB release identifier. Thereafter:

• C users can simply run their C programs compiled against this YottaDB release. [Each YottaDB release requires C programs to be recompiled after including the libyottadb.h file included in that release.]

• M users can run ydb <command line> where <command line> is as described in the M Programmers Guide. [ydb -help outputs a list of options.]

Advanced users and production deployments will likely prefer to create their own scripting instead of using that provided here.

Every YottaDB environment variable (with ydb_ prefixes) has a GT.M counterpart (with gtm or occasionally GTM prefixes). The general YottaDB philosophy is to use a YottaDB environment variable if one is defined, otherwise to use its GT.M counterpart, if it is defined, and otherwise to use a default value (including no value, if that is the default). At this time, support has not been implemented in YottaDB for all ydb_ prefixed envronment varables, but each release will increase that set. So as to not require changes as future YottaDB releases add YottaDB counterparts to GT.M environment variables, sourcing ydb_env_set sets both sets of environment variables to appropriate values, which are usually, but not always, the same. Except where otherwise noted below, the general approach to assigning values ydb_env_set takes is:

• If the ydb_ prefixed YottaDB environment variable has a value, leave it unchanged, but set the corresponding GT.M environment variable to match.
• If the ydb_ prefixed environment variable is not defined, but its gtm prefixed counterpart is, then set the ydb_ prefixed variable from the gtm prefixed counterpart.
• If neither is set, then set both to correct, or appropriate, default values.

Environment variables whose values may optionally be provided before sourcing ydb_env_set are:

• ydb_chset (gtm_chset) - defines whether the M language subsystem of YottaDB should operate in M mode or UTF-8 mode. If neither is set, sourcing the ydb_env_set script sets neither. Any value other than a case-independent "UTF-8" generates an error, and any lower case letters are converted to upper case.

• ydb_icu_version (gtm_icu_version) - if $ydb_chset is "UTF-8". $gtm_icu_version defines the version of ICU (International Components for Unicode) to be used. The environment variable is ignored if $ydb_chset is not "UTF-8". The default is the value returned by the command icu-config --version

Environment variables that sourcing ydb_env_set sets, without regard to any previous value, are:

• ydb_dist (gtm_dist) - the directory where YottaDB is installed

• ydb_rel - the value of the second and fourth pieces of the intrinsic special variable $zyrelease of the YottaDB release, separated by an underscore (_).

• gtmver - the value of the second and fourth pieces of the intrinsic special variable $zversion of the YottaDB release, separated by an underscore (_)..

• ydb_tmp_chset, ydb_tmp_gtm_dist, ydb_tmp_rel, ydb_tmp_routines, ydb_tmp_passwd, and ydb_tmp_ver - temporary variables used by the script and set to the initial values (if any) of ydb_chset (gtm_chset), gtm_dist, ydb_rel, ydb_routines (gtmroutines), and ydb_passwd (gtm_passwd), and gtmver.

Environment variables that sourcing ydb_env_set initializes with reasonable default values if they are unset, and potentially modifies if set, are:

• LD_LIBRARY_PATH - If there is an existing directory in $LD_LIBRARY_PATH of the form /usr/local/lib/yottadb/* replaces it with $ydb_dist; otherwise appends $ydb_dist to LD_LIBRARY_PATH.

• ydb_dir (gtmdir) - a directory with a standard structure, as described below:

  • If both ydb_dir and gtmdir are defined:

    • If both directories $ydb_dir and $gtmdir exist, neither environment variable is modified.
    • If one exists and the other does not, create a symbolic link from the one that does not exist to the one that does.
    • If neither exists, set ydb_dir to $HOME/.yottadb, create a directory there with the standard structure and create a symbolic link at $gtmdir to point to $ydb_dir.

  • If one is defined and the other is not, set the one that is not defined to the value of the one that is.

  • If neither is defined, set ydb_dir to $HOME/.yottadb and create a directory there with the standard structure. Set gtmdir to $HOME/.fis-gtm and create a symbolic link to .yottadb.

• ydb_gbldir (gtmgbldir) - the initial value for the intrinsic special variable $zgbldir, defaulting to $ydb_dir/$ydb_rel/g/yottadb.gld.

• ydb_routines (gtmroutines) - the initial value for the intrinsic special variable $zroutines in three parts:

  1. $ydb_dir/$ydb_rel/o/utf8*($ydb_dir/$ydb_rel/r $ydb_dir/r) if $ydb_chset is "UTF-8" and $ydb_dir/$ydb_rel/o*($ydb_dir/$ydb_rel/r $ydb_dir/r) otherwise. Note that "*" is literally in the string, and does not represent filename expansion.
  2. If there are shared libraries in the selected directories then $gtm_dist/plugin/o/utf8/*.so if $ydb_chset is "UTF-8", or $gtm_dist/plugin/o/*.so otherwise. In this case "*" represents expanded file names, rather than a literal character in the string.
  3. $gtm_dist/libgtmutil.so.

• ydb_log (gtm_log) - a directory where the gtmsecshr program logs its information in the file gtm_secshr_log, defaulting to /tmp/yottadb/$ydb_rel.

• ydb_principal_editing (gtm_principal_editing) - deviceparameters at process startup for $principal, defaulting to "EDITING".

• ydb_retention - the number of days after which the ydb script deletes prior generation journal files and log files in $ydb_tmp generated by commands within the ydb script, defaulting to 42.

• ydb_tmp - a temporary directory in which commands within the ydb script log their use, defaulting to /tmp/yottadb/$ydb_rel.

Sourcing ydb_env_set has no effect on any environment variables except those noted above.

The standard structure for the directory under $ydb_dir has the following subdirectories. Note that routines and object files are relevant only for YottaDB's M language subsystem.

• $ydb_dir/r for routine source files that are not specific to a YottaDB release.

• $ydb_dir/$ydb_rel ($ydb_dir/$gtmver) for files specific to a YottaDB release. Typically, $ydb_dir/$gtmver is a symbolic link to $ydb_dir/$ydb_rel, but if the former subdirectory exists already, sourcing ydb_env_set creates a symbolic link from the latter to the former. Note that $ydb_dir may well have multiple $ydb_rel subdirectories, each corresponding to a different YottaDB release. $ydb_dir/$ydb_rel has the following subdirectores.

  • $ydb_dir/$ydb_rel/g - for a global directory (default name yottadb.gld), database files (default name yottadb.dat) and current and prior generation journal files (default names yottadb.mjl and yottadb.mjl_<timestamp>.

  • $ydb_dir/$ydb_rel/o - for object files compiled in M mode, and $ydb_dir/$ydb_rel/o/utf8 for object files compiled in UTF-8 mode.

  • $ydb_dir/$ydb_rel/r - for M routine source files that are specific to the YottaDB release $ydb_rel.

Sourcing ydb_env_set sets the following aliases:

• dse - aliased to $gtm_dist/dse
• gde - aliased to $gtm_dist/mumps -run GDE
• lke - aliased to $gtm_dist/lke
• mumps - aliased to $gtm_dist/mumps
• mupip - aliased to $gtm_dist/mupip

Sourcing ydb_env_unset unsets the above environment variables, unsets the aliases, and removes any occurrence of $ydb_dist in $LD_LIBRARY_PATH.

The ydb script does the following:

• Sources ydb_env_set.
• Runs mupip journal -recover -backward "*"
• Runs mupip set -journal="on,before" -region "*".
• Deletes any prior generation journal files in $ydb_dir/$ydb_rel/g modified more than $ydb_retention days previously.
• Runs $ydb_dist/mumps $*
• Executes mupip rundown -region "*"
• Deletes any files in $ydb_tmp generated by $USER and modified more than than $ydb_retention days previously.

Description

ydb_env_set is a file that can be sourced from a shell (ideally /bin/sh; requiring bash is acceptable) that creates an environment if one does not exist, and sets the environment variables needed to run YottaDB. ydb is a script that runs YottaDB M programs and the interactive mode, relying on ydb_env_set to ensure that an environment exists and that environment variables have appropriate values.

ydb_env_set and ydb are replacements for the gtmprofile and gtm files.

Draft Release Note

For "out of the box" use of YottaDB, ydb_env_set is a file you can source with a POSIX or compatible shell to configure an environment with a default structure and required environment variables, creating a default environment if one does not exist.

To set up the environment for a YottaDB quick start execute:

source /usr/local/lib/yottadb/r#.##/ydb_env_set

where r#.## is the YottaDB release identifier. Thereafter:

• C users can simply run their C programs compiled against this YottaDB release. [Each YottaDB release requires C programs to be recompiled after including the libyottadb.h file included in that release.]

• M users can run ydb <command line> where <command line> is as described in the M Programmers Guide. [ydb -help outputs a list of options.]

Advanced users and production deployments will likely prefer to create their own scripting instead of using that provided here.

With the exception of gtm_dist, every YottaDB environment variable (with ydb_ prefixes) has, or will have, a GT.M counterpart (with gtm or occasionally GTM prefixes). So as to not require changes as future YottaDB releases add YottaDB counterparts to GT.M environment variables, sourcing ydb_env_set sets both sets of environment variables to appropriate values, which are usually, but not always, the same. Except where otherwise noted below, the general approach to assigning values is:

• If the ydb_ prefixed YottaDB environment variable has a value, leave it unchanged, but set the corresponding GT.M environment variable to match.
• If the ydb_ prefixed environment variable is not defined, but its gtm prefixed counterpart is, then set the ydb_ prefixed variable from the gtm prefixed counterpart.
• If neither is set, then set both to correct, or appropriate, default values.

Environment variables whose values may optionally be provided before sourcing ydb_env_set are:

• ydb_chset (gtm_chset) - defines whether the M language subsystem of YottaDB should operate in M mode or UTF-8 mode. If neither is set, sourcing the ydb_env_set script sets neither. Any value other than a case-independent "UTF-8" generates an error, and any lower case letters are converted to upper case.

• ydb_icu_version (gtm_icu_version) - if $ydb_chset is "UTF-8". $gtm_icu_version defines the version of ICU (International Components for Unicode) to be used. The environment variable is ignored if $ydb_chset is not "UTF-8". The default is the value returned by the command icu-config --version

Environment variables that sourcing ydb_env_set sets, without regard to any previous value, are:

• gtm_dist - the directory where YottaDB is installed

• ydb_rel - the value of the second and fourth pieces of the intrinsic special variable $zyrelease of the YottaDB release separated by an underscore (_).

• gtmver - the value of the second and fourth pieces of the intrinsic special variable $zversion of the YottaDB release separated by an underscore (_)..

• ydb_tmp_chset, ydb_tmp_gtm_dist, ydb_tmp_rel, ydb_tmp_routines, ydb_tmp_passwd, and ydb_tmp_ver - temporary variables used by the script and set to the initial values (if any) of ydb_chset (gtm_chset), gtm_dist, ydb_rel, ydb_routines (gtmroutines), and ydb_passwd (gtm_passwd), and gtmver.

Environment variables that sourcing ydb_env_set initializes with reasonable default values if they are unset, and potentially modifies if set, are:

• LD_LIBRARY_PATH - If there is an existing directory in $LD_LIBRARY_PATH of the form /usr/local/lib/yottadb/* replaces it with $gtm_dist; otherwise appends $gtm_dist to LD_LIBRARY_PATH.

• ydb_dir (gtmdir) - a directory with a standard structure is defined as described below:

  • If both ydb_dir and gtmdir are defined:

    • If both directories $ydb_dir and $gtmdir exist, neither environment variable is modified.
    • If one exists and the other does not, create a symbolic link from the one that does not exist to the one that does.
    • If neither exists, set ydb_dir to $HOME/.yottadb, create a directory there with the standard structure and create a symbolic link at $gtmdir to point to $ydb_dir.

  • If one is defined and the other is not, set the one that is not defined to the value of the one that is.

  • If neither is defined, set ydb_dir to $HOME/.yottadb and create a directory there with the standard structure. Set gtmdir to $HOME/.fis-gtm and create a symbolic link to .yottadb.

• ydb_gbldir (gtmgbldir) - the initial value for the intrinsic special variable $zgbldir, defaulting to $ydb_dir/$ydb_rel/g/yottadb.gld.

• ydb_routines (gtmroutines) - the initial value for the intrinsic special variable $zroutines.

  • If there is an existing value, sourcing ydb_env_set converts any occurrence of $ydb_tmp_routines or $gtmver to $ydb_rel.

  • If there is no existing value, sourcing ydb_env_set creates a default value in three parts:

    1. $ydb_dir/$ydb_rel/o/utf8*($ydb_dir/$ydb_rel/r $ydb_dir/r) if $ydb_chset is "UTF-8" and $ydb_dir/$ydb_rel/o*($ydb_dir/$ydb_rel/r $ydb_dir/r) otherwise. Note that "*" is literally in the string, and does not represent filename expansion.
    2. If there are shared libraries in the selected directories then $gtm_dist/plugin/o/utf8/*.so if $ydb_chset is "UTF-8", or $gtm_dist/plugin/o/*.so otherwise. In this case "*" represents expanded file names, rather than a literal character in the string.
    3. $gtm_dist/libgtmutil.so.

• ydb_log (gtm_log) - a directory where the gtmsecshr program logs its information in the file gtm_secshr_log, defaulting to /tmp/yottadb/$ydb_rel.

• ydb_principal_editing (gtm_principal_editing) - deviceparameters at process startup for $principal, defaulting to "EDITING".

• ydb_retention - the number of days after which the ydb script deletes prior generation journal files and log files in $ydb_tmp generated by commands within the ydb script, defaulting to 42.

• ydb_tmp - a temporary directory in which commands within the ydb script log their use, defaulting to /tmp/yottadb/$ydb_rel.

Sourcing ydb_env_set has no effect on any environment variables except those noted above.

The standard structure for the directory under $ydb_dir has the following subdirectories. Note that routines and object files are relevant only for YottaDB's M language subsystem.

• $ydb_dir/r for routine source files that are not specific to a YottaDB release.

• $ydb_dir/$ydb_rel ($ydb_dir/$gtmver) for files specific to a YottaDB release. Typically, $ydb_dir/$gtmver is a symbolic link to $ydb_dir/$ydb_rel, but if the former subdirectory exists already, sourcing ydb_env_set creates a symbolic link from the latter to the former. Note that $ydb_dir may well have multiple $ydb_rel subdirectories, each corresponding to a different YottaDB release. $ydb_dir/$ydb_rel has the following subdirectores.

  • $ydb_dir/$ydb_rel/g - for a global directory (default name yottadb.gld), database files (default name yottadb.dat) and current and prior generation journal files (default names yottadb.mjl and yottadb.mjl_<timestamp>.

  • $ydb_dir/$ydb_rel/o - for object files compiled in M mode, and $ydb_dir/$ydb_rel/o/utf8 for object files compiled in UTF-8 mode.

  • $ydb_dir/$ydb_rel/r - for M routine source files that are specific to the YottaDB release $ydb_rel.

Sourcing ydb_env_set sets the following aliases:

• dse - aliased to $gtm_dist/dse
• gde - aliased to $gtm_dist/mumps -run GDE
• lke - aliased to $gtm_dist/lke
• mumps - aliased to $gtm_dist/mumps
• mupip - aliased to $gtm_dist/mupip

Sourcing ydb_env_unset unsets the above environment variables, unsets the aliases, and removes any occurrence of $gtm_dist in $LD_LIBRARY_PATH.

The ydb script does the following:

• Sources ydb_env_set.

• Runs mupip dumpfhead on the database file for the DEFAULT region. If the database is not already open:

  • If the database is replicated:

    • Runs mupip journal -rollback -online -backward "*" (the -online flag is required to allow for another process
    • Starts a passive Source Server to a "dummy" secondary instance.
    • Runs mupip set -journal="on,before" -region "*".

  • If the database is not replicated:

    • Runs mupip journal -recover -online -backward "*" (the -online flag is required to allow for another process

  • Whether the database is replicated or not, after the above:

    • Runs mupip set -journal="on,before" -region "*".
    • Deletes any prior generation journal files in $ydb_dir/$ydb_rel/g modified more than $ydb_retention days previously.

• Runs $gtm_dist/mumps $*

• Executes mupip rundown -region "*"

• Deletes any files in $ydb_tmp generated by $USER and modified more than than $ydb_retention days previously.