From caf3f343860e0935ed8766868fbd74b88da5195b Mon Sep 17 00:00:00 2001 From: Amber Zhang <42762957+Amber1990Zhang@users.noreply.github.com> Date: Mon, 27 Jul 2020 15:42:50 +0800 Subject: [PATCH] Node log (#116) * singlog node log * update configs * fix dan and doodle * fix dan3 * dan4 * fix dan5 --- .../3.configurations/2.configs-syntax.md | 72 ++----------------- .../3.configurations/5.storage-config.md | 8 +-- .../3.configurations/8.single-node-log.md | 63 ++++++++++++++++ .../compact.md | 4 +- docs/manual-EN/README.md | 1 + mkdocs.yml | 1 + 6 files changed, 77 insertions(+), 72 deletions(-) create mode 100644 docs/manual-EN/3.build-develop-and-administration/3.configurations/8.single-node-log.md diff --git a/docs/manual-EN/3.build-develop-and-administration/3.configurations/2.configs-syntax.md b/docs/manual-EN/3.build-develop-and-administration/3.configurations/2.configs-syntax.md index fc936e2ecdf..4bc2d05ea1b 100644 --- a/docs/manual-EN/3.build-develop-and-administration/3.configurations/2.configs-syntax.md +++ b/docs/manual-EN/3.build-develop-and-administration/3.configurations/2.configs-syntax.md @@ -11,68 +11,6 @@ - Restart the services after changing the configuration files to take effect. - Configuration changes in console take effect in real time. - - ## SHOW CONFIGS ```ngql @@ -122,16 +60,18 @@ nebula> GET CONFIGS heartbeat_interval_secs; ----------------------------------------------------------------- ``` -## Update CONFIGS +## UPDATE CONFIGS ```ngql UPDATE CONFIGS [graph|meta|storage :] var = value ``` > The updated CONFIGS will be stored into the `meta service` permanently. -> If the configuration's mode is `MUTABLE`, the change will take effects immediately. Otherwise, if the mode is `REBOOT`, the change will not work until server restart. -> Expression is supported in UPDATE CONFIGS. -For example +> If the configuration's mode is `MUTABLE`, the change will take effect immediately. +> The configurations of some RocksDB parameters take effect after the services are restarted. +> Expressions are supported in the `UPDATE CONFIGS` command. + +For example: ```ngql nebula> UPDATE CONFIGS storage:heartbeat_interval_secs=1; diff --git a/docs/manual-EN/3.build-develop-and-administration/3.configurations/5.storage-config.md b/docs/manual-EN/3.build-develop-and-administration/3.configurations/5.storage-config.md index f34626d6e53..78827832e7c 100644 --- a/docs/manual-EN/3.build-develop-and-administration/3.configurations/5.storage-config.md +++ b/docs/manual-EN/3.build-develop-and-administration/3.configurations/5.storage-config.md @@ -88,7 +88,7 @@ max_background_jobs -- Use multi-thread for compacting. The default ``` -> The above parameters can either be dynamically modified by UPDATE CONFIGS syntax, or written in the local configuration file. Please refer to the RocksDB manual for specific functions and whether restarting is needed. +The parameters above can either be dynamically modified by the `UPDATE CONFIGS` syntax, or written in the local configuration file. Please refer to the RocksDB manual for specific functions and whether restarting is needed. ### `rocksdb_column_family_options` @@ -106,9 +106,9 @@ disable_auto_compactions -- Compact automatically when writing data is sto ``` -> The above parameters can either be dynamically modified by UPDATE CONFIGS syntax, or written in the local configuration file. Please refer to the RocksDB manual for specific functions and whether restarting is needed. +> The preceding parameters can either be dynamically modified by UPDATE CONFIGS syntax, or written in the local configuration file. Please refer to the RocksDB manual for specific functions and whether restarting is needed. -The above parameters can be set via the command line as follows: +The preceding parameters can be set via the command line as follows: ```ngql nebula> UPDATE CONFIGS storage:rocksdb_column_family_options = \ @@ -120,7 +120,7 @@ nebula> UPDATE CONFIGS storage:rocksdb_db_options = \ nebula> UPDATE CONFIGS storage:max_edge_returned_per_vertex = 10; -- The parameter is explained below ``` -Configurations done with the configuration file +We recommend the following configuration: ```text rocksdb_db_options = {"stats_dump_period_sec":"200", "write_thread_max_yield_usec":"600"} diff --git a/docs/manual-EN/3.build-develop-and-administration/3.configurations/8.single-node-log.md b/docs/manual-EN/3.build-develop-and-administration/3.configurations/8.single-node-log.md new file mode 100644 index 00000000000..8660f3b9a0d --- /dev/null +++ b/docs/manual-EN/3.build-develop-and-administration/3.configurations/8.single-node-log.md @@ -0,0 +1,63 @@ +# Logs + +**Nebula Graph** uses [glog](https://github.com/google/glog) to print logs, uses [gflag](https://gflags.github.io/gflags/) to control the severity level of the log, and provides an HTTP interface to dynamically change the log level at runtime to facilitate tracking. + +## Log Directory + +The default log directory is `/usr/local/nebula/logs/`. + +**Note:** If you deleted the log directory during runtime, the runtime log would not continue to be printed. However, this operation will not affect the services. Restart the services to recover the logs. + +## Parameter Description + +### Two most commonly used flags in glog + +- minloglevel: The scale of minloglevel is 0-4. The numbers of severity levels INFO(DEBUG), WARNING, ERROR, and FATAL are 0, 1, 2, and 3, respectively. Usually specified as 0 for debug, 1 for production. If you set the minloglevel to 4, no logs are printed. +- v: The scale of v is 0-3. When the value is set to 0, you can further set the severity level of the debug log. The greater the value is, the more detailed the log is. + +### Configuration Files + +The default severity level for the metad, graphd, and storaged logs can be found in the configuration files (usually in `/usr/local/nebula/etc/`). + +## Check and Change the Severity Levels Dynamically + +Check all the flag values (log values included) of the current gflags with the following command. `curl` is only available when the `local_config` parameter is set to `true`. + +```bash +> curl ${ws_ip}:${ws_port}/get_flags +``` + +In the command: + +- `ws_ip` is the IP address for the HTTP service, which can be found in the configuration files above. The default value is 127.0.0.1. +- `ws_port` is the port for the HTTP service, the default values for `metad`, `storaged`, and `graphd` are 11000, 12000，and 13000, respectively. + +For example, check the minloglevel for the `storaged` service: + +```bash +> curl 127.0.0.1:12000/get_flags | grep minloglevel +``` + +You can also change the logs' severity level to **the most detailed** with the following command. + +```bash +> curl "http://127.0.0.1:12000/set_flags?flag=v&value=4" +> curl "http://127.0.0.1:12000/set_flags?flag=minloglevel&value=0" +``` + +In the Nebula console, check the severity minloglevel of graphd and set it to **the most detailed** with the following commands. + +```ngql +nebula> GET CONFIGS graph:minloglevel; +nebula> UPDATE CONFIGS graph:minloglevel=0; +``` + +To change the severity of the storage log, replace the `graph` in the preceding command with `storage`. + +**Note:** **Nebula Graph** only supports modifying the graph and storage log severity by using console. And the severity level of meta logs can only be modified with the `curl` command. + +**Close** all logs print (FATAL only) with the following command. + +```bash +> curl "http://127.0.0.1:12000/set_flags?flag=minloglevel&value=4" +``` diff --git a/docs/manual-EN/3.build-develop-and-administration/5.storage-service-administration/compact.md b/docs/manual-EN/3.build-develop-and-administration/5.storage-service-administration/compact.md index c08cc60bf56..62b2de6faa1 100644 --- a/docs/manual-EN/3.build-develop-and-administration/5.storage-service-administration/compact.md +++ b/docs/manual-EN/3.build-develop-and-administration/5.storage-service-administration/compact.md @@ -2,13 +2,13 @@ This document will walk you through the concept of Compact. -1. Start or stop compact. This method calls the default RocksDB compact. You can use it to merge sst files in small scale during data writing to ensure the read speed in a short time. This kind of compact is usually performed in the day time. +- The default RocksDB compact style. You can use the `UPDATE CONFIG` statement as follows to start or stop it. It merges sst files in small scale during data writing to speed up the data reading in a short time. When enabled, this compaction is automatically performed during the daytime. ```ngql nebula> UPDATE CONFIG storage:disable_auto_compactions=false/true; ``` -2. Start compact with the `SUBMIT JOB COMPACT` command. This method calls the customized compact of **Nebula Graph**. You can use it to perform large scale background operations such as sst files merging in large scale or TTL. This kind of compact is usually performed after midnight. +- The customized compact style for **Nebula Graph**. You can run the `SUBMIT JOB COMPACT` command to start it. You can use it to perform large scale background operations such as sst files merging in large scale or TTL. This kind of compact is usually performed after midnight. In addition, you can modify the number of threads in both methods by the following command. You can decrease the threads during daytime and increase it at night. diff --git a/docs/manual-EN/README.md b/docs/manual-EN/README.md index 7143a421d10..e848a9d0eee 100644 --- a/docs/manual-EN/README.md +++ b/docs/manual-EN/README.md @@ -123,6 +123,7 @@ It is the optimal solution in the world capable of hosting graphs with dozens of * [Storaged Configuration](3.build-develop-and-administration/3.configurations/5.storage-config.md) * [Console Configuration](3.build-develop-and-administration/3.configurations/6.console-config.md) * [Kernel Configuration](3.build-develop-and-administration/3.configurations/7.kernel-config.md) + * [Single Node Log](3.build-develop-and-administration/3.configurations/8.single-node-log.md) * Account Management Statement * [Alter User Syntax](3.build-develop-and-administration/4.account-management-statements/alter-user-syntax.md) * [Authentication](3.build-develop-and-administration/4.account-management-statements/authentication.md) diff --git a/mkdocs.yml b/mkdocs.yml index d61ef5cb95f..02e1d904410 100755 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -164,6 +164,7 @@ nav: - Storaged Configuration: manual-EN/3.build-develop-and-administration/3.configurations/5.storage-config.md - Console Configuration: manual-EN/3.build-develop-and-administration/3.configurations/6.console-config.md - Kernel Configuration: manual-EN/3.build-develop-and-administration/3.configurations/7.kernel-config.md + - Single Node Log: manual-EN/3.build-develop-and-administration/3.configurations/8.single-node-log.md - Account Management Statement: - Alter User Syntax: manual-EN/3.build-develop-and-administration/4.account-management-statements/alter-user-syntax.md - Authentication: manual-EN/3.build-develop-and-administration/4.account-management-statements/authentication.md