diff --git a/docs/README.md b/docs/README.md index 1f364442..918340cf 100644 --- a/docs/README.md +++ b/docs/README.md @@ -56,6 +56,7 @@ English | [中文](README.zh_CN.md) * [custom metrics plugin](./en/custom_metrics.md) * [prometheus](./en/prometheus_metrics.md) * Logging + * [local logging tools](./en/local_logging.md) * [custom logging plugin](./en/custom_logging.md) * [cls](https://github.com/trpc-ecosystem/cpp-logging-cls/blob/main/README.md) * Tracing diff --git a/docs/README.zh_CN.md b/docs/README.zh_CN.md index 07d68c32..f21b14f5 100644 --- a/docs/README.zh_CN.md +++ b/docs/README.zh_CN.md @@ -55,6 +55,7 @@ * [开发自定义metrics插件](./zh/custom_metrics.md) * [prometheus](./zh/prometheus_metrics.md) * Logging插件 + * [本地日志工具](./zh/local_logging.md) * [开发自定义logging插件](./zh/custom_logging.md) * [cls](https://github.com/trpc-ecosystem/cpp-logging-cls/blob/main/README.zh_CN.md) * Tracing插件 diff --git a/docs/en/local_logging.md b/docs/en/local_logging.md new file mode 100644 index 00000000..b123c878 --- /dev/null +++ b/docs/en/local_logging.md @@ -0,0 +1,403 @@ +# Overview +Logs are an important part of software development, and good logs are an important source of information for analyzing the running state of software. The tRPC-Cpp framework provides a complete solution for log printing, collection, and reporting for businesses. The purpose of this article is to provide users with the following information: + +- What logging features does the framework provide? +- How does the framework implement these features? +- How to use the standard logging interface? +- How to configure log settings? + +Terminology: +- Instance: Equivalent to logger, that is, multiple instances means multiple loggers +- Flow style: Refers to the use of TRPC_FLOW_LOG and TRPC_FLOW_LOG_EX log macros + +# Feature Introduction +## Overview of Features + +As shown in the figure, the tRPC-Cpp log module is based on SpdLog implementation, which includes two aspects: `log printing programming interface` and `docking with the log output end`. Logger and Sink concepts are introduced in the implementation. +![architecture design](../images/log_design.png) + +Logger and Sink are responsible for log printing and docking with the log service system, respectively. They both adopt plug-in programming and can be customized. Users can refer to[how to develop custom log plugins](./custom_logging.md).The definitions of Logger and Sink are as follows: +- Logger is responsible for implementing common interfaces related to log printing. Logger adopts plug-in programming and supports multi-Logger functions. Users can choose their logger to achieve differentiated log printing and collection. It is docked with the log backend through Sink plug-ins. +- Sink, also known as "log output end", is responsible for log collection and reporting functions, including log reporting format and docking with the backend log system, etc. Each Logger can have multiple Sinks, for example, Default, the default logger, can support output to local logs and remote cls at the same time. Sink adopts plug-in programming and can be flexibly expanded. + +The tRPC-Cpp framework provides four styles of log printing functions: +- Python style: The most elegant and convenient, recommended.[Detailed syntax](https://fmt.dev/latest/syntax.html) +- Printf style: Printf syntax, type-safe.[Detailed syntax](https://en.wikipedia.org/wiki/Printf_format_string) +- Stream style: Compatible with the old version, the framework prints logs in a more cumbersome way, not recommended for users. +- Flow style: Its underlying layer uses the stream style, but it can support inputting context and specifying logger. It is recommended for scenarios where business logs are printed separately from framework logs. + +## Log Printing Interface +First, the framework refers to industry standards to divide the log printing levels from low to high as follows: +| Level | Description | +| :--- | :---: | +| **trace** | The main function of this level of log is to accurately record the running status of every step of specific events in the system| +| **debug** | Indicates that fine-grained information events are very helpful for debugging applications, mainly used for printing some running information during development| +| **info**| Messages at the coarse-grained level emphasize the running process of the application. Print some interesting or important information, which can be used in the production environment to output some important information of the program running, but do not abuse it to avoid printing too many logs| +|**warn** |Indicates potential error situations, some information is not error information, but also needs to give some hints to the programmer| +|**error** |Indicates that although an error event occurs, it does not affect the continued operation of the system. Print error and exception information. If you do not want to output too many logs, you can use this level| +| **critical**|Indicates that every serious error event will cause the application to exit. This level is relatively high. Major errors, at this level you can stop the program directly| + +To accommodate different users' habits, the framework provides log macros similar to `python`, `printf`, `stream`, and `flow` for each level of log functions. The examples here are just to give users a rough understanding, and more detailed log macros are introduced below. + +- Recommended usage: Python-style log macros, as the performance of fmt::format is about 20% higher than std::ostream. + +``` +// Python style +TRPC_FMT_DEBUG("msg:{}", "test"); +// Printf style +TRPC_PRT_DEBUG("msg: %s", "test"); +// Stream style +TRPC_LOG_DEBUG("msg:" << "test"); +// Flow style +TRPC_FLOW_LOG_EX(ctx, "custom1", "msg:" << "test"); Can specify the input ctx and logger name +``` + +### 2.2.1 Default Log Macro Interface +- Macro interface description: It uses the default logger and has only one parameter, which represents the log content to be printed. tRPC-Cpp framework logs are based on this. +- Usage scenarios: Framework logs, debug logs +```cpp + int i = 0; + // Python-like + TRPC_FMT_TRACE("trpc-{}: {}", "cpp"); // trpc-cpp + TRPC_FMT_DEBUG("print floats {:03.2f}", 1.23456); // "print floats 001.23" + TRPC_FMT_INFO("hello trpc-{}: {}", "cpp", ++i); // "hello trpc-cpp: 2" + TRPC_FMT_ERROR("{} is smaller than {}?", 1, 2); // "1 is smaller than 2?" + TRPC_FMT_CRITICAL("coredump msg: {}?", "oom"); // "coredump msg: oom" + + // Printf-like + TRPC_PRT_INFO("just like %s: %d", "printf", ++i); // "just like printf: 3" + ... + + // Stream-like + TRPC_LOG_INFO("stream trace: " << ++i); // "stream trace: 1" + ... +``` +### Log Macro Interface with Context +- Macro interface description: It has 2 parameters, the first one is context, and the second one is log content. It should be noted that the context here is the base class of ClientContext and ServerContext, so it is applicable on both the client and server sides. + +```cpp +TRPC_LOG_TRACE_EX(context, msg); +TRPC_LOG_DEBUG_EX(context, msg); +TRPC_LOG_INFO_EX(context, msg); +TRPC_LOG_WARN_EX(context, msg); +TRPC_LOG_ERROR_EX(context, msg); +TRPC_LOG_DEBUG_EX(context, msg); + +// The log level of the framework is fixed at the info level +TRPC_FLOW_LOG_EX(context, instance, msg) +// Example +TRPC_FLOW_LOG_EX(ctx, "custom", "msg：" << "test") // ctx is the context object, custom is the logger name, output: msg: test +``` +Explanation: +instance refers to the name of the logger, users can specify different loggers through this macro, and can also get the context information, making the interface more flexible. +The log level of the framework is fixed at the info level, if users want to modify the log level, they can set it based on `TRPC_STREAM`, or they can use it to encapsulate a log macro, here is the implementation of `TRPC_FLOW_LOG` +``` +#define TRPC_FLOW_LOG(instance, msg) TRPC_STREAM(instance, trpc::Log::info, nullptr, msg) +#define TRPC_FLOW_LOG_EX(context, instance, msg) \ + TRPC_STREAM(instance, trpc::Log::info, context, msg) +``` + +### 2.2.3 Log Macro Interface with Specified Logger +- Macro interface description: Can specify logger, customize log output. +- Usage scenarios: +- Separate business logs from framework logs +- Different business logs specify different loggers +- Business logs output to remote + +Any of the above scenarios, or any combination of them, can be implemented through the log macro interface with a specified logger. +``` + // Flow style macro + TRPC_FLOW_LOG(instance, msg) + TRPC_FLOW_LOG_EX(context, instance, msg) + // Example + TRPC_FLOW_LOG("custom1", "hello" << "trpc") // hello trpc + TRPC_FLOW_LOG_EX("server_context", "custom2", "hello" << "trpc") // hello trpc + + // fmt style macro interface + TRPC_LOGGER_FMT_TRACE(instance, format, args...) + TRPC_LOGGER_FMT_DEBUG(instance, format, args...) + TRPC_LOGGER_FMT_INFO(instance, format, args...) + TRPC_LOGGER_FMT_WARN(instance, format, args...) + TRPC_LOGGER_FMT_ERROR(instance, format, args...) + TRPC_LOGGER_FMT_CRITICAL(instance, format, args...) + // Example + TRPC_LOGGER_FMT_WARN("custom3", "i am not {} logger", "default"); // "i am not default logger" +``` + +### Log Macro Interface with IF Condition +- Macro interface description: Conditional log macro, i.e., when the condition is met, the log is output, otherwise, it is not output. +- Usage scenario 1: Logs are output only if a custom condition is met +``` + // fmt style + TRPC_FMT_TRACE_IF(condition, format, args...) + TRPC_FMT_DEBUG_IF(condition, format, args...) + TRPC_FMT_INFO_IF(condition, format, args...) + TRPC_FMT_WARN_IF(condition, format, args...) + TRPC_FMT_ERROR_IF(condition, format, args...) + TRPC_FMT_CRITICAL_IF(condition, format, args...) + // Example + TRPC_FMT_INFO_IF(true, "if true, print: {}", "msg"); // "if true, print: msg" + + // printf style + TRPC_PRT_TRACE_IF(condition, format, args...) + TRPC_PRT_DEBUG_IF(condition, format, args...) + TRPC_PRT_INFO_IF(condition, format, args...) + TRPC_PRT_WARN_IF(condition, format, args...) + TRPC_PRT_ERROR_IF(condition, format, args...) + TRPC_PRT_CRITICAL_IF(condition, format, args...) + // Example + TRPC_PRT_INFO_IF(true, "if true, print: %s", "msg"); // "if true, print: msg" + + // stream style + TRPC_LOG_TRACE_IF(condition, msg) + TRPC_LOG_DEBUG_IF(condition, msg) + TRPC_LOG_INFO_IF(condition, msg) + TRPC_LOG_WARN_IF(condition, msg) + TRPC_LOG_ERROR_IF(condition, msg) + TRPC_LOG_CRITICAL_IF(condition, msg) + // Example + TRPC_LOG_INFO_IF(true, "if true, print: " << "msg"); // "if true, print: msg" + + // All the above macro interfaces can also be used in combination with LOGGER/Context, for example: + TRPC_LOGGER_FMT_INFO_IF(true, "if true, print: {}", "msg"); // "if true, print: msg" + TRPC_LOGGER_PRT_INFO_EX(context, true, "if true, print: %s", "msg"); // "if true, print: msg" + TRPC_LOGGER_LOG_INFO_IF_EX(context, true, "if true, print: " << "msg"); // "if true, print: msg" +``` +- Usage scenario 2: Users want to output logs but do not want to output too many logs causing performance degradation. +- In this scenario, you can use conditional macro operators +- TRPC_EVERY_N(c): Set this log to output at most 1 print for every c triggers, the first time must be triggered. +- TRPC_WITHIN_N(ms): Set this log to output at most 1 time within the set ms milliseconds if there are multiple triggers. The first time must be triggered. Usage examples are as follows +``` +TRPC_FMT_INFO_IF(TRPC_EVERY_N(1000), "condition every 1000"); # Print a log for every 1000 calls +TRPC_FMT_INFO_IF(TRPC_WITHIN_N(1000), "condition within 1000 ms"); # If there is more than one call within 1000ms, print at most one log +``` + +## Multiple Logger Support +tRPC-Cpp supports multiple loggers at the same time, with each logger having different log levels, print formats, reporting output backends, and businesses being able to set different loggers for different business scenarios. For example: + +- Different business modules use different log files for storage +- Different events, based on the degree of attention to events, use different log levels to collect logs + +The multi-Logger feature greatly increases the flexibility of log usage. + +## Multiple Sink Support +For the same logger, the framework also provides the ability to output logs to multiple log output backends (referred to as "sink") at the same time. Since the sink is also registered to the framework as a plugin, users can choose local, remote (such as cls, smart research logs, etc.), or none. The framework currently supports 4 output endpoints (Sink): +- Local rolling file (LocalFileSink): Configuration name is `local_file` +- Local console output (StdoutSink): Configuration name is stdout. + The above sink belongs to 2 different types: +- DefaultSink： + - The above LocalFileSink and StdoutSink are examples. + - The configuration is written under the sinks field. + - Requires returning a spdsink. + - All DefaultSinks belonging to an instance are attached to the built-in spdlogger of DefaultLog, so the received messages are the same. +- RawSink + - The clsLogSink mentioned above is an example. + - The configuration is written under the raw_sinks field. + - Requires implementing a log interface, receiving an additional context parameter, called synchronously by DefaultLog, and ensuring its own thread safety. + - Its behavior is completely defined by itself. Currently, clsLogSink creates a new spdlogger and spdsink internally and decides how to output based on the context. + +It should be emphasized that the log print settings are configured at the Sink level. Users need to configure each output endpoint, such as: the same log needs to be saved in the local log file and output to cls, then the local_file and cls_log sinks must be configured separately. + +# Log Configuration +## Configuration Structure +First, let's look at the overall structure of the log configuration. Since the log module implements both logger and sink using a plugin approach, log configurations must be placed in the "plugins" area. +```yaml +plugins: + log: + default: + - name: default # Default logger name + ... + sinks: # DefaultSink type plugins + local_file: # Local log sink + ... + raw_sinks: # RawSink type plugins + cls_log: # cls log sink + - name: custom1 # Custom logger name + ... + sinks: + local_file: + ... + raw_sinks: + cls_log: +``` +Explanation: +- The above configuration reflects the multi-logger and multi-sink features. "default" and "custom1" represent the logger names. Each logger configuration is an array of sinks. +- line 4: default represents the framework default logger +- line 6: sinks indicate that the configuration contains DefaultSink type plugins, such as local_file +- line 9: raw_sink indicates that the configuration contains RawtSink type plugins, such as cls_log + +## Logger Configuration +Logger configuration is usually inherited directly from the sink it manages, so some fields in the sink can be left unconfigured, such as format, to simplify configuration. First, let's introduce its parameter configuration, designed as follows. +|Configuration item | Type | Required | Default value |Configuration explanation| +| :--- | :---: | :---: | :---: | :---: | +| name | string | Optional | Empty | Logger name | +| min_level | int | Optional | 2 | Set the minimum log level, logs will only be output if the log level is greater than this | +| format | string | Optional | Empty | [%Y-%m-%d %H:%M:%S.%e] [thread %t] [%l] [%@] %v | +| mode | int | Optional | 2 | Optional: 1 Synchronous, 2 Asynchronous, 3 Extreme speed | + +Explanation： +format: Refer to [Format Description](https://github.com/gabime/spdlog/wiki/3.-Custom-formatting) +mode: +- Asynchronous：When printing logs, store the logs in a thread-safe cache queue and return directly. There is a separate thread inside to consume this queue and output to each output endpoint. If the queue is full, the output is blocked; if the queue is empty, the consumption is blocked. Recommended usage. +- Synchronous：When printing logs, directly traverse each output endpoint and wait for completion, +- Extreme speed: Basically the same as asynchronous mode, the only difference is that the output never blocks. Once the queue is full, the oldest log is discarded, and the current log is stored and returned. + +``` +plugins: + log: + default: + - name: default + min_level: 1 # 0-trace, 1-debug, 2-info, 3-warn, 4-error, 5-critical + format: "[%Y-%m-%d %H:%M:%S.%e] [thread %t] [%l] [%@] [%!] %v" + mode: 2 # 1-synchronous, 2-asynchronous, 3-extreme speed + sinks: + ... +``` + +## Sink Configuration +### Use Local Log +#### Local File Log +When the sink is set to "local_file", it means that the log is printed to the local log file. First, let's introduce its configuration items, as follows: +|Configuration item | Type | Required | Default value | Configuration explanation | +| :--- | :---: | :---: | :---: | :---: | +| format | string | Optional | Empty | Log output format, not filled in by default inherits logger | +| eol | bool | Optional | true | Whether to wrap when formatting output logs | +| filename | string | Required | Empty | Local log file name, default to the current path, can include the path | +| roll_type | string | Optional | by_size | Set the log segmentation method, default to by_size, optional by_day-day segmentation, by_hour-hour segmentation | +| reserve_count | int | Optional | 10 | Rolling log retention file count | +| roll_size | int | Optional | 100 x 1024 x 1024 | Only set this variable for by_size type, which represents the size of a rolling file | +| rotation_hour | int | Optional | 0 | Only set this variable for by_day type, which represents the cut-off time by day, the specific time is specified by rotation_hour:rotation_minute | +| rotation_minute | int | Optional | 0 | | +| remove_timout_file_switch | bool | Optional | false | Only set this variable for by_hour type, under the day segmentation mode, the flag for deleting outdated files | +| hour_interval | int | Optional | 1 | | hour_interval | int | Optional | 1 | This variable is set for the by_hour type and represents the interval in hours between the hours || + +Local log file output supports three output modes: +- By size: by_size +- By day: by_day +- By hour: by_hour + +The following introduces how to configure each +##### Output mode by size +```yaml +plugins: + log: + default: + - name: custom # Can be the default default, can use other logger names + format: "[%Y-%m-%d %H:%M:%S.%e] [thread %t] [%l] [%@] %v" + sinks: + local_file: # Local log sink + roll_type: by_size # Output local log mode + eol: true # Whether to wrap each output automatically. Default wrap + filename: /usr/local/trpc/bin/log_by_size.log # Log file name, including path (relative or absolute), default to trpc.log + roll_size: 10000 # Output single file size, default 100M + reserve_count: 5 # Output local log rolling file count, default 9 +``` + +##### Output mode by day +```yaml +plugins: + log: + default: + - name: custom # Can be the default default, can use other logger names + format: "[%Y-%m-%d %H:%M:%S.%e] [thread %t] [%l] [%@] %v" + sinks: + local_file: # Local log sink + roll_type: by_day # Output local log mode by day + eol: true # Whether to wrap each output automatically. Default wrap + filename: /usr/local/trpc/bin/log_by_day.log # Log file name, including path (relative or absolute), default to trpc.log + reserve_count: 5 # Output local log rolling file count, default 9 + rotation_hour: 0 # Represents the cut-off time by day, the specific time is specified by rotation_hour:rotation_minute + rotation_minute: 0 + remove_timout_file_switch: false # Under the day segmentation mode, the flag for deleting outdated files, default not to delete +``` + +##### Output mode by hour +```yaml +plugins: + log: + default: + - name: custom # Can be the default default, can use other logger names + format: "[%Y-%m-%d %H:%M:%S.%e] [thread %t] [%l] [%@] %v" + sinks: + local_file: # Local log sink + roll_type: by_hour # Output local log mode by hour + eol: true # Whether to wrap each output automatically. Default wrap + filename: /usr/local/trpc/bin/log_by_hour.log # Log file name, including path (relative or absolute), default to trpc.log + reserve_count: 5 # Output local log rolling file count, default 9 + hour_interval: 1 # Represents the hour segmentation interval, in hours, default interval is one hour +``` + + +#### Local Console Output +When the sink is set to "stdout", it means that the log is printed to the local console output. First, let's introduce its configuration items, as follows: +| Configuration item | Type | Required | Default value | Configuration explanation | +| :--- | :---: | :---: | :---: | :---: | +| format | string | Optional | Empty | Log output format, not filled in by default inherits logger | +| eol | bool | Optional | true | Whether to wrap when formatting output logs | +| stderr_level | int | Optional | 4 | Logs of this level and above are output to standard error output (stderr) | + +```yaml +plugins: + log: + default: + - name: custom # Can be the default default, can use other logger names + format: "[%Y-%m-%d %H:%M:%S.%e] [thread %t] [%l] [%@] %v" + sinks: + stdout: # Local console sink + eol: true # Whether to wrap each output automatically. Default wrap + stderr_level: 4 # The lowest level output to stderr, default is 4 (ERROR) +``` + +# Classic Scenario Usage Instructions +## Scenario 1: Separating business logs from framework logs, and customizing output for different business logs +Steps: +- Choose macro interface: Choose any of the specified logger log macros to print logs +- Add configuration: Complete logger and sink configuration. Logger configuration can specify format, mode, sink configuration can choose local, remote. + +## Scenario 6: How to initialize the log plugin in pure client mode and output the log to a file +Pure client: +In the pure client scenario, users need to output logs to a file and need to call the interface in their code to complete the initialization of the plugin. There are two ways here: +- Initialize all plugins: +``` +trpc::TrpcPlugin::GetInstance()->RegisterPlugins(); +``` +- Only need to initialize the log plugin +``` +trpc::TrpcPlugin::GetInstance()->InitLogging(); +``` +Note: The server-side framework will help complete the initialization of the log plugin when the service starts, so users don't need to worry about this. + +## Scenario 7: How to get the context when printing logs without context being transparently transmitted between business methods? +If no configuration is written or the log module has not been initialized, the logs will be output to the console: level < error will be output to std::cout, otherwise std::cerr. +To avoid excessive log output (such as trace, debug, etc.), by default, only logs with level >= info will be output. To support different user needs, users can define the TRPC_NOLOG_LEVEL macro to change the default behavior, which needs to be passed to the framework's log module. +For example, if you need to specify debug, you can try the following two ways: +- 1 Execute bazel test with the following option + > --copt=-DTRPC_NOLOG_LEVEL=1 +- 2 Write in the configuration file .bazelrc, so you don't have to manually specify it every time: 0-trace, 1-debug, 2-info, 3-warn, 4-error, 5-critical + 0-trace, 1-debug, 2-info, 3-warn, 4-error, 5-critical + > build --copt="-DTRPC_NOLOG_LEVEL=1" + +## Scenario 8: If the context is not transparently transmitted between business methods, how to get the context when printing logs? +The framework supports automatically setting the context to the thread's private variables (depending on the business configuration of the thread environment, the fiber environment is set to fiberLocal variables, and the ordinary thread environment is set to threadLocal variables), and provides two methods SetLocalServerContext() and GetLocalServerContext() for business to manually set or use context. The usage example is as follows: +``` +#include "trpc/server/server_context.h" +... + +::trpc::Status GreeterServiceImpl::SayHello(::trpc::ServerContextPtr context, + const ::trpc::test::helloworld::HelloRequest* request, + ::trpc::test::helloworld::HelloReply* reply) { + // If you use the future separation mode or are currently in a new thread/fiber manually started by the business, you need to set the context first + trpc::SetLocalServerContext(context); + // If you use fiber or future merge mode, the framework has set it up, no need to manually set context, you can use it directly + + // Do not pass context and enter other business methods + ExecBusiness(); + return ::trpc::kSuccStatus; +} + +void ExecBusiness() { + // Use context + trpc::ServerContextPtr ctx = trpc::GetLocalServerContext(); + TRPC_FMT_INFO("remote address: {}:{}", ctx->GetIp(), ctx->GetPort()); +} +``` diff --git a/docs/images/log_design.png b/docs/images/log_design.png new file mode 100644 index 00000000..de526be5 Binary files /dev/null and b/docs/images/log_design.png differ diff --git a/docs/zh/local_logging.md b/docs/zh/local_logging.md new file mode 100644 index 00000000..9355b6b6 --- /dev/null +++ b/docs/zh/local_logging.md @@ -0,0 +1,403 @@ +[TOC] +# 1 前言 +日志是软件开发中的一个重要组成部分，好的日志是我们分析软件运行状态的一个重要信息来源。 tRPC-Cpp框架为业务提供了一套的日志打印，收集和上报的解决方案。通过本文的介绍，旨在为用户提供以下信息： + +- 框架提供了哪些日志功能？ +- 框架如何实现这些功能的？ +- 如何使用标准日志接口？ +- 日志配置该如何配？ + +术语： +- 实例： 等价logger，即多实例就是多logger +- flow风格： 使用TRPC_FLOW_LOG,TRPC_FLOW_LOG_EX两种日志宏的统称 + +# 功能介绍 +## 功能概述 + +如图所示，tRPC-Cpp 日志模块底层是基于SpdLog实现，它包括两方面的实现： `日志打印编程接口` 和 `与日志输出端对接` 。在实现中引入了Logger 和Sink 两个概念 +![architecture design](docs/images/arch_design.png) + +Logger和Sink分别负责日志打印和于日志服务系统对接，它们都采用了插件化编程，可定制化开发 ，用户可以参考[如何开发自定义日志插件](./custom_logging.md)。Logger 和Sink 的定义如下： +- Logger用于负责实现日志打印相关的通用接口，Logger采用插件化编程，支持多Logger功能，用户可以自行选择logger来实现差异化日志打印和收集。通过Sink插件和日志后端进行对接。 +- Sink又称为“日志输出端”，用于负责日志的收集上报功能，包括日志上报格式和与后端日志系统对接等。每个Logger都可以有多个Sink， 例如：Default这个默认的logger, 它可以支持同时输出到本地日志和远程CLS。Sink采用插件化编程，可灵活扩展。 + +框架提供了4种风格的日志打印函数： +- python 风格：最优雅，方便，推荐。[详细语法](https://fmt.dev/latest/syntax.html) +- prinrf 风格: printf语法，类型安全。[详细语法](https://en.wikipedia.org/wiki/Printf_format_string) +- stream 风格: 兼容旧版本，框架打印日志的方式，使用较为繁琐，不推荐用户使用 +- flow 风格： 它的底层沿用的是 stream风格，但可以支持传入context和指定logger。 推荐用于打印业务日志时与框架日志分开的场景。 + +## 日志打印接口 +首先框架参考业界标准对日志打印级别按由低到高的顺序进行了如下划分： +| 级别 | 描述 | +| :--- | :---: | +| **trace** | 该级别日志的主要作用是对系统特定事件的每一步的运行状态进行精确的记录| +| **debug** | 指出细粒度信息事件对调试应用程序是非常有帮助的，主要用于开发过程中打印一些运行信息| +| **info**| 消息在粗粒度级别上突出强调应用程序的运行过程。打印一些你感兴趣的或者重要的信息，这个可以用于生产环境中输出程序运行的一些重要信息，但是不能滥用，避免打印过多的日志| +|**warn** |表明会出现潜在错误的情形，有些信息不是错误信息，但是也要给程序员一些提示 | +|**error** |指出虽然发生错误事件，但仍然不影响系统的继续运行。打印错误和异常信息，如果不想输出太多的日志，可以使用这个级别 | +| **critical**|指出每个严重的错误事件将会导致应用程序的退出。这个级别比较高了。重大错误，这种级别你可以直接停止程序了 | + +为照顾不同用户的使用习惯，框架为每个级别的日志函数提供了类似于 `python` 、 `printf` 、`stream` 、 `flow`四种风格的打印日志宏，此处举例只是让用户有个大概的了解，更详细的日志宏在下文介绍。 + +- 推荐使用：python 风格的日志宏，因为性能方面fmt::format要比std::ostream高20%左右。 + +``` +// python 风格 +TRPC_FMT_DEBUG("msg:{}", "test"); +// printf 风格 +TRPC_PRT_DEBUG("msg: %s", "test"); +// stream 风格 +TRPC_LOG_DEBUG("msg:" << "test"); +// flow 风格 +TRPC_FLOW_LOG_EX(ctx, "custom1", "msg:" << "test"); 可指定传入ctx和logger name +``` + +### 2.2.1 Default 日志宏接口 +- 宏接口说明：它使用了default logger, 且只有一个参数, 参数表示需要打印的日志内容。tRPC-Cpp框架日志都出于此。 +- 使用场景：框架日志, 调试日志 +```cpp + int i = 0; + // python-like + TRPC_FMT_TRACE("trpc-{}: {}", "cpp"); // trpc-cpp + TRPC_FMT_DEBUG("print floats {:03.2f}", 1.23456); // "print floats 001.23" + TRPC_FMT_INFO("hello trpc-{}: {}", "cpp", ++i); // "hello trpc-cpp: 2" + TRPC_FMT_ERROR("{} is smaller than {}?", 1, 2); // "1 is smaller than 2?" + TRPC_FMT_CRITICAL("coredump msg: {}?", "oom"); // "coredump msg: oom" + + // printf-like + TRPC_PRT_INFO("just like %s: %d", "printf", ++i); // "just like printf: 3" + ... + + // stream-like + TRPC_LOG_INFO("stream trace: " << ++i); // "stream trace: 1" + ... +``` +### 带Context的日志宏接口 +- 宏接口说明：它有2个参数， 第一个是context， 第二个为日志内容。需要注意的是，这里的context是ClientContext和ServerContext的基类, 所以在客户端和服务端都适用。 + +```cpp +TRPC_LOG_TRACE_EX(context, msg); +TRPC_LOG_DEBUG_EX(context, msg); +TRPC_LOG_INFO_EX(context, msg); +TRPC_LOG_WARN_EX(context, msg); +TRPC_LOG_ERROR_EX(context, msg); +TRPC_LOG_DEBUG_EX(context, msg); + +// 日志等级框架写死为info级别 +TRPC_FLOW_LOG_EX(context, instance, msg) +// 举例 +TRPC_FLOW_LOG_EX(ctx, "custom", "msg：" << "test") // ctx是上下文对象, custom是logger name, 输出：msg: test +``` +说明： +line 8: instance指的是logger的name, 用户可通过这个宏指定不同的logger, 并且也可获得context的信息， 接口更加灵活。 +line 9: 日志等级框架写死为info级别 ,用户如果想修改日志级别可以基于`TRPC_STREAM`来设置，也可自己用它来封一个日志宏，下面是TRPC_FLOW_LOG的实现 +``` +#define TRPC_FLOW_LOG(instance, msg) TRPC_STREAM(instance, trpc::Log::info, nullptr, msg) +#define TRPC_FLOW_LOG_EX(context, instance, msg) \ + TRPC_STREAM(instance, trpc::Log::info, context, msg) +``` + +### 2.2.3 指定logger的日志宏接口 +- 宏接口说明： 可以指定logger, 定制化日志输出方式。 +- 使用场景： +- 把业务日志和框架日志分离 +- 不同的业务日志指定不同的logger +- 业务日志输出到远程 + +上述场景任意一种，或者任意几种的组合都可以通过指定logger的日志宏接口实现。 +``` + // flow风格的宏 + TRPC_FLOW_LOG(instance, msg) + TRPC_FLOW_LOG_EX(context, instance, msg) + // 举例 + TRPC_FLOW_LOG("custom1", "hello" << "trpc") // hello trpc + TRPC_FLOW_LOG_EX("server_context", "custom2", "hello" << "trpc") // hello trpc + + // fmt风格定义宏接口 + TRPC_LOGGER_FMT_TRACE(instance, format, args...) + TRPC_LOGGER_FMT_DEBUG(instance, format, args...) + TRPC_LOGGER_FMT_INFO(instance, format, args...) + TRPC_LOGGER_FMT_WARN(instance, format, args...) + TRPC_LOGGER_FMT_ERROR(instance, format, args...) + TRPC_LOGGER_FMT_CRITICAL(instance, format, args...) + // 举例 + TRPC_LOGGER_FMT_WARN("custom3", "i am not {} logger", "default"); // "i am not default logger" +``` + +### 带IF条件的日志宏接口 +- 宏接口说明：带条件的日志宏，即当条件满足时，则输出日志，否则不输出 +- 使用场景 1：日志只有满足自定义的条件才输出 +``` + // fmt风格 + TRPC_FMT_TRACE_IF(condition, format, args...) + TRPC_FMT_DEBUG_IF(condition, format, args...) + TRPC_FMT_INFO_IF(condition, format, args...) + TRPC_FMT_WARN_IF(condition, format, args...) + TRPC_FMT_ERROR_IF(condition, format, args...) + TRPC_FMT_CRITICAL_IF(condition, format, args...) + // 举例 + TRPC_FMT_INFO_IF(true, "if true, print: {}", "msg"); // "if true, print: msg" + + // printf风格 + TRPC_PRT_TRACE_IF(condition, format, args...) + TRPC_PRT_DEBUG_IF(condition, format, args...) + TRPC_PRT_INFO_IF(condition, format, args...) + TRPC_PRT_WARN_IF(condition, format, args...) + TRPC_PRT_ERROR_IF(condition, format, args...) + TRPC_PRT_CRITICAL_IF(condition, format, args...) + // 举例 + TRPC_PRT_INFO_IF(true, "if true, print: %s", "msg"); // "if true, print: msg" + + // stream风格 + TRPC_LOG_TRACE_IF(condition, msg) + TRPC_LOG_DEBUG_IF(condition, msg) + TRPC_LOG_INFO_IF(condition, msg) + TRPC_LOG_WARN_IF(condition, msg) + TRPC_LOG_ERROR_IF(condition, msg) + TRPC_LOG_CRITICAL_IF(condition, msg) + // 举例 + TRPC_LOG_INFO_IF(true, "if true, print: " << "msg"); // "if true, print: msg" + + // 以上所有的宏接口也可以和LOGGER/Context配合使用，例如： + TRPC_LOGGER_FMT_INFO_IF(true, "if true, print: {}", "msg"); // "if true, print: msg" + TRPC_LOGGER_PRT_INFO_EX(context, true, "if true, print: %s", "msg"); // "if true, print: msg" + TRPC_LOGGER_LOG_INFO_IF_EX(context, true, "if true, print: " << "msg"); // "if true, print: msg" +``` +- 使用场景 2：用户希望输出日志但又不希望输出过多导致性能下降。 +这种场景下可使用 条件宏算子 +- TRPC_EVERY_N(c) ：设置本条日志每触发c次至多输出打印1次，第一次必定触发。 +- TRPC_WITHIN_N(ms)：设置为本条日志在设定的ms毫秒内若存在多次触发，则至多只输出1次，第一次必定触发。使用示例如下 +``` +TRPC_FMT_INFO_IF(TRPC_EVERY_N(1000), "condition every 1000"); # 每1000次调用打印一次日志 +TRPC_FMT_INFO_IF(TRPC_WITHIN_N(1000), "condition within 1000 ms"); # 每1000ms时间内若触发一次以上调用，则打印至多一次日志 +``` + +## 多logger支持 +tRPC-Cpp支持同时存在多个logger，每个logger设置不同的日志级别，打印格式，上报输出后端，业务可以为不同的业务场景设置不同的logger。比如 + +- 不同的业务模块使用不同的日志文件存储 +- 不同的事件，基于事件的关注度不同，采用不同的日志级别收集日志 + +多Logger功能大大增加了日志使用的灵活性。 + +## 多sink支持 +对于同一个logger，框架也提供了日志同时输出到多个日志输出后端（简称 “sink”）的能力。 因为sink也是以插件的方式注册到框架的，所以用户可选本地，远程（比如 CLS，智研日志等），或者一个都不选。目前框架支持4种输出端（Sink）： +- 本地滚动文件（LocalFileSink）：配置名字为 `local_file` +- 本地控制台输出（StdoutSink）：配置名字为 `stdout` + 上述sink分属2种不同的类型： +- DefaultSink： + - 上述 LocalFileSink、StdoutSink 即是。 + - 配置写在 sinks 字段下面。 + - 要求返回一个 spdsink。 + - 归属于一个实例的所有 DefaultSink，由于都挂接在 DefaultLog 内置的spdlogger中，于是接收的消息是一样的 +- RawSink + - 上述 CLSLogSink 即是。 + - 配置写在 raw_sinks下面。 + - 要求实现一个 log 接口，额外接收 context 参数，由 DefaultLog 同步调用，其线程安全性也需要其自己保证。 + - 其行为完全由其自己定义，目前 CLSLogSink 会在内部新建一个 spdlogger 和一个 spdsink，然后根据 context 决定如何输出。 + +这里需要强调的是：日志的打印设置是以Sink粒度来配置的。用户需要为每个输出端做配置，比如：同一份日志即需要保存在本地日志文件，又要输出到cls， 那就必须要分别配置local_file和cls_log这两个sink。 + +# 日志配置 +## 配置结构 +首先我们来看看日志配置的总体结构。 由于日志模块对于logger和sink的实现都是采用插件的方式来实现的，所以日志配置都必须要放到“plugins”区域内。 +```yaml +plugins: + log: + default: + - name: default # 默认logger名 + ... + sinks: # DefaultSink类型插件 +local_file: # 本地日志的sink + ... + raw_sinks: # RawSink类型插件 + cls_log: # cls日志的sink + - name: custom1 # 自定义logger名字 + ... + sinks: + local_file: + ... + raw_sinks: + cls_log: +``` +说明： +- 上面的配置体现了多logger和多sink功能。“default”，“custom1”表示logger的名称。每个logger的配置是一个sink的数组。 +- line 4: defaul表示框架默认logger +- line 6: sinks表示配置中放的是DefaultSink类型的插件， 比如local_file +- line 9: raw_sink表示配置中放的是RawtSink类型的插件， 比如cls_log + +## logger配置 +logger的配置通常是方便其管理的sink直接继承, 因此sink中有些字段可以不做配置，比如format，这样可以简化配置。先介绍下它的参数配置，设计如下。 +|配置项 | 类型 | 是否必填 | 默认值 |配置解释| +| :--- | :---: | :---: | :---: | :---: | +| name | string | 可选 | 空 | logger名 | +| min_level | int | 可选 | 2 | 设置的最小日志级别, 只有日志级别大于它时，日志才会输出 | +| format | string | 可选 | 空 | [%Y-%m-%d %H:%M:%S.%e] [thread %t] [%l] [%@] %v | +| mode | int | 可选 | 2 | 可选：1 同步，2 异步 3 极速 | + +说明： +format： 参考 [格式说明](https://github.com/gabime/spdlog/wiki/3.-Custom-formatting) +mode: +- 异步：打印日志时，把日志存入一个线程安全的缓存队列后直接返回。内部有独立线程 消费此队列，输出到各个输出端。如果队列满，则输出阻塞，如果队列空，则消费阻塞。**推荐使用方式**。 +- 同步：打印日志时，直接遍历各个输出端并等待完成, 阻塞方式; + +- 极速： 和异步模式基本相同，唯一区别在于输出从不阻塞，一旦队列满，则直接丢弃最旧的日志，然后存入当前日志并返回 + +``` +plugins: + log: + default: + - name: default + min_level: 1 # 0-trace, 1-debug, 2-info, 3-warn, 4-error, 5-critical + format: "[%Y-%m-%d %H:%M:%S.%e] [thread %t] [%l] [%@] [%!] %v" + mode: 2 # 1-同步, 2-异步, 3-极速 + sinks: + ... +``` + +## sink配置 +### 使用本地日志 +#### 本地文件日志 +当sink设置为“local_file”时，代表日志打本地日志文件。先介绍下它的配置项如下。 +|配置项 | 类型 | 是否必填 | 默认值 |配置解释| +| :--- | :---: | :---: | :---: | :---: | +| format | string | 可选 | 空 | 日志输出格式，不填则默认继承logger的 | +| eol | bool | 可选 | true | 格式化输出日志时是否换行 | +| filename | string | 必填 | 空 | 本地日志文件名，默认为当前路径，可带路径 | +| roll_type | string | 可选 | by_size | 设置日志分割方式，默认为by_size, 可选by_day-按天分割， by_hour-按小时分割| +| reserve_count | int | 可选 | 10 | 滚动日志保留的文件个数 | +| roll_size | int | 可选 | 100 x 1024 x 1024 | by_size类型才设置此变量, 表示一个滚动文件的大小 | +| rotation_hour | int | 可选 | 0 | by_day类型才设置此变量，表示按天切割的时刻，具体是时刻通过rotation_hour:rotation_minute指定| +| rotation_minute | int | 可选 | 0 | | +| remove_timout_file_switch | bool | 可选 | false | by_hour类型才设置此变量，按天分割模式下, 删除过时文件的标识 | +| hour_interval | int | 可选 | 1 | by_hour类型才设置此变量， 表示小时分割的间隔，单位为小时| + +本地日志文件输出支持三种输出模式： +- 按大小：by_size +- 按天：by_day +- 按小时：by_hour + +下面分别介绍如何配置 +##### 按大小输出模式 +```yaml +plugins: + log: + default: + - name: custom # 可以默认的default,可以使用其他的logger名 + format: "[%Y-%m-%d %H:%M:%S.%e] [thread %t] [%l] [%@] %v" + sinks: + local_file: # 本地日志的sink + roll_type: by_size # 输出本地日志模式 + eol: true # 每次输出是否自动换行。默认换行 + filename: /usr/local/trpc/bin/log_by_size.log # 日志文件名，包含路径（相对或绝对路径), 默认为trpc.log + roll_size: 10000 # 输出单个文件大小,默认100M + reserve_count: 5 # 输出本地日志滚动文件个数,默认9个 +``` +##### 按天输出模式 +```yaml +plugins: + log: + default: + - name: custom # 可以默认的default,可以使用其他的logger名 + format: "[%Y-%m-%d %H:%M:%S.%e] [thread %t] [%l] [%@] %v" + sinks: + local_file: # 本地日志的sink + roll_type: by_day # 按天输出本地日志模式 + eol: true # 每次输出是否自动换行。默认换行 + filename: /usr/local/trpc/bin/log_by_day.log # 日志文件名，包含路径（相对或绝对路径), 默认为trpc.log + reserve_count: 5 # 输出本地日志滚动文件个数,默认9个 + rotation_hour: 0 # 表示按天切割的时刻，具体是时刻通过rotation_hour:rotation_minute指定 + rotation_minute: 0 + remove_timout_file_switch: false # 按天分割模式下,删除过时文件的标识, 默认不删除 +``` + +##### 按小时输出模式 +```yaml +plugins: + log: + default: + - name: custom # 可以默认的default,可以使用其他的logger名 + format: "[%Y-%m-%d %H:%M:%S.%e] [thread %t] [%l] [%@] %v" + sinks: + local_file: # 本地日志的sink + roll_type: by_hour # 按小时输出本地日志模式 + eol: true # 每次输出是否自动换行。默认换行 + filename: /usr/local/trpc/bin/log_by_hour.log # 日志文件名，包含路径（相对或绝对路径), 默认为trpc.log + reserve_count: 5 # 输出本地日志滚动文件个数,默认9个 + hour_interval: 1 # 表示小时分割的间隔，单位为小时，默认间隔是一小时 +``` + +#### 本地控制台输出 +当sink设置为"stdout"时，代表日志打本地控制台输出。先介绍下它的[参数配置](https://git.woa.com/trpc-cpp/trpc-cpp/blob/master/trpc/common/config/stdout_sink_conf.h)，设计如下。 +|配置项 | 类型 | 是否必填 | 默认值 |配置解释| +| :--- | :---: | :---: | :---: | :---: | +| format | string | 可选 | 空 | 日志输出格式，不填则默认继承logger的 | +| eol | bool | 可选 | true | 格式化输出日志时是否换行 | +| stderr_level | int | 可选 | 4 | 该级别及以上的日志输出到标准错误输出（stderr） | + +```yaml +plugins: + log: + default: + - name: custom # 可以默认的default,可以使用其他的logger名 + format: "[%Y-%m-%d %H:%M:%S.%e] [thread %t] [%l] [%@] %v" + sinks: + stdout: # 本地控制台的sink + eol: true # 每次输出是否自动换行。默认换行 + stderr_level: 4 # 输出到stderr的最低等级，默认为4（ERROR） +``` + +# 经典场景使用说明 +## 场景1： 业务日志与框架日志分离，不同的业务日志需定制化输出 +步骤： +- 选择宏接口：选择 `指定logger的日志宏`中的任意一种来打印日志 +- 添加配置：完成logger，sink的配置。其中logger配置可以指定format, mode, sink配置可以选择本地，远程。 + +## 场景6：纯客户端模式下如何把初始化日志插件并把日志输出到文件中 +纯客户端： +用户在纯客户端的场景下需要把日志输出到文件，需要自己在代码中调用接口完成插件的初始化。 这里也分为两种方式： +- 初始化全部的插件: +``` +trpc::TrpcPlugin::GetInstance()->RegisterPlugins(); +``` +- 只需要初始化日志插件 +``` +trpc::TrpcPlugin::GetInstance()->InitLogging(); +``` +说明：服务端在服务启动的时候框架会帮完成日志插件的初始化，所以用户不用管这块。 + +## 场景7：单测或者客户端没有配置日志插件，如何把trpc的log日志输出到终端？ +如果不写配置，或者日志模块还未初始化，那么日志将输出到控制台：`level < error` 将会输出到 std::cout，否则 std::cerr。 +为了避免过多的日志输出（如trace、debug等），默认情况下，只有 `level >= info` 的日志才会输出。为了支持不同用户的需求，用户可以定义宏`TRPC_NOLOG_LEVEL`改变默认行为，这个宏需要传递给框架的日志模块。 +例如，你需要指定为 debug，有如下两种方式可以试下 +- 1 执行bazel test时加上 下面这个选项 + > --copt=-DTRPC_NOLOG_LEVEL=1 +- 2 “在写在配置文件 .bazelrc 中，这样就不用每次手动指定了： + 0-trace, 1-debug, 2-info, 3-warn, 4-error, 5-critical + > build --copt="-DTRPC_NOLOG_LEVEL=1" + +## 场景8：context没有在业务方法之间透传，如何在打印日志时获取到context？ +框架支持自动把context设置到线程的私有变量中（根据业务配置线程环境的不同，fiber环境设置到fiberLocal变量中，普通线程环境设置到threadLocal变量中），并提供了SetLocalServerContext()和GetLocalServerContext()两个方法供业务手动设置或取用context，使用示例如下： +``` +#include "trpc/server/server_context.h" +... + +::trpc::Status GreeterServiceImpl::SayHello(::trpc::ServerContextPtr context, + const ::trpc::test::helloworld::HelloRequest* request, + ::trpc::test::helloworld::HelloReply* reply) { + // 如果采用future分离模式或者当前处于业务手动起的新线程/fiber里，需要先设置一次context + trpc::SetLocalServerContext(context); + // 如果采用fiber或者future合并模式，框架已经设置好，无需手动设置context，可以直接取用 + + // 不透传context并进入到其他业务方法 + ExecBusiness(); + return ::trpc::kSuccStatus; +} + +void ExecBusiness() { + // 取用context + trpc::ServerContextPtr ctx = trpc::GetLocalServerContext(); + TRPC_FMT_INFO("remote address: {}:{}", ctx->GetIp(), ctx->GetPort()); +} +``` diff --git a/trpc/util/log/default/default_log.cc b/trpc/util/log/default/default_log.cc index cd06f1e9..fa5fe4f5 100644 --- a/trpc/util/log/default/default_log.cc +++ b/trpc/util/log/default/default_log.cc @@ -53,6 +53,20 @@ bool DefaultLog::ShouldLog(const char* instance_name, Level level) const { return level >= instance.config.min_level; } +bool DefaultLog::ShouldLog(Level level) const { + if (!inited_) { + std::cerr << "DefaultLog not inited" << std::endl; + return false; + } + if (!inited_trpc_logger_instance_) { + std::cout<<"not inited!"<= trpc_logger_instance_.config.min_level; +} + void DefaultLog::LogIt(const char* instance_name, Level level, const char* filename_in, int line_in, const char* funcname_in, std::string_view msg, const std::unordered_map& filter_data) const { @@ -60,19 +74,30 @@ void DefaultLog::LogIt(const char* instance_name, Level level, const char* filen std::cerr << "DefaultLog not inited" << std::endl; return; } - auto iter = instances_.find(instance_name); - if (iter == instances_.end()) { - std::cerr << "DefaultLog instance" << instance_name << " does not exit" << std::endl; - return; + + const DefaultLog::Logger* instance = nullptr; + // It is preferred if it is the output of the tRPC-Cpp framework log + if (!strcmp(instance_name, kTrpcLogCacheStringDefault)) { + if (inited_trpc_logger_instance_ == false) { + std::cerr << "DefaultLog instance:" << kTrpcLogCacheStringDefault << "does not exit" << std::endl; + return ; + } + instance = &trpc_logger_instance_; + } else { + auto iter = instances_.find(instance_name); + if (iter == instances_.end()) { + std::cerr << "DefaultLog instance: " << instance_name << " does not exit" << std::endl; + return; + } + instance = &iter->second; } - const auto& instance = iter->second; - if (instance.logger) { - instance.logger->log(spdlog::source_loc{filename_in, line_in, funcname_in}, SpdLevel(level), msg); + if (instance->logger) { + instance->logger->log(spdlog::source_loc{filename_in, line_in, funcname_in}, SpdLevel(level), msg); } // Output to a remote plugin (if available) - for (const auto& sink : instance.raw_sinks) { + for (const auto& sink : instance->raw_sinks) { sink->Log(level, filename_in, line_in, funcname_in, msg, filter_data); } } @@ -108,6 +133,9 @@ std::pair DefaultLog::SetLevel(const char* instance_name, Leve auto old = static_cast(instance.config.min_level); instance.config.min_level = static_cast(level); + if (!strcmp(instance_name, kTrpcLogCacheStringDefault)) { + trpc_logger_instance_.config.min_level = static_cast(level); + } return std::make_pair(old, true); } diff --git a/trpc/util/log/default/default_log.h b/trpc/util/log/default/default_log.h index 07fb0542..2be9b3d5 100644 --- a/trpc/util/log/default/default_log.h +++ b/trpc/util/log/default/default_log.h @@ -76,6 +76,11 @@ class DefaultLog : public Log { /// @return true/false bool ShouldLog(const char* instance_name, Level level) const override; + /// @brief Determine whether the log level of the tRPC-Cpp framework instance meets the requirements for printing this log. + /// @param level Log instance level + /// @return true/false + bool ShouldLog(Level level) const override; + /// @brief Output log to a sink instance. void LogIt(const char* instance_name, Level level, const char* filename_in, int line_in, const char* funcname_in, std::string_view msg, const std::unordered_map& filter_data = {}) const override; @@ -108,6 +113,11 @@ class DefaultLog : public Log { // Add the new sink to the logger's sinks instance.logger->sinks().push_back(sink->SpdSink()); + // The tRPC-Cpp framework logging instance has been configured + if (!strcmp(logger_name, "default")) { + inited_trpc_logger_instance_ = true; + trpc_logger_instance_ = instance; + } return true; } @@ -148,6 +158,13 @@ class DefaultLog : public Log { // Initialization flags bool inited_{false}; + // Whether the tRPC-Cpp framework logging instance is configured + // If false, all framework logging will be logged to the console + bool inited_trpc_logger_instance_{false}; + + // tRPC-Cpp framework logger instance + DefaultLog::Logger trpc_logger_instance_; + // Collection of log instances std::unordered_map instances_; }; diff --git a/trpc/util/log/log.h b/trpc/util/log/log.h index 865a865a..f0e05c37 100644 --- a/trpc/util/log/log.h +++ b/trpc/util/log/log.h @@ -83,6 +83,11 @@ class Log : public RefCounted { /// @private For internal use purpose only. virtual bool ShouldLog(const char* instance_name, Level level) const = 0; + /// @brief Determine whether the log level of the tRPC-Cpp framework instance meets the requirements for printing this log. + /// @param level Log instance level + /// @return true/false + virtual bool ShouldLog(Level level) const = 0; + /// @brief Output log to a sink instance. /// @param instance_name Log instance name /// @param level Log instance level diff --git a/trpc/util/log/logging.h b/trpc/util/log/logging.h index c8d3ec7d..0204a337 100644 --- a/trpc/util/log/logging.h +++ b/trpc/util/log/logging.h @@ -153,7 +153,7 @@ constexpr char kTrpcLogCacheStringDefault[] = "default"; #define TRPC_LOGGER_PRT_ERROR(instance, format, args...) TRPC_PRT(instance, ::trpc::Log::error, format, ##args) #define TRPC_LOGGER_PRT_CRITICAL(instance, format, args...) TRPC_PRT(instance, ::trpc::Log::critical, format, ##args) -/// @brief printf-like style log macros with conditions. +/// @brief printf-like style log macros with conditions for tRPC-Cpp framework #define TRPC_LOGGER_PRT_TRACE_IF(instance, condition, format, args...) \ TRPC_PRT_IF(instance, condition, ::trpc::Log::trace, format, ##args) #define TRPC_LOGGER_PRT_DEBUG_IF(instance, condition, format, args...) \ @@ -167,6 +167,7 @@ constexpr char kTrpcLogCacheStringDefault[] = "default"; #define TRPC_LOGGER_PRT_CRITICAL_IF(instance, condition, format, args...) \ TRPC_PRT_IF(instance, condition, ::trpc::Log::critical, format, ##args) +/// @brief printf-like style log macros with conditions #define TRPC_LOGGER_PRT_TRACE_EX(context, instance, format, args...) \ TRPC_PRT_EX(context, instance, ::trpc::Log::trace, format, ##args) #define TRPC_LOGGER_PRT_DEBUG_EX(context, instance, format, args...) \ @@ -180,6 +181,7 @@ constexpr char kTrpcLogCacheStringDefault[] = "default"; #define TRPC_LOGGER_PRT_CRITICAL_EX(context, instance, format, args...) \ TRPC_PRT_EX(context, instance, ::trpc::Log::critical, format, ##args) +/// @brief printf-like style log macros with conditions and context #define TRPC_LOGGER_PRT_TRACE_IF_EX(context, instance, condition, format, args...) \ TRPC_PRT_IF_EX(context, instance, condition, ::trpc::Log::trace, format, ##args) #define TRPC_LOGGER_PRT_DEBUG_IF_EX(context, instance, condition, format, args...) \ @@ -194,29 +196,29 @@ constexpr char kTrpcLogCacheStringDefault[] = "default"; TRPC_PRT_IF_EX(context, instance, condition, ::trpc::Log::critical, format, ##args) /// @brief Log will be output to the "default" logger instance provided by the framework "default" plugin. -#define TRPC_PRT_TRACE(format, args...) TRPC_LOGGER_PRT_TRACE(::trpc::log::kTrpcLogCacheStringDefault, format, ##args) -#define TRPC_PRT_DEBUG(format, args...) TRPC_LOGGER_PRT_DEBUG(::trpc::log::kTrpcLogCacheStringDefault, format, ##args) -#define TRPC_PRT_INFO(format, args...) TRPC_LOGGER_PRT_INFO(::trpc::log::kTrpcLogCacheStringDefault, format, ##args) -#define TRPC_PRT_WARN(format, args...) TRPC_LOGGER_PRT_WARN(::trpc::log::kTrpcLogCacheStringDefault, format, ##args) -#define TRPC_PRT_ERROR(format, args...) TRPC_LOGGER_PRT_ERROR(::trpc::log::kTrpcLogCacheStringDefault, format, ##args) +#define TRPC_PRT_TRACE(format, args...) TRPC_PRT_DEFAULT(::trpc::log::kTrpcLogCacheStringDefault, ::trpc::Log::trace, format, ##args) +#define TRPC_PRT_DEBUG(format, args...) TRPC_PRT_DEFAULT(::trpc::log::kTrpcLogCacheStringDefault, ::trpc::Log::debug, format, ##args) +#define TRPC_PRT_INFO(format, args...) TRPC_PRT_DEFAULT(::trpc::log::kTrpcLogCacheStringDefault, ::trpc::Log::info, format, ##args) +#define TRPC_PRT_WARN(format, args...) TRPC_PRT_DEFAULT(::trpc::log::kTrpcLogCacheStringDefault, ::trpc::Log::warn, format, ##args) +#define TRPC_PRT_ERROR(format, args...) TRPC_PRT_DEFAULT(::trpc::log::kTrpcLogCacheStringDefault, ::trpc::Log::error, format, ##args) #define TRPC_PRT_CRITICAL(format, args...) \ - TRPC_LOGGER_PRT_CRITICAL(::trpc::log::kTrpcLogCacheStringDefault, format, ##args) + TRPC_PRT_DEFAULT(::trpc::log::kTrpcLogCacheStringDefault, ::trpc::Log::critical, format, ##args) /// @brief Log macro for the framework default log with conditions. #define TRPC_PRT_TRACE_IF(condition, format, args...) \ - TRPC_LOGGER_PRT_TRACE_IF(::trpc::log::kTrpcLogCacheStringDefault, condition, format, ##args) + TRPC_PRT_IF_DEFAULT(::trpc::log::kTrpcLogCacheStringDefault, condition, ::trpc::Log::trace, format, ##args) #define TRPC_PRT_DEBUG_IF(condition, format, args...) \ - TRPC_LOGGER_PRT_DEBUG_IF(::trpc::log::kTrpcLogCacheStringDefault, condition, format, ##args) + TRPC_PRT_IF_DEFAULT(::trpc::log::kTrpcLogCacheStringDefault, condition, ::trpc::Log::debug, format, ##args) #define TRPC_PRT_INFO_IF(condition, format, args...) \ - TRPC_LOGGER_PRT_INFO_IF(::trpc::log::kTrpcLogCacheStringDefault, condition, format, ##args) + TRPC_PRT_IF_DEFAULT(::trpc::log::kTrpcLogCacheStringDefault, condition, ::trpc::Log::info, format, ##args) #define TRPC_PRT_WARN_IF(condition, format, args...) \ - TRPC_LOGGER_PRT_WARN_IF(::trpc::log::kTrpcLogCacheStringDefault, condition, format, ##args) + TRPC_PRT_IF_DEFAULT(::trpc::log::kTrpcLogCacheStringDefault, condition, ::trpc::Log::warn, format, ##args) #define TRPC_PRT_ERROR_IF(condition, format, args...) \ - TRPC_LOGGER_PRT_ERROR_IF(::trpc::log::kTrpcLogCacheStringDefault, condition, format, ##args) + TRPC_PRT_IF_DEFAULT(::trpc::log::kTrpcLogCacheStringDefault, condition, ::trpc::Log::error, format, ##args) #define TRPC_PRT_CRITICAL_IF(condition, format, args...) \ - TRPC_LOGGER_PRT_CRITICAL_IF(::trpc::log::kTrpcLogCacheStringDefault, condition, format, ##args) + TRPC_PRT_IF_DEFAULT(::trpc::log::kTrpcLogCacheStringDefault, condition, ::trpc::Log::critical, format, ##args) -/// @brief stream-like log macros +/// @brief stream-like log macros for tRPC-Cpp framework #define TRPC_LOG_TRACE(msg) TRPC_LOG_MSG(::trpc::Log::trace, msg) #define TRPC_LOG_DEBUG(msg) TRPC_LOG_MSG(::trpc::Log::debug, msg) #define TRPC_LOG_INFO(msg) TRPC_LOG_MSG(::trpc::Log::info, msg) @@ -224,6 +226,14 @@ constexpr char kTrpcLogCacheStringDefault[] = "default"; #define TRPC_LOG_ERROR(msg) TRPC_LOG_MSG(::trpc::Log::error, msg) #define TRPC_LOG_CRITICAL(msg) TRPC_LOG_MSG(::trpc::Log::critical, msg) +/// @brief stream-like log macros +#define TRPC_LOGGER_TRACE(instance, msg) TRPC_LOGGER_MSG(::trpc::Log::trace, instance, msg) +#define TRPC_LOGGER_DEBUG(instance, msg) TRPC_LOGGER_MSG(::trpc::Log::debug, instance, msg) +#define TRPC_LOGGER_INFO(instance, msg) TRPC_LOGGER_MSG(::trpc::Log::info, instance, msg) +#define TRPC_LOGGER_WARN(instance, msg) TRPC_LOGGER_MSG(::trpc::Log::warn, instance, msg) +#define TRPC_LOGGER_ERROR(instance, msg) TRPC_LOGGER_MSG(::trpc::Log::error, instance, msg) +#define TRPC_LOGGER_CRITICAL(instance, msg) TRPC_LOGGER_MSG(::trpc::Log::critical, instance, msg) + /// @brief stream-like style log macros with conditions #define TRPC_LOG_TRACE_IF(condition, msg) TRPC_LOG_MSG_IF(::trpc::Log::trace, condition, msg) #define TRPC_LOG_DEBUG_IF(condition, msg) TRPC_LOG_MSG_IF(::trpc::Log::debug, condition, msg) @@ -232,6 +242,13 @@ constexpr char kTrpcLogCacheStringDefault[] = "default"; #define TRPC_LOG_ERROR_IF(condition, msg) TRPC_LOG_MSG_IF(::trpc::Log::error, condition, msg) #define TRPC_LOG_CRITICAL_IF(condition, msg) TRPC_LOG_MSG_IF(::trpc::Log::critical, condition, msg) +#define TRPC_LOGGER_TRACE_IF(instance, condition, msg) TRPC_LOG_MSG_IF(::trpc::Log::trace, instance, condition, msg) +#define TRPC_LOGGER_DEBUG_IF(instance, condition, msg) TRPC_LOG_MSG_IF(::trpc::Log::debug, instance, condition, msg) +#define TRPC_LOGGER_INFO_IF(instance, condition, msg) TRPC_LOG_MSG_IF(::trpc::Log::info, instance, condition, msg) +#define TRPC_LOGGER_WARN_IF(instance, condition, msg) TRPC_LOG_MSG_IF(::trpc::Log::warn, instance, condition, msg) +#define TRPC_LOGGER_ERROR_IF(instance, condition, msg) TRPC_LOG_MSG_IF(::trpc::Log::error, instance, condition, msg) +#define TRPC_LOGGER_CRITICAL_IF(instance, condition, msg) TRPC_LOG_MSG_IF(::trpc::Log::critical, instance, condition, msg) + /// @brief logger instances can be specified to customize the log output. /// @note Use case: Separate business logs from framework logs, /// Different business logs specify different loggers. diff --git a/trpc/util/log/printf_like.h b/trpc/util/log/printf_like.h index e0d7bafc..8e16e38a 100644 --- a/trpc/util/log/printf_like.h +++ b/trpc/util/log/printf_like.h @@ -17,10 +17,34 @@ #include "fmt/printf.h" -//#include "trpc/client/client_context.h" -//#include "trpc/server/server_context.h" #include "trpc/util/log/log.h" +/// @brief printf-like log macros for tRPC-Cpp framework log +#define TRPC_PRT_DEFAULT(instance, level, formats, args...) \ + do { \ + const auto& __TRPC_PRINTF_LIKE_INSTANCE__ = ::trpc::LogFactory::GetInstance()->Get(); \ + if (__TRPC_PRINTF_LIKE_INSTANCE__) { \ + if (__TRPC_PRINTF_LIKE_INSTANCE__->ShouldLog(level)) { \ + TRPC_LOG_TRY { \ + std::string __TRPC_PRINTF_LIKE_MSG__ = ::trpc::Log::LogSprintf(formats, ##args); \ + __TRPC_PRINTF_LIKE_INSTANCE__->LogIt(instance, level, __FILE__, __LINE__, __FUNCTION__, \ + __TRPC_PRINTF_LIKE_MSG__); \ + } \ + TRPC_LOG_CATCH(instance) \ + } \ + } else { \ + if (::trpc::Log::ShouldNoLog(instance, level)) { \ + TRPC_LOG_TRY { \ + std::string __TRPC_PRINTF_LIKE_MSG__ = ::trpc::Log::LogSprintf(formats, ##args); \ + ::trpc::Log::NoLog(instance, level, __FILE__, __LINE__, __FUNCTION__, \ + std::string_view(__TRPC_PRINTF_LIKE_MSG__.data(), __TRPC_PRINTF_LIKE_MSG__.size())); \ + } \ + TRPC_LOG_CATCH(instance) \ + } \ + } \ + } while (0) + + /// @brief printf-like log macros #define TRPC_PRT(instance, level, formats, args...) \ do { \ @@ -46,9 +70,14 @@ } \ } while (0) -#define TRPC_PRT_IF(instance, condition, level, formats, args...) \ - if (condition) { \ - TRPC_PRT(instance, level, formats, ##args); \ +#define TRPC_PRT_IF(instance, condition, level, formats, args...) \ + if (condition) { \ + TRPC_PRT_DEFAULT(instance, level, formats, ##args); \ + } + +#define TRPC_PRT_IF_DEFAULT(instance, condition, level, formats, args...) \ + if (condition) { \ + TRPC_PRT(instance, level, formats, ##args); \ } #define TRPC_PRT_EX(context, instance, level, formats, args...) \ diff --git a/trpc/util/log/python_like.h b/trpc/util/log/python_like.h index 43f7185a..698556ac 100644 --- a/trpc/util/log/python_like.h +++ b/trpc/util/log/python_like.h @@ -21,6 +21,30 @@ #include "trpc/util/log/log.h" +/// @brief python-like style log macros for tRPC-Cpp framework log +#define TRPC_FMT_DEFAULT(instance, level, formats, args...) \ + do { \ + const auto& __TRPC_PYTHON_LIKE_INSTANCE__ = ::trpc::LogFactory::GetInstance()->Get(); \ + if (__TRPC_PYTHON_LIKE_INSTANCE__) { \ + if (__TRPC_PYTHON_LIKE_INSTANCE__->ShouldLog(level)) { \ + TRPC_LOG_TRY { \ + __TRPC_PYTHON_LIKE_INSTANCE__->LogIt(instance, level, __FILE__, __LINE__, __FUNCTION__, \ + ::trpc::Log::LogFormat(formats, ##args)); \ + } \ + TRPC_LOG_CATCH(instance) \ + } \ + } else { \ + if (::trpc::Log::ShouldNoLog(instance, level)) { \ + TRPC_LOG_TRY { \ + ::trpc::Log::NoLog(instance, level, __FILE__, __LINE__, __FUNCTION__, \ + ::trpc::Log::LogFormat(formats, ##args)); \ + } \ + TRPC_LOG_CATCH(instance) \ + } \ + } \ + } while (0) + + /// @brief python-like style log macros #define TRPC_FMT(instance, level, formats, args...) \ do { \ diff --git a/trpc/util/log/stream_like.h b/trpc/util/log/stream_like.h index e6ace3b3..acad9810 100644 --- a/trpc/util/log/stream_like.h +++ b/trpc/util/log/stream_like.h @@ -47,6 +47,56 @@ } \ } while (0) + +/// @brief stream-like log macros for tRPC-Cpp framework log +#define TRPC_STREAM_DEFAULT(instance, level, msg) \ + do { \ + const auto& __TRPC_CPP_STREAM_LOGGER_INSTANCE__ = ::trpc::LogFactory::GetInstance()->Get(); \ + if (__TRPC_CPP_STREAM_LOGGER_INSTANCE__) { \ + if (__TRPC_CPP_STREAM_LOGGER_INSTANCE__->ShouldLog(level)) { \ + TRPC_LOG_TRY { \ + STREAM_APPENDER(msg); \ + __TRPC_CPP_STREAM_LOGGER_INSTANCE__->LogIt(instance, level, __FILE__, __LINE__, __FUNCTION__, \ + __TRPC_STREAM__.str()); \ + } \ + TRPC_LOG_CATCH(instance) \ + } \ + } else { \ + if (::trpc::Log::ShouldNoLog(instance, level)) { \ + TRPC_LOG_TRY { \ + STREAM_APPENDER(msg); \ + ::trpc::Log::NoLog(instance, level, __FILE__, __LINE__, __FUNCTION__, __TRPC_STREAM__.str()); \ + } \ + TRPC_LOG_CATCH(instance) \ + } \ + } \ + } while (0) + +/// @brief stream-like log macros for tRPC-Cpp framework +#define TRPC_STREAM_EX_DEFAULT(instance, level, context, msg) \ + do { \ + const auto& __TRPC_CPP_STREAM_LOGGER_INSTANCE__ = ::trpc::LogFactory::GetInstance()->Get(); \ + if (__TRPC_CPP_STREAM_LOGGER_INSTANCE__) { \ + if (__TRPC_CPP_STREAM_LOGGER_INSTANCE__->ShouldLog(level)) { \ + TRPC_LOG_TRY { \ + STREAM_APPENDER(msg); \ + __TRPC_CPP_STREAM_LOGGER_INSTANCE__->LogIt(instance, level, __FILE__, __LINE__, __FUNCTION__, \ + __TRPC_STREAM__.str(), (context)->GetAllFilterData()); \ + } \ + TRPC_LOG_CATCH(instance) \ + } \ + } else { \ + if (::trpc::Log::ShouldNoLog(instance, level)) { \ + TRPC_LOG_TRY { \ + STREAM_APPENDER(msg); \ + ::trpc::Log::NoLog(instance, level, __FILE__, __LINE__, __FUNCTION__, __TRPC_STREAM__.str()); \ + } \ + TRPC_LOG_CATCH(instance) \ + } \ + } \ + } while (0) + + /// @brief stream-like log macros #define TRPC_STREAM_EX(instance, level, context, msg) \ do { \ @@ -71,25 +121,31 @@ } \ } while (0) -#define TRPC_LOG_MSG(level, msg) TRPC_STREAM(::trpc::log::kTrpcLogCacheStringDefault, level, msg) +#define TRPC_LOG_MSG(level, msg) TRPC_STREAM_DEFAULT(::trpc::log::kTrpcLogCacheStringDefault, level, msg) + +#define TRPC_LOGGER_MSG(level, instance, msg) TRPC_STREAM(instance, level, msg) + +#define TRPC_LOG_MSG_IF(level, condition, msg) \ + if (condition) { \ + TRPC_STREAM_DEFAULT(::trpc::log::kTrpcLogCacheStringDefault, level, msg); \ + } -#define TRPC_LOG_MSG_IF(level, condition, msg) \ - if (condition) { \ - TRPC_LOG_MSG(level, msg); \ +#define TRPC_LOGGER_MSG_IF(level, condition, instance, msg) \ + if (condition) { \ + TRPC_STREAM(instance, level, msg); \ } /// @brief uses default logger for logging with context -#define TRPC_LOG_MSG_EX(level, context, msg) \ - TRPC_STREAM_EX(::trpc::log::kTrpcLogCacheStringDefault, level, context, msg) +#define TRPC_LOG_MSG_IF_EX(level, context, condition, msg) \ + if (condition) { \ + TRPC_LOG_MSG_EX(level, context, msg); \ + } -#define TRPC_LOG_MSG_IF_EX(level, context, condition, msg) \ - if (condition) { \ - TRPC_LOG_MSG_EX(level, context, msg); \ +#define TRPC_LOGGER_MSG_IF_EX(level, context, instance, condition, msg) \ + if (condition) { \ + TRPC_LOGGER_MSG_EX(level, context, instance, msg); \ } #define TRPC_LOGGER_MSG_EX(level, context, instance, msg) TRPC_STREAM_EX(instance, level, context, msg) -#define TRPC_LOGGER_MSG_IF_EX(level, context, instance, condition, msg) \ - if (condition) { \ - TRPC_LOGGER_MSG_EX(level, context, instance, msg); \ - } +#define TRPC_LOG_MSG_EX(level, context, msg) TRPC_STREAM_EX_DEFAULT(::trpc::log::kTrpcLogCacheStringDefault, level, context, msg)