Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[core.logging] Use host timezone in log records by default #90255

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
b2b67fe
Use host timezone in log records by default
TinaHeiligers Feb 3, 2021
5e9a458
Updates logging_system test snapshots
TinaHeiligers Feb 3, 2021
dd6cc7a
Updates legacy logging integration test snapshots
TinaHeiligers Feb 3, 2021
82e8cbe
Merge branch 'master' into logging/es-compatible-tz-default
kibanamachine Feb 4, 2021
b5ce8ba
[Search Session][Management] Rename "cancel" button and delete "Reloa…
lizozom Feb 4, 2021
c37b0e1
[Discover] Minor cleanup (#90260)
Feb 4, 2021
e013389
[Security Solution][Detections] Reduce detection engine reliance on _…
marshallmain Feb 4, 2021
1741cef
[Lens] Hide column in table (#88680)
flash1293 Feb 4, 2021
e676617
Test user for maps tests under import geoJSON tests (#86015)
bhavyarm Feb 4, 2021
7a42a6f
[APM] Enabling yesterday option when 24 hours is selected (#90017)
cauemarcondes Feb 4, 2021
5052499
Add readme to geo containment alert covering test alert setup (#89625)
Feb 4, 2021
0f45439
skip flaky suite (#85086)
mistic Dec 8, 2020
54b1fb6
Use newfeed.service config for all newsfeeds (#90252)
gtback Feb 4, 2021
85e92b5
Remove UI filters from UI (#89793)
smith Feb 4, 2021
82df009
[App Search] Relevance Tuning logic - actions and selectors only, no …
JasonStoltz Feb 4, 2021
0ef276d
Use doc link service in more Stack Monitoring pages (#89050)
lcawl Feb 4, 2021
2955d65
[Enterprise Search] Refactor MockRouter test helper to not store payl…
constancecchen Feb 4, 2021
284842d
[Workplace Search] Fix Source Settings bug (#90242)
scottybollinger Feb 4, 2021
9e7e1e1
[Fleet] Managed Agent Policy (#88688)
Feb 4, 2021
808bd44
Use doc link services in index pattern management (#89937)
lcawl Feb 4, 2021
b4e597c
updates kbn settings docs for logging.timezone
TinaHeiligers Feb 4, 2021
35fd85b
Elastic Maps Server config is `host` not `hostname` (#90234)
nickpeihl Feb 4, 2021
bb3ed33
RFC for automatically generated typescript API documentation for ever…
stacey-gammon Feb 4, 2021
bf3df5f
Merge branch 'logging/es-compatible-tz-default' of github.com:TinaHei…
TinaHeiligers Feb 4, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ readonly links: {
readonly installation: string;
readonly configuration: string;
readonly elasticsearchOutput: string;
readonly elasticsearchModule: string;
readonly startup: string;
readonly exportedFields: string;
};
Expand All @@ -29,6 +30,10 @@ readonly links: {
};
readonly metricbeat: {
readonly base: string;
readonly configure: string;
readonly httpEndpoint: string;
readonly install: string;
readonly start: string;
};
readonly enterpriseSearch: {
readonly base: string;
Expand Down
4 changes: 2 additions & 2 deletions docs/maps/connect-to-ems.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ endif::[]

[cols="2*<"]
|===
| [[ems-hostname]]`hostname`
| [[ems-host]]`host`
| Specifies the host of the backend server. To allow remote users to connect, set the value to the IP address or DNS name of the {hosted-ems} container. *Default: _your-hostname_*. <<server-host,Equivalent {kib} setting>>.

| `port`
Expand Down Expand Up @@ -199,7 +199,7 @@ TIP: The available basemaps and boundaries can be explored from the `/maps` endp
[[elastic-maps-server-kibana]]
==== Kibana configuration

With {hosted-ems} running, add the `map.emsUrl` configuration key in your <<settings, kibana.yml>> file pointing to the root of the service. This setting will point {kib} to request EMS basemaps and boundaries from {hosted-ems}. Typically this will be the URL to the <<ems-hostname,hostname and port>> of {hosted-ems}. For example, `map.emsUrl: https://my-ems-server:8080`.
With {hosted-ems} running, add the `map.emsUrl` configuration key in your <<settings, kibana.yml>> file pointing to the root of the service. This setting will point {kib} to request EMS basemaps and boundaries from {hosted-ems}. Typically this will be the URL to the <<ems-host,host and port>> of {hosted-ems}. For example, `map.emsUrl: https://my-ems-server:8080`.


[float]
Expand Down
5 changes: 3 additions & 2 deletions docs/setup/settings.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -306,10 +306,11 @@ the `polling` method could be used enabling that option. *Default: `false`*
suppress all logging output. *Default: `false`*

| `logging.timezone`
| Set to the canonical time zone ID
| Optional. Set to the canonical time zone ID
(for example, `America/Los_Angeles`) to log events using that time zone.
For possible values, refer to
https://en.wikipedia.org/wiki/List_of_tz_database_time_zones[database time zones]. *Default: `UTC`*
https://en.wikipedia.org/wiki/List_of_tz_database_time_zones[database time zones]. When not set,
log dates will be in the {kib} host timezone.

| [[logging-verbose]] `logging.verbose:` {ess-icon}
| Set to `true` to log all events, including system usage information and all
Expand Down
Binary file added rfcs/images/api_doc_pick.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added rfcs/images/api_doc_tech.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added rfcs/images/api_doc_tech_compare.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added rfcs/images/api_docs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added rfcs/images/api_docs_package_current.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added rfcs/images/api_info.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added rfcs/images/current_api_doc_links.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added rfcs/images/new_api_docs_with_links.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added rfcs/images/repeat_primitive_signature.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added rfcs/images/repeat_type_links.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
442 changes: 442 additions & 0 deletions rfcs/text/0014_api_documentation.md

Large diffs are not rendered by default.

14 changes: 14 additions & 0 deletions src/core/public/doc_links/doc_links_service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ export class DocLinksService {
base: `${ELASTIC_WEBSITE_URL}guide/en/beats/filebeat/${DOC_LINK_VERSION}`,
installation: `${ELASTIC_WEBSITE_URL}guide/en/beats/filebeat/${DOC_LINK_VERSION}/filebeat-installation-configuration.html`,
configuration: `${ELASTIC_WEBSITE_URL}guide/en/beats/filebeat/${DOC_LINK_VERSION}/configuring-howto-filebeat.html`,
elasticsearchModule: `${ELASTIC_WEBSITE_URL}guide/en/beats/filebeat/${DOC_LINK_VERSION}/filebeat-module-elasticsearch.html`,
elasticsearchOutput: `${ELASTIC_WEBSITE_URL}guide/en/beats/filebeat/${DOC_LINK_VERSION}/elasticsearch-output.html`,
startup: `${ELASTIC_WEBSITE_URL}guide/en/beats/filebeat/${DOC_LINK_VERSION}/filebeat-starting.html`,
exportedFields: `${ELASTIC_WEBSITE_URL}guide/en/beats/filebeat/${DOC_LINK_VERSION}/exported-fields.html`,
Expand All @@ -53,6 +54,10 @@ export class DocLinksService {
},
metricbeat: {
base: `${ELASTIC_WEBSITE_URL}guide/en/beats/metricbeat/${DOC_LINK_VERSION}`,
configure: `${ELASTIC_WEBSITE_URL}guide/en/beats/metricbeat/${DOC_LINK_VERSION}/configuring-howto-metricbeat.html`,
httpEndpoint: `${ELASTIC_WEBSITE_URL}guide/en/beats/metricbeat/${DOC_LINK_VERSION}/http-endpoint.html`,
install: `${ELASTIC_WEBSITE_URL}guide/en/beats/metricbeat/${DOC_LINK_VERSION}/metricbeat-installation-configuration.html`,
start: `${ELASTIC_WEBSITE_URL}guide/en/beats/metricbeat/${DOC_LINK_VERSION}/metricbeat-starting.html`,
},
heartbeat: {
base: `${ELASTIC_WEBSITE_URL}guide/en/beats/heartbeat/${DOC_LINK_VERSION}`,
Expand Down Expand Up @@ -116,6 +121,7 @@ export class DocLinksService {
addData: `${ELASTIC_WEBSITE_URL}guide/en/kibana/${DOC_LINK_VERSION}/connect-to-elasticsearch.html`,
kibana: `${ELASTIC_WEBSITE_URL}guide/en/kibana/${DOC_LINK_VERSION}/index.html`,
elasticsearch: {
mapping: `${ELASTICSEARCH_DOCS}mapping.html`,
remoteClusters: `${ELASTICSEARCH_DOCS}modules-remote-clusters.html`,
remoteClustersProxy: `${ELASTICSEARCH_DOCS}modules-remote-clusters.html#proxy-mode`,
remoteClusersProxySettings: `${ELASTICSEARCH_DOCS}modules-remote-clusters.html#remote-cluster-proxy-settings`,
Expand Down Expand Up @@ -193,8 +199,11 @@ export class DocLinksService {
alertsKibanaDiskThreshold: `${ELASTIC_WEBSITE_URL}guide/en/kibana/${DOC_LINK_VERSION}/kibana-alerts.html#kibana-alerts-disk-usage-threshold`,
alertsKibanaJvmThreshold: `${ELASTIC_WEBSITE_URL}guide/en/kibana/${DOC_LINK_VERSION}/kibana-alerts.html#kibana-alerts-jvm-memory-threshold`,
alertsKibanaMissingData: `${ELASTIC_WEBSITE_URL}guide/en/kibana/${DOC_LINK_VERSION}/kibana-alerts.html#kibana-alerts-missing-monitoring-data`,
metricbeatBlog: `${ELASTIC_WEBSITE_URL}blog/external-collection-for-elastic-stack-monitoring-is-now-available-via-metricbeat`,
monitorElasticsearch: `${ELASTICSEARCH_DOCS}configuring-metricbeat.html`,
monitorKibana: `${ELASTIC_WEBSITE_URL}guide/en/kibana/${DOC_LINK_VERSION}/monitoring-metricbeat.html`,
monitorLogstash: `${ELASTIC_WEBSITE_URL}guide/en/logstash/${DOC_LINK_VERSION}/monitoring-with-metricbeat.html`,
troubleshootKibana: `${ELASTIC_WEBSITE_URL}guide/en/kibana/${DOC_LINK_VERSION}/monitor-troubleshooting.html`,
},
security: {
apiKeyServiceSettings: `${ELASTICSEARCH_DOCS}security-settings.html#api-key-service-settings`,
Expand Down Expand Up @@ -257,6 +266,7 @@ export interface DocLinksStart {
readonly installation: string;
readonly configuration: string;
readonly elasticsearchOutput: string;
readonly elasticsearchModule: string;
readonly startup: string;
readonly exportedFields: string;
};
Expand All @@ -265,6 +275,10 @@ export interface DocLinksStart {
};
readonly metricbeat: {
readonly base: string;
readonly configure: string;
readonly httpEndpoint: string;
readonly install: string;
readonly start: string;
};
readonly enterpriseSearch: {
readonly base: string;
Expand Down
5 changes: 5 additions & 0 deletions src/core/public/public.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -474,6 +474,7 @@ export interface DocLinksStart {
readonly installation: string;
readonly configuration: string;
readonly elasticsearchOutput: string;
readonly elasticsearchModule: string;
readonly startup: string;
readonly exportedFields: string;
};
Expand All @@ -482,6 +483,10 @@ export interface DocLinksStart {
};
readonly metricbeat: {
readonly base: string;
readonly configure: string;
readonly httpEndpoint: string;
readonly install: string;
readonly start: string;
};
readonly enterpriseSearch: {
readonly base: string;
Expand Down
20 changes: 10 additions & 10 deletions src/core/server/legacy/integration_tests/logging.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ describe('logging service', () => {
const loggedString = getPlatformLogsFromMock(mockConsoleLog);
expect(loggedString).toMatchInlineSnapshot(`
Array [
"[xxxx-xx-xxTxx:xx:xx.xxxZ][INFO ][test-file] handled by NP",
"[xxxx-xx-xxTxx:xx:xx.xxx-xx:xx][INFO ][test-file] handled by NP",
]
`);
});
Expand Down Expand Up @@ -131,9 +131,9 @@ describe('logging service', () => {

expect(getPlatformLogsFromMock(mockConsoleLog)).toMatchInlineSnapshot(`
Array [
"[xxxx-xx-xxTxx:xx:xx.xxxZ][INFO ][test-file] info",
"[xxxx-xx-xxTxx:xx:xx.xxxZ][WARN ][test-file] warn",
"[xxxx-xx-xxTxx:xx:xx.xxxZ][ERROR][test-file] error",
"[xxxx-xx-xxTxx:xx:xx.xxx-xx:xx][INFO ][test-file] info",
"[xxxx-xx-xxTxx:xx:xx.xxx-xx:xx][WARN ][test-file] warn",
"[xxxx-xx-xxTxx:xx:xx.xxx-xx:xx][ERROR][test-file] error",
]
`);

Expand Down Expand Up @@ -162,9 +162,9 @@ describe('logging service', () => {

expect(getPlatformLogsFromMock(mockConsoleLog)).toMatchInlineSnapshot(`
Array [
"[xxxx-xx-xxTxx:xx:xx.xxxZ][INFO ][test-file] info",
"[xxxx-xx-xxTxx:xx:xx.xxxZ][WARN ][test-file] warn",
"[xxxx-xx-xxTxx:xx:xx.xxxZ][ERROR][test-file] error",
"[xxxx-xx-xxTxx:xx:xx.xxx-xx:xx][INFO ][test-file] info",
"[xxxx-xx-xxTxx:xx:xx.xxx-xx:xx][WARN ][test-file] warn",
"[xxxx-xx-xxTxx:xx:xx.xxx-xx:xx][ERROR][test-file] error",
]
`);

Expand Down Expand Up @@ -199,9 +199,9 @@ describe('logging service', () => {

expect(getPlatformLogsFromMock(mockConsoleLog)).toMatchInlineSnapshot(`
Array [
"[xxxx-xx-xxTxx:xx:xx.xxxZ][INFO ][test-file] info",
"[xxxx-xx-xxTxx:xx:xx.xxxZ][WARN ][test-file] warn",
"[xxxx-xx-xxTxx:xx:xx.xxxZ][ERROR][test-file] error",
"[xxxx-xx-xxTxx:xx:xx.xxx-xx:xx][INFO ][test-file] info",
"[xxxx-xx-xxTxx:xx:xx.xxx-xx:xx][WARN ][test-file] warn",
"[xxxx-xx-xxTxx:xx:xx.xxx-xx:xx][ERROR][test-file] error",
]
`);

Expand Down
32 changes: 16 additions & 16 deletions src/core/server/logging/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,12 +110,12 @@ Example of `%meta` output:

##### date
Outputs the date of the logging event. The date conversion specifier may be followed by a set of braces containing a name of predefined date format and canonical timezone name.
Timezone name is expected to be one from [TZ database name](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)
Timezone name is expected to be one from [TZ database name](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). Timezone defaults to the host timezone when not explicitly specified.
Example of `%date` output:

| Conversion pattern | Example |
| ---------------------------------------- | ---------------------------------------------------------------- |
| `%date` | `2012-02-01T14:30:22.011Z` uses `ISO8601` format by default |
| `%date` | `2012-02-01T09:30:22.011-05:00` uses `ISO8601` format with timezone by default |
| `%date{ISO8601}` | `2012-02-01T14:30:22.011Z` |
| `%date{ISO8601_TZ}` | `2012-02-01T09:30:22.011-05:00` `ISO8601` with timezone |
| `%date{ISO8601_TZ}{America/Los_Angeles}` | `2012-02-01T06:30:22.011-08:00` |
Expand Down Expand Up @@ -410,22 +410,22 @@ loggerWithNestedContext.debug('Message with `debug` log level.');

And assuming logger for `server` context with `console` appender and `trace` level was used, console output will look like this:
```bash
[2017-07-25T18:54:41.639Z][TRACE][server] Message with `trace` log level.
[2017-07-25T18:54:41.639Z][DEBUG][server] Message with `debug` log level.
[2017-07-25T18:54:41.639Z][INFO ][server] Message with `info` log level.
[2017-07-25T18:54:41.639Z][WARN ][server] Message with `warn` log level.
[2017-07-25T18:54:41.639Z][ERROR][server] Message with `error` log level.
[2017-07-25T18:54:41.639Z][FATAL][server] Message with `fatal` log level.

[2017-07-25T18:54:41.639Z][TRACE][server.http] Message with `trace` log level.
[2017-07-25T18:54:41.639Z][DEBUG][server.http] Message with `debug` log level.
[2017-07-25T11:54:41.639-07:00][TRACE][server] Message with `trace` log level.
[2017-07-25T11:54:41.639-07:00][DEBUG][server] Message with `debug` log level.
[2017-07-25T11:54:41.639-07:00][INFO ][server] Message with `info` log level.
[2017-07-25T11:54:41.639-07:00][WARN ][server] Message with `warn` log level.
[2017-07-25T11:54:41.639-07:00][ERROR][server] Message with `error` log level.
[2017-07-25T11:54:41.639-07:00][FATAL][server] Message with `fatal` log level.

[2017-07-25T11:54:41.639-07:00][TRACE][server.http] Message with `trace` log level.
[2017-07-25T11:54:41.639-07:00][DEBUG][server.http] Message with `debug` log level.
```

The log will be less verbose with `warn` level for the `server` context:
```bash
[2017-07-25T18:54:41.639Z][WARN ][server] Message with `warn` log level.
[2017-07-25T18:54:41.639Z][ERROR][server] Message with `error` log level.
[2017-07-25T18:54:41.639Z][FATAL][server] Message with `fatal` log level.
[2017-07-25T11:54:41.639-07:00][WARN ][server] Message with `warn` log level.
[2017-07-25T11:54:41.639-07:00][ERROR][server] Message with `error` log level.
[2017-07-25T11:54:41.639-07:00][FATAL][server] Message with `fatal` log level.
```

### Logging config migration
Expand Down Expand Up @@ -488,7 +488,7 @@ logging.root.level: all

#### logging.timezone
Set to the canonical timezone id to log events using that timezone. New logging config allows
to [specify timezone](#date) for `layout: pattern`.
to [specify timezone](#date) for `layout: pattern`. Defaults to host timezone when not specified.
```yaml
logging:
appenders:
Expand Down Expand Up @@ -530,7 +530,7 @@ TBD

| Parameter | Platform log record in **pattern** format | Legacy Platform log record **text** format |
| --------------- | ------------------------------------------ | ------------------------------------------ |
| @timestamp | ISO8601 `2012-01-31T23:33:22.011Z` | Absolute `23:33:22.011` |
| @timestamp | ISO8601_TZ `2012-01-31T23:33:22.011-05:00` | Absolute `23:33:22.011` |
| context | `parent.child` | `['parent', 'child']` |
| level | `DEBUG` | `['debug']` |
| meta | stringified JSON object `{"to": "v8"}` | N/A |
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading