[Execution context] Add nested context support #107523

mshustov · 2021-08-03T13:30:43Z

Summary

Adds support for declaring parent-child relationships between execution contexts. With this change, Kibana might log and pass to Elasticsearch information about the origin of every spawned execution context.
kibana logs

{
   "parent":{
      "type":"application",
      "name":"dashboard",
      "id":"722b74f0-b882-11e8-a6d9-e546fe2bba5f",
      "description":"[eCommerce] Revenue Dashboard",
      "url":"/view/722b74f0-b882-11e8-a6d9-e546fe2bba5f"
   },
   "type":"visualization",
   "name":"metrics",
   "id":"45e07720-b890-11e8-a6d9-e546fe2bba5f",
   "description":"[eCommerce] Promotion Tracking",
   "url":"/app/visualize#/edit/45e07720-b890-11e8-a6d9-e546fe2bba5f"
}

x-opaque-id header

75221494-b6fb-40e2-a392-fdab0f500eb8;kibana:application:dashboard:722b74f0-b882-11e8-a6d9-e546fe2bba5f;visualization:metrics:45e07720-b890-11e8-a6d9-e546fe2bba5f

Client-side API

Due to lack of native JS API, we have to pass a parent context during instantiation of a child context:

// in the parent app
const parentContext = {
  type: 'application',
  name: 'dashboard',
  ...
});

createChild(parentContext);

// in a child app
const context = executionContext.create({
  parent: parentContext,
  type: 'vis',
  ...
});

Note that EmbeddableInput is expected to be compatible with SerializableState interface, so I had to remove IExecutionContextContainer usage in favor of KibanaExecutionContext which is POJO. With this change, using execution_context service on the client-side looks like overhead, so I removed it completely.

Server-side API

On the server-side, plugins may register execution context by wrapping any function in withContext. Provided context object is carried over all async operations spawned by the passed function. Sequentual calls withContext will create a stack of context.

// in the parent app
executionContext.withContext({
  type: 'task',
  name: 'my-task',
  ...
}, async () => {
  return await fetchData();
});

How to test

logging:
  appenders:
    myconsole:
      type: console
      layout:
        type: pattern
        pattern: '%date|%meta'
  loggers:
    - name: elasticsearch.query
      level: all
      appenders: [myconsole]
    - name: elasticsearch.query.taskManager
      level: off
    - name: execution_context
      level: debug
      appenders: [console]

run Kibana
navigate to Kibana Browser App
Install sample data. I tested on Sample eCommerce orders.
Navigate to dashboards.
You should be able to see the output like this:

[2021-08-03T15:30:58.891+03:00][DEBUG][execution_context] set the execution context: {"parent":{"type":"application","name":"dashboard","id":"722b74f0-b882-11e8-a6d9-e546fe2bba5f","description":"[eCommerce] Revenue Dashboard","url":"/view/722b74f0-b882-11e8-a6d9-e546fe2bba5f"},"type":"visualization","name":"metrics","id":"45e07720-b890-11e8-a6d9-e546fe2bba5f","description":"[eCommerce] Promotion Tracking","url":"/app/visualize#/edit/45e07720-b890-11e8-a6d9-e546fe2bba5f"}

2021-08-03T15:30:58.899+03:00|{"http":{"request":{"id":"75221494-b6fb-40e2-a392-fdab0f500eb8;kibana:application:dashboard:722b74f0-b882-11e8-a6d9-e546fe2bba5f;visualization:metrics:45e07720-b890-11e8-a6d9-e546fe2bba5f"}}}

Checklist

Delete any items that are not applicable to this PR.

Unit or functional tests were updated or added to match the most common scenarios

For maintainers

This was checked for breaking API changes and was labeled appropriately

ExecutionContextContaier is not compatible with SerializableState, so I had to fall back to passing context as POJO. With this change, using a service looks like overhead.

elasticmachine · 2021-08-04T04:46:59Z

Pinging @elastic/kibana-core (Team:Core)

ppisljar

app services code LGTM

poffdeluxe

Dashboard changes look fine to me!

joshdover · 2021-08-05T16:11:46Z

Wanted to get to this today but ran out of time, will do tomorrow - Aug 6

src/core/public/execution_context/index.ts

joshdover · 2021-08-06T14:27:10Z

src/core/public/execution_context/execution_context_container.ts

@@ -34,7 +34,7 @@ export interface IExecutionContextContainer {
 }

 export class ExecutionContextContainer implements IExecutionContextContainer {


Maybe this class would make more sense to be named as ExecutionContextSerializer now?

We will have different abstractions on the client and server then. I'd keep it as is. Considering that it's an internal API, we can always rename it later.

src/core/public/execution_context/execution_context_container.ts

src/core/public/execution_context/execution_context_service.mock.ts

src/core/public/public.api.md

src/core/server/execution_context/execution_context_service.ts

joshdover · 2021-08-06T14:44:44Z

src/core/server/execution_context/integration_tests/tracing.test.ts

+          description: 'new-description',
+        };
+        router.get({ path: '/execution-context', validate: false }, async (context, req, res) => {
+          const { headers } = await executionContext.withContext(newContext, () =>


Should we provide executionContext.withContext on RequestHandlerContext too?

I'd wait until the first use case.
I don't think lots of endpoints will use this API manually to create a domain-specific context. For most of the cases, instrumentation performed by Core will be enough.

src/core/server/execution_context/integration_tests/tracing.test.ts

src/core/server/execution_context/execution_context_service.test.ts

joshdover · 2021-08-06T15:01:56Z

src/core/server/execution_context/execution_context_service.test.ts

+          name: 'name-b',
+          id: 'id-b',
+          description: 'description-b',
+          parent: undefined,


I'm curious if this is the behavior we actually want instead of subsequent calls using the previous context as the parent. If pluginA calls a sync API on pluginB and they both set context, wouldn't we want to track that relationship?

If pluginA calls a sync API on pluginB and they both set context, wouldn't we want to track that relationship?

set is Core-specific API which won't be provided to plugins. withContext which is exposed to plugins, supports context stacking

kibana/src/core/server/execution_context/execution_context_service.test.ts

Line 231 in ae45719

it('supports nested contexts', async () => {

kertal

KibanaApp owned code LGTM, tested adding TSVB + Lens visualization to Dashboard, since embeddable code was modified, works as expected

joshdover

LGTM

mshustov · 2021-08-09T11:35:34Z

@elasticmachine merge upstream

…o issue-102626-nested-context

kibanamachine · 2021-08-09T15:37:34Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`core`	368	367	-1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`dashboard`	221.5KB	221.9KB	+413.0B
`lens`	1.6MB	1.6MB	-149.0B
`visualizations`	101.8KB	101.8KB	+29.0B
total			+293.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`core`	423.9KB	423.4KB	-513.0B
`dashboard`	329.2KB	329.3KB	+68.0B
`data`	791.3KB	791.2KB	-132.0B
`lens`	29.1KB	29.0KB	-112.0B
total			-689.0B

Unknown metric groups

API count

id	before	after	diff
`core`	2455	2436	-19
`dashboard`	160	161	+1
total			-18

API count missing comments

id	before	after	diff
`core`	1166	1160	-6
`dashboard`	137	138	+1
total			-5

History

💚 Build #143632 succeeded 65da137
💔 Build #143614 failed 27aca6c
💔 Build #143601 failed 14f5201
💛 Build #142850 was flaky 63e59a9
💔 Build #142736 failed 6206ac2

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* Add nested context support * remove execution context service on the client side ExecutionContextContaier is not compatible with SerializableState, so I had to fall back to passing context as POJO. With this change, using a service looks like overhead. * update docs * fix test * address comments from Josh * put export back * update docs * remove outdated export * use input.title for unsaved vis Co-authored-by: Kibana Machine <[email protected]>

* Add nested context support * remove execution context service on the client side ExecutionContextContaier is not compatible with SerializableState, so I had to fall back to passing context as POJO. With this change, using a service looks like overhead. * update docs * fix test * address comments from Josh * put export back * update docs * remove outdated export * use input.title for unsaved vis Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Kibana Machine <[email protected]>

Add nested context support

ab44dd0

mshustov added chore Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.15.0 labels Aug 3, 2021

mshustov added 3 commits August 3, 2021 19:15

remove execution context service on the client side

23a69d2

ExecutionContextContaier is not compatible with SerializableState, so I had to fall back to passing context as POJO. With this change, using a service looks like overhead.

update docs

6206ac2

fix test

63e59a9

mshustov marked this pull request as ready for review August 4, 2021 04:46

mshustov requested a review from a team August 4, 2021 04:46

mshustov requested review from a team as code owners August 4, 2021 04:46

ppisljar approved these changes Aug 4, 2021

View reviewed changes

botelastic bot added Feature:Embedding Embedding content via iFrame Feature:ExpressionLanguage Interpreter expression language (aka canvas pipeline) labels Aug 4, 2021

poffdeluxe approved these changes Aug 4, 2021

View reviewed changes

joshdover reviewed Aug 6, 2021

View reviewed changes

Merge branch 'master' into issue-102626-nested-context

ae45719

kertal approved these changes Aug 9, 2021

View reviewed changes

mshustov added 3 commits August 9, 2021 12:19

address comments from Josh

cf28920

put export back

0fde05c

update docs

14f5201

mshustov requested a review from joshdover August 9, 2021 09:39

remove outdated export

27aca6c

mshustov mentioned this pull request Aug 9, 2021

Implement Elasticsearch query tracing to a source in Kibana #101587

Closed

34 tasks

joshdover approved these changes Aug 9, 2021

View reviewed changes

kibanamachine and others added 3 commits August 9, 2021 07:35

Merge branch 'master' into issue-102626-nested-context

65da137

use input.title for unsaved vis

ac5e2c0

Merge remote-tracking branch 'origin/issue-102626-nested-context' int…

b91100e

…o issue-102626-nested-context

mshustov merged commit 5480c4d into elastic:master Aug 10, 2021

mshustov deleted the issue-102626-nested-context branch August 10, 2021 06:53

mshustov mentioned this pull request Aug 10, 2021

[7.x] [Execution context] Add nested context support (#107523) #108002

Merged

nreese mentioned this pull request Apr 24, 2024

[dashboard] investigate execution context #181511

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Execution context] Add nested context support #107523

[Execution context] Add nested context support #107523

mshustov commented Aug 3, 2021 •

edited

Loading

elasticmachine commented Aug 4, 2021

ppisljar left a comment

poffdeluxe left a comment

joshdover commented Aug 5, 2021

joshdover Aug 6, 2021

mshustov Aug 9, 2021

joshdover Aug 6, 2021

mshustov Aug 9, 2021

joshdover Aug 6, 2021

mshustov Aug 9, 2021

kertal left a comment

joshdover left a comment

mshustov commented Aug 9, 2021

kibanamachine commented Aug 9, 2021

API count

API count missing comments

		@@ -34,7 +34,7 @@ export interface IExecutionContextContainer {
		}

		export class ExecutionContextContainer implements IExecutionContextContainer {

[Execution context] Add nested context support #107523

[Execution context] Add nested context support #107523

Conversation

mshustov commented Aug 3, 2021 • edited Loading

Summary

Client-side API

Server-side API

How to test

Checklist

For maintainers

elasticmachine commented Aug 4, 2021

ppisljar left a comment

Choose a reason for hiding this comment

poffdeluxe left a comment

Choose a reason for hiding this comment

joshdover commented Aug 5, 2021

joshdover Aug 6, 2021

Choose a reason for hiding this comment

mshustov Aug 9, 2021

Choose a reason for hiding this comment

joshdover Aug 6, 2021

Choose a reason for hiding this comment

mshustov Aug 9, 2021

Choose a reason for hiding this comment

joshdover Aug 6, 2021

Choose a reason for hiding this comment

mshustov Aug 9, 2021

Choose a reason for hiding this comment

kertal left a comment

Choose a reason for hiding this comment

joshdover left a comment

Choose a reason for hiding this comment

mshustov commented Aug 9, 2021

kibanamachine commented Aug 9, 2021

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Page load bundle

API count

API count missing comments

History

mshustov commented Aug 3, 2021 •

edited

Loading