Register privileges in Kibana Platform Security plugin and remove legacy `getUser` API. #65472

azasypkin · 2020-05-06T13:12:56Z

Summary

In this PR we're:

Moving privileges registration to the Kibana Platform Security plugin from the legacy one relying on newly introduced Core's Status service.
Completely removing remaining public API exposed by the legacy Security plugin.
Registering Kibana Platform Security plugin with SASS linter.
Removing remaining dependency on injected vars (xpack.spaces.enabled)

Note to reviewers: it's a bit easier to review commit by commit (git move + changes).

If you received a review ping: main X-Pack API integration test config was migrated to TS and hence required fix in all references.

Blocked by: #65461

elasticmachine · 2020-05-06T13:12:58Z

Pinging @elastic/kibana-security (Team:Security)

… UI.

azasypkin · 2020-05-18T13:30:37Z

x-pack/plugins/security/public/management/roles/edit_role/edit_role_page.tsx

+      fetchedSpaces => setSpaces({ enabled: true, list: fetchedSpaces }),
+      (err: IHttpFetchError) => {
+        // Spaces plugin can be disabled and hence this endpoint can be unavailable.
+        if (err.response?.status === 404) {


note: I moved this change to a separate commit as I want to hear your thoughts. It's "the least evil" solution I could come up with. Alternatively we can introduce an endpoint that will return isSpacesEnabled only based on the "Spaces Service" that Spaces registers with Security during setup.

I think this is tolerable. I think it'd make sense (in a followup) for us to export a useSpaces hook from the Spaces plugin itself, so that we could at least have the Spaces plugin own this determination.

The disabled check aside, I could see other plugins benefiting from this hook too.

Yeah, I agree

legrego · 2020-05-19T12:08:20Z

x-pack/plugins/security/server/authorization/authorization_service.ts

+        validateFeaturePrivileges(allFeatures);
+        validateReservedPrivileges(allFeatures);
+
+        await registerPrivilegesWithCluster(


I think we have a gap in error handling here. The legacy implementation had retry logic (with backoff) in the event of a transient failure, which I think we want here as well. If await registerPrivilegesWithCluster() throws an error, but neither this.status.core$ nor this.license.features$ emit a new value, then Kibana will be stuck in an unusable state from an authorization perspective. We will either have to wait for one of those two observables to change (if that ever happens), or Kibana will have to be restarted in order to reattempt privilege registration.

That's a good point, thanks! I naively expected status service to catch intermittent 503s and emit new status updates, but it seems not be the case since you're bringing this up 🙂 I'm not sure how I feel about having this logic (not simple, prone to errors) in every plugin though. Let me think a bit...

That's a good point, thanks! I naively expected status service to catch intermittent 503s and emit new status updates

I think it will definitely catch some of these, but I could also imagine a race condition where core thinks everything is OK, but then we get network blip where we lose our connection for a very short period of time. core might not catch this blip if it wasn't doing anything with ES at the time.

It's also possible that the ES connection would be technically ok, but if the cluster is starved for resources, then it's possible that our specific call to register privileges would fail (disk space watermark, readonly indices, etc.)

I'm not sure how I feel about having this logic (not simple, prone to errors) in every plugin though. Let me think a bit...

Yeah, that's definitely a tradeoff. That's probably one of the reasons we originally had this as a reusable utility, so that each plugin wouldn't have to implement its own.

It's also possible that the ES connection would be technically ok, but if the cluster is starved for resources, then it's possible that our specific call to register privileges would fail (disk space watermark, readonly indices, etc.)

The thing that concerns me most is that Core determines the first time when we try to perform an action, but if that action fails, then responsibility shifts to a consumer that now decides when and if it's appropriate to do a re-try. It seems we don't have a way to force Core to check whether ES is available or not at the time we do a re-try. If ES is struggling for whatever reason we (every plugin that needs to do a re-try) don't want to make things worse via bombarding it with the requests. Ideally Core should be a single source of truth that decides whether it's appropriate to do a re-try or not.

That's a fair point. @elastic/kibana-platform do you have any thoughts around this?

To summarize, we are waiting for Core to tell us when ES is ready via the exposed CoreStatus service. Once all prerequisites are met, we attempt to make a couple of calls to ES. If these checks fail though, we would like to be able to retry them at some point in the future, but it's possible that everything appears "fine" from the perspective of CoreStatus, so we will never get an updated value from the Observable.

Having each plugin implement its own retry logic is feeling a bit complex and error prone. Aleh also makes a good point that plugins shouldn't be exacerbating the problem by retrying whenever they feel it is appropriate, but instead should wait on a signal from Core. The problem is that we don't have a way to alert Core that something went wrong.

@legrego I don't remember we discussed this case. Added a comment to the status service meta-issue #41983 (comment)

cchaos

SASS change is fine. Thanks for adding the Sass linting!

azasypkin · 2020-06-03T06:50:36Z

x-pack/plugins/security/server/authorization/authorization_service.ts

+    let retryTimeout: NodeJS.Timeout;
+
+    // Register cluster privileges once Elasticsearch is available and Security plugin is enabled.
+    this.statusSubscription = combineLatest([


note: presumably we'll get rid of this code in 7.10, so I gave up on learning how to properly leverage retryWhen here and went with plain setTimeout approach 🙈

azasypkin · 2020-06-03T06:51:47Z

@legrego should be ready for another pass, thanks!

elasticmachine · 2020-06-04T10:57:23Z

Pinging @elastic/apm-ui (Team:apm)

elasticmachine · 2020-06-04T10:57:24Z

Pinging @elastic/uptime (Team:uptime)

elasticmachine · 2020-06-04T10:57:26Z

Pinging @elastic/ingest-management (Team:Ingest Management)

mikecote

Alerting changes LGTM

sorenlouv

apm changes lgtm

legrego

note I'm running a flaky test suite for the api_integration ciGroup to make sure this is stable. I worry that we might not be immediately notified of the license change, or that the privilege registration will take longer than the subsequent API request to get the next set of privileges

https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/512/

Once the flaky runner completes, this LGTM! Thanks for the edits!

x-pack/test/api_integration/apis/security/license_downgrade.ts

jen-huang

Ingest manager README changes LGTM

kibanamachine · 2020-06-04T18:54:36Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 7983d5e

History

💚 Build #51729 succeeded 00cc050
💔 Build #51687 failed 108d1cf
💚 Build #51116 succeeded 8bd3ea3
💚 Build #48624 succeeded a108eb9
💚 Build #48608 succeeded baa580c

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

azasypkin · 2020-06-05T06:11:36Z

note I'm running a flaky test suite for the api_integration ciGroup to make sure this is stable. I worry that we might not be immediately notified of the license change, or that the privilege registration will take longer than the subsequent API request to get the next set of privileges
https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/512/

Thanks! All 42 runs of flaky test suite passed, but I'll keep eye on this. In case that start failing I'll just convert final expect into busy-waiting loop.

…acy `getUser` API. (elastic#65472) # Conflicts: # x-pack/README.md # x-pack/plugins/uptime/README.md # x-pack/scripts/functional_tests.js

azasypkin · 2020-06-05T12:12:56Z

7.x/7.9.0: 2890d5b

azasypkin added chore blocked Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! release_note:skip Skip the PR/issue when compiling release notes Feature:Security/Authorization Platform Security - Authorization Feature:NP Migration v7.9.0 labels May 6, 2020

azasypkin force-pushed the issue-xxx-np-migration branch 2 times, most recently from d4c6f7b to e44e540 Compare May 13, 2020 09:14

azasypkin mentioned this pull request May 13, 2020

Migrate Security test plugins to KP #66057

Closed

Git move.

2769237

azasypkin removed the blocked label May 18, 2020

azasypkin force-pushed the issue-xxx-np-migration branch from e44e540 to 44b39b3 Compare May 18, 2020 10:58

Register cluster privileges in Kibana Platform Security plugin.

baa580c

azasypkin force-pushed the issue-xxx-np-migration branch from 44b39b3 to baa580c Compare May 18, 2020 11:24

Remove explicit dependency on Spaces plugin from edit role management…

a108eb9

… UI.

azasypkin commented May 18, 2020

View reviewed changes

azasypkin marked this pull request as ready for review May 18, 2020 14:01

azasypkin requested review from a team as code owners May 18, 2020 14:01

Merge branch 'master' into issue-xxx-np-migration

ca2adb4

legrego reviewed May 19, 2020

View reviewed changes

Merge branch 'master' into issue-xxx-np-migration

1aa53a6

cchaos approved these changes May 20, 2020

View reviewed changes

joshdover mentioned this pull request May 20, 2020

Migrate status service and status page to New Platform #41983

Closed

15 tasks

azasypkin added 2 commits June 2, 2020 08:58

Merge branch 'master' into issue-xxx-np-migration

c8b57bc

Retry privileges registration if it fails.

8bd3ea3

azasypkin commented Jun 3, 2020

View reviewed changes

Fix test config paths.

00cc050

azasypkin requested review from a team as code owners June 4, 2020 10:57

azasypkin requested a review from a team June 4, 2020 10:57

azasypkin requested a review from a team as a code owner June 4, 2020 10:57

azasypkin requested a review from a team June 4, 2020 10:57

azasypkin requested review from a team as code owners June 4, 2020 10:57

botelastic bot added Team:APM All issues that need APM UI Team support Team:Fleet Team label for Observability Data Collection Fleet team Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability labels Jun 4, 2020

mikecote approved these changes Jun 4, 2020

View reviewed changes

azasypkin mentioned this pull request Jun 4, 2020

Record security feature usage #67526

Merged

sorenlouv approved these changes Jun 4, 2020

View reviewed changes

legrego approved these changes Jun 4, 2020

View reviewed changes

x-pack/test/api_integration/apis/security/license_downgrade.ts Show resolved Hide resolved

Merge branch 'master' into issue-xxx-np-migration

7983d5e

jen-huang approved these changes Jun 4, 2020

View reviewed changes

azasypkin merged commit c6e2fed into elastic:master Jun 5, 2020

azasypkin deleted the issue-xxx-np-migration branch June 5, 2020 06:12

azasypkin mentioned this pull request Jun 5, 2020

[7.x] Register privileges in Kibana Platform Security plugin and remove legacy getUser API. (#65472) #68344

Merged

azasypkin added backport pending backported and removed backport pending labels Jun 5, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Register privileges in Kibana Platform Security plugin and remove legacy `getUser` API. #65472

Register privileges in Kibana Platform Security plugin and remove legacy `getUser` API. #65472

azasypkin commented May 6, 2020 •

edited

Loading

elasticmachine commented May 6, 2020

azasypkin May 18, 2020

legrego May 19, 2020

azasypkin May 19, 2020

legrego May 19, 2020

azasypkin May 19, 2020

legrego May 19, 2020

azasypkin May 19, 2020

legrego May 19, 2020

mshustov May 20, 2020

cchaos left a comment

azasypkin Jun 3, 2020

azasypkin commented Jun 3, 2020

elasticmachine commented Jun 4, 2020

elasticmachine commented Jun 4, 2020

elasticmachine commented Jun 4, 2020

mikecote left a comment

sorenlouv left a comment

legrego left a comment •

edited

Loading

jen-huang left a comment

kibanamachine commented Jun 4, 2020

azasypkin commented Jun 5, 2020

azasypkin commented Jun 5, 2020

Register privileges in Kibana Platform Security plugin and remove legacy getUser API. #65472

Register privileges in Kibana Platform Security plugin and remove legacy getUser API. #65472

Conversation

azasypkin commented May 6, 2020 • edited Loading

Summary

elasticmachine commented May 6, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cchaos left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

azasypkin commented Jun 3, 2020

elasticmachine commented Jun 4, 2020

elasticmachine commented Jun 4, 2020

elasticmachine commented Jun 4, 2020

mikecote left a comment

Choose a reason for hiding this comment

sorenlouv left a comment

Choose a reason for hiding this comment

legrego left a comment • edited Loading

Choose a reason for hiding this comment

jen-huang left a comment

Choose a reason for hiding this comment

kibanamachine commented Jun 4, 2020

💚 Build Succeeded

History

azasypkin commented Jun 5, 2020

azasypkin commented Jun 5, 2020

Register privileges in Kibana Platform Security plugin and remove legacy `getUser` API. #65472

Register privileges in Kibana Platform Security plugin and remove legacy `getUser` API. #65472

azasypkin commented May 6, 2020 •

edited

Loading

legrego left a comment •

edited

Loading