SavedObjects tagging MVP #79096
Merged
SavedObjects tagging MVP #79096
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
41 tasks
pgayvallet
added
v7.11.0
Team:Core
…tart to wire to SO management page.
ryankeairns
approved these changes
Oct 30, 2020
There was a problem hiding this comment.
Awesome work Pierre, this will generate a ton of excitement!
There was one small copy change that Gail requested, but I don't want to force another CI run on this PR since it's green. We can pick it up post merge.
Trust me, that's not the final build 😅 . Will do the wording change in this PR. |
mshustov
approved these changes
Nov 2, 2020
| @@ -51,6 +57,20 @@ export interface SavedObjectsExportOptions { | |||
| namespace?: string; | |||
| } | |||
|
|
|||
| type SavedObjectsFetchByTypeOptions = Pick<Required<SavedObjectsExportOptions>, 'types'> & | |||
There was a problem hiding this comment.
I'd rather declare the public interfaces explicitly:
interface SavedObjectsFetchByTypeOptions {
/** optional array of saved object types. */
types: string[];
/** optional array of references to search object for when exporting by types */
hasReference?: SavedObjectsFindOptionsReference[];
/** optional query string to filter exported objects. */
search?: string;
/** an instance of the SavedObjectsClient. */
savedObjectsClient: SavedObjectsClientContract;
/** the maximum number of objects to export. */
exportSizeLimit: number;
/** optional namespace to override the namespace used by the savedObjectsClient. */
namespace?: string;
}
interface SavedObjectsFetchByObjectOptions {
/** optional array of objects to export. */
objects: Array<{
/** the saved object id. */
id: string;
/** the saved object type. */
type: string;
}>;
/** an instance of the SavedObjectsClient. */
savedObjectsClient: SavedObjectsClientContract;
/** the maximum number of objects to export. */
exportSizeLimit: number;
/** optional namespace to override the namespace used by the savedObjectsClient. */
namespace?: string;
}| types, | ||
| }: SavedObjectsExportOptions): | ||
| | SavedObjectsFetchByTypeOptions | ||
| | SavedObjectsFetchByObjectOptions => { |
There was a problem hiding this comment.
would it be better to define return type explicitly instead?
const fetchByObjectOptions: SavedObjectsFetchByObjectOptions = {
objects,
exportSizeLimit,
savedObjectsClient,
namespace,
};
return fetchByObjectOptions;| ); | ||
|
|
||
| if (body.failures?.length) { | ||
| throw SavedObjectsErrorHelpers.createConflictError(type, id); |
There was a problem hiding this comment.
ok, at least something. I didn't manage to find the error interface in ES either https://github.com/elastic/elasticsearch/blob/291f5bd1b2e889e9447d660e5407f3120cffb1a5/client/rest-high-level/src/test/java/org/elasticsearch/client/UpdateByQueryIT.java#L234-L235
| this.api = api; | ||
| }, | ||
| (error) => { | ||
| this.apiRegistered = false; |
There was a problem hiding this comment.
ok, even though registerTaggingApi doesn't know what code created provider promise, it might be overengineering to add a proper error handling. console.error is fine, imo
| "kibanaVersion": "kibana", | ||
| "server": true, | ||
| "ui": true, | ||
| "configPath": ["xpack", "saved_object_tagging"], |
There was a problem hiding this comment.
Until we've refactored all config keys in a single big refactor, all existing and new config keys should keep using camelCase for consistency.
I believe it's for the existing keys. I'd rather using snake_case for the new ones to prevent refactoring in the future.
| }, | ||
| }; | ||
|
|
||
| export const USERS = { |
|
@pgayvallet as requested, here's a quick description of tags for the the release notes. Thoughts?
|
💛 Build succeeded, but was flaky
Test FailuresFirefox UI Functional Tests.test/functional/apps/visualize/_tsvb_chart·ts.visualize app visual builder metric should populate fields for basic functionsStandard OutStack TraceMetrics [docs]@kbn/optimizer bundle module count
async chunk count
async chunks size
distributable file count
page load bundle size
Saved Objects .kibana field count
History
To update your PR or re-run it, just comment with: |
pgayvallet
added a commit
to pgayvallet/kibana
that referenced
this pull request
Nov 3, 2020
* create xpack plugin skeleton, start to implement management section * add tag creation modal * first implementation of the tags table * use InMemoryTable * add edit modal and delete action * update plugin list * add tag list, fix types * add capabilities check on client-side * add tag combo box component * add missing i18n keys * fix privilege FTR tests * add base structure for FTR tests * fix feature ftr test * use string literals for i18n * create savedObjectsTaggingOss plugin, move API types to oss plugin, start to wire to SO management page. * update plugin list * fix types * allow to use `_find` with multiple references * add FTR test for _find API on references fields * add _find integration tests * update generated doc * start to implement tag filtering on SO management section * update generated docs * wire tagging API to dashboard listing page * fix i18n namespace * fix type & tests * update dashboard listing snapshots * adapt FTR listingTable service to search for parsable queries * wite tagging API to visualize listing * update tagging plugin limits * add server-side and client-side validation for tag create/edit * rename title field to name * fix types * fix types bis * add removeReferencesTo API to SOR/SOC * update generated doc * add server-side unit test for `savedObjectsTagging` plugin * move tagging API types to its own file * add savedObjectsTaggingOss mock * add tags_cache tests * add tests for client-side tag client * extract uiApi to distinct files * various API improvements * add more tests * add link between tag and so management sections + add connection counts * add base functional test suite for tagging * add more FTR tests * improve feature control func test * update codeowners * update generated doc * fix access to proxy modal * adapt SO save modal to allow to add tag field * add SO decorator registry and tag implementation * add unit tests for SO tag decorator * add functional tests for visualize integration * add tag SO read permission for vis/dash feature * add RBAC api integ tests * add API integration tests * add test for getTagConnectionsUrl * add SOM test suite * add dashboard integration suite * remove test line * add missing unit tests * improve API types doc * fix create modal save button label * remove console.log * improve doc * self review * add refresh interval for tag cache * improve page object doc * minor cleanup * address review comments * small layout fixes * add initial focus * use lazy accessor for tag request handler context * adapt SOM export and export route to handle references * remove icon from feature config due to master changes * fix SO table tests * update generated docs * sort tags by name in filter dropdown and listing component * wire SO tagging to dashboard save modal * fix types * - add 'create tag' action in tag selector - add notifications on update/create/delete from management - delete modal wording * add description max length validation * remove real-time validation * fix i18n bundle id * update expected size of savedObjectsTagging plugin * use own useIfMounted * update limit again, contract components cannot be lazy loaded atm. * math is hard * remove single usage of lodash for bundle size * add async imports for create/edit modal * add FTR test for 'create tag' action from tag selector * allow 'create new' option to prepopulate name field * extract savedObjectToTag * add advancedSettings read user for security api_integ suite * add audit login for security client wrapper * use import type when possible * wire SO tagging to lens visualization * fix lens jest test * Fix `create tag` option being selected when closing the selector dropdown * add sorting to tag column from getTableColumnDef * address some of restrry comments * rename tag selector's setSelected option to onTagsSelected * fix audit logging even type for saved_object_remove_references * update plugin size limit to current size * adapt maxlength validation wording * remove selection column until we have batch action menu * remove connections link when user lack read privilege to savedObjectManagement * forbid registering multiple SO decorators with the same priority * add so decorator test * extract getTagFindReferences and create API mock * update audit-logging ascidoc * doc nit * throw conflict error if update returns any failure * use refresh=true as default * wording nits * export: rename `references` to `hasReference` * update generated doc * set description max length to 100 * do not initialize tag cache on anonymous pages * split fetchObjectsToExport into two distinct functions * change tag client `delete` call order * tsdoc nits * more nits * add README for oss plugin * add oss plugin start tests * SavedObject.find: rename `references` to `hasReference` * change section description label * remove url prefix constants * last nits and comments * update generated doc # Conflicts: # .github/CODEOWNERS # packages/kbn-optimizer/limits.yml # x-pack/scripts/functional_tests.js
pgayvallet
added a commit
that referenced
this pull request
Nov 3, 2020
* SavedObjects tagging MVP (#79096) * create xpack plugin skeleton, start to implement management section * add tag creation modal * first implementation of the tags table * use InMemoryTable * add edit modal and delete action * update plugin list * add tag list, fix types * add capabilities check on client-side * add tag combo box component * add missing i18n keys * fix privilege FTR tests * add base structure for FTR tests * fix feature ftr test * use string literals for i18n * create savedObjectsTaggingOss plugin, move API types to oss plugin, start to wire to SO management page. * update plugin list * fix types * allow to use `_find` with multiple references * add FTR test for _find API on references fields * add _find integration tests * update generated doc * start to implement tag filtering on SO management section * update generated docs * wire tagging API to dashboard listing page * fix i18n namespace * fix type & tests * update dashboard listing snapshots * adapt FTR listingTable service to search for parsable queries * wite tagging API to visualize listing * update tagging plugin limits * add server-side and client-side validation for tag create/edit * rename title field to name * fix types * fix types bis * add removeReferencesTo API to SOR/SOC * update generated doc * add server-side unit test for `savedObjectsTagging` plugin * move tagging API types to its own file * add savedObjectsTaggingOss mock * add tags_cache tests * add tests for client-side tag client * extract uiApi to distinct files * various API improvements * add more tests * add link between tag and so management sections + add connection counts * add base functional test suite for tagging * add more FTR tests * improve feature control func test * update codeowners * update generated doc * fix access to proxy modal * adapt SO save modal to allow to add tag field * add SO decorator registry and tag implementation * add unit tests for SO tag decorator * add functional tests for visualize integration * add tag SO read permission for vis/dash feature * add RBAC api integ tests * add API integration tests * add test for getTagConnectionsUrl * add SOM test suite * add dashboard integration suite * remove test line * add missing unit tests * improve API types doc * fix create modal save button label * remove console.log * improve doc * self review * add refresh interval for tag cache * improve page object doc * minor cleanup * address review comments * small layout fixes * add initial focus * use lazy accessor for tag request handler context * adapt SOM export and export route to handle references * remove icon from feature config due to master changes * fix SO table tests * update generated docs * sort tags by name in filter dropdown and listing component * wire SO tagging to dashboard save modal * fix types * - add 'create tag' action in tag selector - add notifications on update/create/delete from management - delete modal wording * add description max length validation * remove real-time validation * fix i18n bundle id * update expected size of savedObjectsTagging plugin * use own useIfMounted * update limit again, contract components cannot be lazy loaded atm. * math is hard * remove single usage of lodash for bundle size * add async imports for create/edit modal * add FTR test for 'create tag' action from tag selector * allow 'create new' option to prepopulate name field * extract savedObjectToTag * add advancedSettings read user for security api_integ suite * add audit login for security client wrapper * use import type when possible * wire SO tagging to lens visualization * fix lens jest test * Fix `create tag` option being selected when closing the selector dropdown * add sorting to tag column from getTableColumnDef * address some of restrry comments * rename tag selector's setSelected option to onTagsSelected * fix audit logging even type for saved_object_remove_references * update plugin size limit to current size * adapt maxlength validation wording * remove selection column until we have batch action menu * remove connections link when user lack read privilege to savedObjectManagement * forbid registering multiple SO decorators with the same priority * add so decorator test * extract getTagFindReferences and create API mock * update audit-logging ascidoc * doc nit * throw conflict error if update returns any failure * use refresh=true as default * wording nits * export: rename `references` to `hasReference` * update generated doc * set description max length to 100 * do not initialize tag cache on anonymous pages * split fetchObjectsToExport into two distinct functions * change tag client `delete` call order * tsdoc nits * more nits * add README for oss plugin * add oss plugin start tests * SavedObject.find: rename `references` to `hasReference` * change section description label * remove url prefix constants * last nits and comments * update generated doc # Conflicts: # .github/CODEOWNERS # packages/kbn-optimizer/limits.yml # x-pack/scripts/functional_tests.js * fix FTR mapping files for 7.x
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Implements MVP of the savedObjects tagging feature as defined in #74571
Some screenshots
tags management page
tag creation modal
vis save modal
vis listing
dashboard save modal
dashboard listing
SO management listing
Technical implementation
Object to tag relations
The base implementation of tagging is quite simple. We introduced a new
TagsavedObject type, and assigning tags to saved objects is done by adding areferenceto the tag in any other saved object, using the already existingreferencesfield. Full discussion about the chosen design can be seen in #74571, and more details are available in the discussion starting hereTagging API consumption from OSS
The
savedObjectsTaggingplugin is in xpack. However the tagging API needs to be accessed from various OSS plugins. This is the reason why thesavedObjectsTaggingOssplugin was introduced. This oss plugins allows a provider to register the tagging API, and will then act as a bridge to expose it to oss plugins. As the types also needs to be accessible from OSS, the types definition for theSavedObjectsTaggingAPIare also living in this plugin.Changes
This PR impacts the following parts of the codebase:
Core
findAPI to allow searching for multiple references.hasReferencenow accepts an array, allowing to search for object matching multiple references. A newhasReferenceOperatoralso has been introduced to switch betweenANDandORwhen searching for references (defaults toOR)removeReferencesTo(type, id)API to the SO repository and clientThis API updates all objects referencing the given object to remove the reference, using a
update_by_queryoperation. This is required to remove references to atagwhen deleting it.exporthttp endpoint to allow exporting byreferencesWhen exporting by
types, it is now possible to specify a list of references to use when searching for the root objects to include in the export. This is used by the SO management page to filter by tag when exporting the currently displayed objectsoss plugins
savedObjectsTaggingOss
Introduce the new
savedObjectsTaggingOssthat is a bridge for accessing the xpack'ssavedObjectsTaggingplugin API. The plugin also contains all the typescript types definition for theSavedObjectsTaggingAPI.savedObjectsManagement
Tagscolumn and filterinitialQueryquery parametersavedObjects
The way the client-side
SavedObjectclass was handling references was an issue (see #74571 (comment) for context), as the class makes the assumption that the config-providedinjectReferencesandextractReferencesare the single source of truth for references, which was causing problems with theTagreferences we added.The PR adds a
decoratorregistry to thesavedObjectplugin to allow enhancing theSavedObjectclass with proper separation of concerns.Also, the
optionsprop from theSavedObjectSaveModalcomponent was added toSavedObjectSaveModalOrigin, to allow passing down additionaloptionsthan the ones generated within the origin modal. This is used to pass thetagsselection dropdown to the modal. Eventually, these modals would probably need a global rewrite, but I just did not felt like doing it there.dashboard
tagscolumn and filterstagsselection widgetvisualize
tagscolumn and filters.tagsselection widget for savedVisx-pack plugins
savedObjectsTagging
Introduce the new
savedObjectsTaggingpluginspaces
Update the spaces client wrapper to implements
removeReferencesTosecurity
Update the security client wrapper to implements
removeReferencesToencrypted_saved_objects
Update the encrypted saved object client wrapper to implements
removeReferencesTolens
tagsselection widgetWhat is outside of the scope of the PR
Tags are currently visible in the SO management section
We are currently not doing the distinction between types that are exportable/importable and the types we are displaying on the SO management section. Setting the
tagtype as not exportable would cause issues when exporting objects with references to tag, in addition to making it impossible to import them back.Setting the
tagtype ashiddenwould more even more problem than that, the first being that it would be outside of the security plugin's RBAC.We need to improve the
SavedObjectsTypeManagementDefinitiontype to allow to define types that are exportable, while not being visible/searchable on the SO management section. However as the PR is already quite big, I would rather handle that in a follow-up.Tag components / widgets may not be all in final state
@elastic/kibana-core-ui as already discussed sync on slack, some components may not be in final design state. Ideally, this should not block the PR to be merged, and improvements would instead be performed in follow-ups by you team. I will still, of course, handle anything that is judged blocker to merge the current PR.
The batch action menu is missing in the So tagging management listing table
Waiting for elastic/eui#4095 to be delivered in next EUI update to be able to add the menu. It can be done as a follow-up, to avoid blocking the feature just for that (will be done before 7.11)
Release Note
Categorize and organize your dashboards, visualizations and other saved objects with tags. Tags can be centrally managed in Stack Management and created in-line across Kibana.