Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update string field mappings to text/keyword #6409

Closed
Bargs opened this issue Mar 3, 2016 · 7 comments
Closed

Update string field mappings to text/keyword #6409

Bargs opened this issue Mar 3, 2016 · 7 comments
Assignees
@jbudz
Labels
blocker Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc Team:Operations Team label for Operations Team v5.6.0

Comments

@Bargs
Copy link
Contributor

Bargs commented Mar 3, 2016 •
edited by epixa
Loading

Blocked by #6404

Elasticsearch deprecated the string field type in version 5.0 and removed it in 6.0 (master).

elastic/elasticsearch#16877

We need to update all Kibana index mappings to use text/keyword instead of string, and we need an upgrade path for existing installs.
@Bargs Bargs added the P1 label Mar 3, 2016
@Bargs Bargs self-assigned this Mar 3, 2016
@Bargs Bargs mentioned this issue Mar 4, 2016
Closed
@epixa
Copy link
Contributor

epixa commented Aug 19, 2016

@Bargs Is this still a blocker for 5.0?

@Bargs
Copy link
Contributor Author

Bargs commented Aug 19, 2016 •
edited
Loading

These are the details based on the ES breaking change doc:

  • string fields on pre-5.0 indices will function as before.
  • New string fields can be added to pre-5.0 indices as before.
  • text and keyword fields can also be added to pre-5.0 indices.
  • When adding a string field to a new index, the field mapping will be rewritten as a text or keyword field if possible, otherwise an exception will be thrown. Certain configurations that were possible with string fields are no longer possible with text/keyword fields such as enabling term_vectors on a not-analyzed keyword field.

So based on those bullets, I don't think Kibana will straight up break if we don't update our mappings. In that sense, it's not a blocker. But with string being deprecated, it seems silly to continue to use them and rely on poorly specified behavior (bullet 4) with new indices.

@Bargs
Copy link
Contributor Author

Bargs commented Sep 16, 2016

@epixa it's all coming back now. The main issue with changing our string mappings to text/keyword is that if a user created some saved object types, but not others, the new mappings will fail. For example:

Say a user saved a Vis in 4.x, but never created a saved search. The description field in the vis mapping is a string. If the user tries to create a saved search in 5.0, and its description field is mapped as text it will conflict with description in the vis mapping, since both types exist in the same index. Our proposed solution in the past was to version the .kibana index. That's a pretty big change to make at this stage.

My feeling is that we've been using the existing mapping definitions without issue through all the alphas, so this isn't really a blocker. It might actually be easier to update the .kibana mappings when strings are removed entirely because there will be fewer states the user's mappings could be in, giving us fewer scenarios to worry about.

@Bargs Bargs mentioned this issue Sep 16, 2016
Closed
4 tasks
@epixa epixa added v6.0.0 and removed v5.0.0 labels Sep 20, 2016
@Bargs
Copy link
Contributor Author

Bargs commented Sep 20, 2016

Based on what I said in the above comment @epixa and I decided to leave the string mappings as is for now. To solve this once and for all we'll need to provide a process to seamlessly re-index the user's .kibana index. When ES removes strings completely I imagine users will need to reindex their data as well, so perhaps we can handle the user data and .kibana index via the same mechanism.

@Bargs
Copy link
Contributor Author

Bargs commented Nov 3, 2016

Closing in favor of #6404 since this can't really be done without it.

@Bargs Bargs closed this as completed Nov 3, 2016
@Bargs Bargs mentioned this issue Nov 3, 2016
Closed
@epixa
Copy link
Contributor

epixa commented Nov 30, 2016

I don't think we should conflate the two. #6404 is a blocker for this, but it should be done without also introducing all the string/keyword changes.

@epixa epixa reopened this Nov 30, 2016
@epixa epixa added the blocked label Nov 30, 2016
@epixa epixa unassigned Bargs Nov 30, 2016
@epixa epixa added Team:Operations Team label for Operations Team Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc labels Nov 30, 2016
This was referenced Nov 30, 2016
Closed
Merged
@LeeDr LeeDr mentioned this issue Nov 30, 2016
Closed
@epixa epixa added v5.4.0 and removed blocked labels Jan 18, 2017
@jbudz jbudz self-assigned this Apr 5, 2017
@epixa epixa removed the P1 label Apr 25, 2017
@epixa epixa added v5.6.0 and removed v5.5.0 labels Jun 13, 2017
@epixa
Copy link
Contributor

epixa commented Jul 12, 2017

I'm going to close this out as it is handled by the new change to single-typed indices

@epixa epixa closed this as completed Jul 12, 2017
1Copenut added a commit that referenced this issue Dec 2, 2022
@1Copenut @cee-chen
Bumping EUI to v70.4.0 (#145888)
7376d9c 
`[email protected]` ⏩ `[email protected]`

- "Fixed EuiButtonGroup firing onChange twice" required changing some
tests from `click` to `change`
___

## [`70.4.0`](https://github.com/elastic/eui/tree/v70.4.0)

- Updated `EuiTourStep.footerAction` type to accept `ReactNode[]`
([#6384](elastic/eui#6384))
- Vertically aligned all footer content so that `euiTourStepIndicator`
is always centered ([#6384](elastic/eui#6384))
- Added `filterInCircle` glyph to `EuiIcon`
([#6385](elastic/eui#6385))
- Added `color` prop to `EuiBeacon`
([#6420](elastic/eui#6420))
- Added the `euiMaxBreakpoint` and `euiMinBreakpoint` CSS-in-JS
utilities for creating min/max-width media queries
([#6431](elastic/eui#6431))

**Bug fixes**

- Restores the previous match operator behaviour when the query value is
split into multiple terms after analysis.
([#6409](elastic/eui#6409))
- Fixed missing slide-in animation on `EuiCollapsibleNav`s and left-side
`EuiFlyout`s ([#6422](elastic/eui#6422))
- Fix bug in `EuiCard` where footer were not aligned to the bottom of
the card ([#6424](elastic/eui#6424))
- Fixed multiple component media queries for consumers with custom theme
breakpoints ([#6431](elastic/eui#6431))

## [`70.3.0`](https://github.com/elastic/eui/tree/v70.3.0)

- `EuiSearchBar` now automatically wraps special characters not used by
query syntax in quotes
([#6356](elastic/eui#6356))
- Added `alignment` prop to `EuiBetaBadge`
([#6361](elastic/eui#6361))
- `EuiButton` now accepts `minWidth={false}`
([#6373](elastic/eui#6373))

**Bug fixes**

- Fixed `EuiPageTemplate` not correctly passing the `component` prop to
the inner main content wrapper.
([#6352](elastic/eui#6352))
- `EuiSkipLink` now correctly calls `onClick` even when
`fallbackDestination` is invalid
([#6355](elastic/eui#6355))
- Permanently fixed `EuiModal` to not cause scroll-jumping issues on
modal open ([#6360](elastic/eui#6360))
- Re-fixed `EuiPageSection` not correctly merging `contentProps.css`
([#6365](elastic/eui#6365))
- Fixed `EuiTab` not defaulting to size `m`
([#6366](elastic/eui#6366))
- Fixed the shadow sizes of `.eui-yScrollWithShadows` and
`.eui-xScrollWithShadows`
([#6374](elastic/eui#6374))
- Fixed bug in `EuiCard` where the inner content in vertical cards was
not growing 100% in width
([#6377](elastic/eui#6377))
- Fixed incorrect margins in `EuiSuperDatePicker` caused by `EuiFlex`
CSS gap change ([#6380](elastic/eui#6380))
- Fixed visual bug in nested `EuiFlexGroup`s, where the parent
`EuiFlexGroup` is responsive but a child `EuiFlexGroup` is not
([#6381](elastic/eui#6381))

**CSS-in-JS conversions**

- Converted `EuiModal` to Emotion
([#6321](elastic/eui#6321))

**Fixes**

- `EuiButton` no longer outputs unnecessary inline styles for
`minWidth={0}` or `minWidth={false}`
([#6373](elastic/eui#6373))
- `EuiFacetButton` no longer reports type issues when passing props
accepted by `EuiButton`
([#6373](elastic/eui#6373))

Co-authored-by: Constance Chen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jbudz jbudz

Labels
blocker Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc Team:Operations Team label for Operations Team v5.6.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update string field mappings to text/keyword Bargs/kibana
3 participants
@epixa @jbudz @Bargs