scripted fields preview and validate script before saving #20746

nreese · 2018-07-12T20:59:59Z

fixes #19504, #11804

This PR adds the ability to run scripted fields in the script edit view that allows the user to view the results and see if the script works as intended. Users can select additional fields to view in the preview to provide document value context.

And the request generated to preview the results

{
  "query": {
    "match_all": {}
  },
  "script_fields": {
    "kb": {
      "script": {
        "lang": "painless",
        "source": "doc['bytes'].value * 0.001"
      }
    }
  },
  "size": 10,
  "_source": [
    "bytes"
  ]
}

elasticmachine · 2018-07-12T22:13:34Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

* Move previewed results into flyout. Execute script as soon as the tab is selected. * Format error in a danger callout.

…ted_fields_preview

elasticmachine · 2018-07-13T22:21:29Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-07-14T00:24:25Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

cjcenizal · 2018-07-14T04:16:24Z

This also fixes #5088

…ate on save

elasticmachine · 2018-07-14T14:31:31Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-07-14T18:01:21Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-07-14T19:38:11Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

…ted_fields_preview

elasticmachine · 2018-07-16T01:29:28Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

cjcenizal · 2018-07-16T04:28:36Z

This also partially fixes #14546

elasticmachine · 2018-07-16T17:28:46Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-07-17T00:26:12Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

…ted_fields_preview

elasticmachine · 2018-07-17T02:07:15Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

jen-huang

Very nice!! Tested locally and it functions well. Just have a small suggestion and noticed a few console errors:

Otherwise LGTM!

jen-huang · 2018-07-17T22:13:27Z

src/ui/public/field_editor/components/scripting_help/test_script.js

+          <h3>Preview results</h3>
+          <p>
+            Run your script to preview the first 10 results. You can also select some
+            additional fields to include in your results to gain more context.


would be nice to clarify this a little bit. initially, I thought that adding additional fields somehow affected my actual script, but they are only shown as part of the preview.

not sure if there's value, but maybe we can also add a "Show query" link that displays the request, which will include the additional _source fields

How about this:

Run your script to preview the first 10 results. You can also select some additional fields to include in each result to help tell your results apart.

…ted_fields_preview

nreese · 2018-07-17T22:21:46Z

How did you produce those console warnings? I am unable to re-create them

jen-huang · 2018-07-17T22:39:06Z

@nreese Click 'Edit' on an existing field (regular field, not scripted field). You should see the console errors right away. If not, refresh the edit page. I should have tried to replicate with my original review, since it took me a little while to find it again 😄

elasticmachine · 2018-07-17T23:35:24Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

jen-huang · 2018-07-18T16:44:33Z

src/ui/public/field_editor/field_editor.js

-            isVisible={field.scripted && showScriptingHelp}
-            onClose={this.hideScriptingHelp}
-          />
+          {scriptDisabledCallout}


I'd love to see this moved out to a separate renderScriptingPanels() method or similar, which can then check field.scripted and return these components, or null. then this render method can avoid the let vars and use {this.renderScriptingPanels()}

elasticmachine · 2018-07-18T16:47:12Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

…ted_fields_preview

cjcenizal

Looks great! Just had one suggestion. Reviewed code and UI.

cjcenizal · 2018-07-18T17:59:50Z

src/ui/public/field_editor/field_editor.js

+    const { field, hasScriptError } = this.state;
+    const isInvalid = !field.script || !field.script.trim() || hasScriptError;
+    const errorMsg = hasScriptError
+      ? 'Script is invalid. View script preview for details'


You can wrap this in a span with a test subject selector for use in tests:

<span data-test-subj="invalidScriptError">Script is invalid. View script preview for details.</span>

cjcenizal · 2018-07-18T18:00:18Z

test/functional/apps/management/_scripted_fields.js

+      await PageObjects.settings.setScriptedFieldScript(`doc['iHaveNoClosingTick].value`);
+      await PageObjects.settings.clickSaveScriptedField();
+      await retry.try(async () => {
+        const formErrorExists = await find.existsByDisplayedByCssSelector('.euiFormErrorText');


Then we can find the error with the test subject selector instead of the className.

…ser management

nreese · 2018-07-18T18:19:49Z

The last commit just moves the delete button to the right to be like other delete buttons

cjcenizal · 2018-07-18T18:28:04Z

src/ui/public/field_editor/field_editor.js


    return field.scripted ? (
      <EuiFormRow
        label="Script"
-        helpText={(<EuiLink onClick={this.showScriptingHelp}>Scripting help</EuiLink>)}
+        helpText={(


We can remove this and instead have renderScript return the help as a paragraph, to give it a bit more visual prominence:

return field.scripted ? ( <Fragment> <EuiFormRow label="Script" isInvalid={isInvalid} error={isInvalid ? errorMsg : null} > <EuiTextArea value={field.script} data-test-subj="editorFieldScript" onChange={this.onScriptChange} isInvalid={isInvalid} /> </EuiFormRow> <EuiFormRow> <EuiText> Access fields with <code>doc['some_field'].value</code>. You can also{' '} <a onClick={this.showScriptingHelp} data-test-subj="scriptedFieldsHelpLink"> get help with the syntax and preview the results of your script </a>. </EuiText> </EuiFormRow> </Fragment> ) : null;

Unfortunately EuiText makes that link keyboard-inaccessible. Here's new code which will make the link keyboard-accessible.

return field.scripted ? ( <Fragment> <EuiFormRow label="Script" isInvalid={isInvalid} error={isInvalid ? errorMsg : null} > <EuiTextArea value={field.script} data-test-subj="editorFieldScript" onChange={this.onScriptChange} isInvalid={isInvalid} /> </EuiFormRow> <EuiFormRow> <Fragment> <EuiText>Access fields with <code>doc['some_field'].value</code>.</EuiText> <br /> <EuiLink onClick={this.showScriptingHelp} data-test-subj="scriptedFieldsHelpLink"> Get help with the syntax and preview the results of your script. </EuiLink> </Fragment> </EuiFormRow> </Fragment> ) : null;

It does affect the formatting though.

… tests

elasticmachine · 2018-07-18T19:20:05Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

nreese · 2018-07-18T21:33:18Z

jenkins, test this

elasticmachine · 2018-07-18T22:54:27Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

malcolmm83 · 2019-12-23T14:01:43Z

I would love to take advantage of this feature, but the index I'm using includes several log files, and running on the first 10 entries doesn't get to the log file I need. It would be great if I could add a filter so I can test on the right log entries.

nreese · 2019-12-26T14:01:01Z

@malcolmm83 There is a community PR open that adds filtering to scripted fields preview, #44220.

add preview for creating scripted fields

d662d6f

nreese added WIP Work in progress :Management release_note:enhancement v7.0.0 v6.4.0 labels Jul 12, 2018

cjcenizal and others added 2 commits July 13, 2018 14:41

Move previewed results into flyout. (#25)

7597d1b

* Move previewed results into flyout. Execute script as soon as the tab is selected. * Format error in a danger callout.

Merge branch 'master' of https://github.com/elastic/kibana into scrip…

0a4082b

…ted_fields_preview

add functional test for preview

063aa58

move executeScript functionallity into lib so it can be used to valid…

8b64303

…ate on save

validate script on save

4e4191a

add functional test for script validation during save

78a724b

Merge branch 'master' of https://github.com/elastic/kibana into scrip…

e02a0f9

…ted_fields_preview

nreese changed the title ~~scripted fields preview~~ scripted fields preview and validate script before saving Jul 16, 2018

fix jest tests

222bcd5

fix jest tests

cd383e2

nreese added 2 commits July 16, 2018 18:38

Merge branch 'master' of https://github.com/elastic/kibana into scrip…

4f08de9

…ted_fields_preview

only validate script when field is scripted

0e4d2c7

nreese removed the WIP Work in progress label Jul 17, 2018

nreese requested review from jen-huang and cjcenizal July 17, 2018 02:26

nreese added the review label Jul 17, 2018

jen-huang approved these changes Jul 17, 2018

View reviewed changes

Merge branch 'master' of https://github.com/elastic/kibana into scrip…

083ce26

…ted_fields_preview

only add script callouts and help flyout to DOM for scripted fields

24219cf

jen-huang reviewed Jul 18, 2018

View reviewed changes

nreese added 2 commits July 18, 2018 11:37

Merge branch 'master' of https://github.com/elastic/kibana into scrip…

5da925a

…ted_fields_preview

move scripting stuff to its own render method

c71bfce

cjcenizal approved these changes Jul 18, 2018

View reviewed changes

right justify delete button to have same style as delete button for u…

c3af10c

…ser management

cjcenizal reviewed Jul 18, 2018

View reviewed changes

nreese added 2 commits July 18, 2018 12:34

use data-test-subj key to find invalid script error msg in functional…

5639215

… tests

move help flyout link out of formRow help text and into its own row

568f2a6

nreese merged commit 0f5322e into elastic:master Jul 18, 2018

This was referenced Jul 19, 2018

[6.x] scripted fields preview and validate script before saving (#20746) #20959

Merged

Test button for Scripted Fields inside the Editor #11804

Closed

Scripted Fields Lack Validation #5088

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

scripted fields preview and validate script before saving #20746

scripted fields preview and validate script before saving #20746

nreese commented Jul 12, 2018 •

edited

Loading

elasticmachine commented Jul 12, 2018

elasticmachine commented Jul 13, 2018

elasticmachine commented Jul 14, 2018

cjcenizal commented Jul 14, 2018

elasticmachine commented Jul 14, 2018

elasticmachine commented Jul 14, 2018

elasticmachine commented Jul 14, 2018

elasticmachine commented Jul 16, 2018

cjcenizal commented Jul 16, 2018

elasticmachine commented Jul 16, 2018

elasticmachine commented Jul 17, 2018

elasticmachine commented Jul 17, 2018

jen-huang left a comment

jen-huang Jul 17, 2018

cjcenizal Jul 18, 2018

nreese commented Jul 17, 2018

jen-huang commented Jul 17, 2018 •

edited

Loading

elasticmachine commented Jul 17, 2018

jen-huang Jul 18, 2018

elasticmachine commented Jul 18, 2018

cjcenizal left a comment •

edited

Loading

cjcenizal Jul 18, 2018

cjcenizal Jul 18, 2018

nreese commented Jul 18, 2018

cjcenizal Jul 18, 2018

cjcenizal Jul 18, 2018

elasticmachine commented Jul 18, 2018

nreese commented Jul 18, 2018

elasticmachine commented Jul 18, 2018

malcolmm83 commented Dec 23, 2019

nreese commented Dec 26, 2019

scripted fields preview and validate script before saving #20746

scripted fields preview and validate script before saving #20746

Conversation

nreese commented Jul 12, 2018 • edited Loading

elasticmachine commented Jul 12, 2018

💔 Build Failed

elasticmachine commented Jul 13, 2018

💔 Build Failed

elasticmachine commented Jul 14, 2018

💔 Build Failed

cjcenizal commented Jul 14, 2018

elasticmachine commented Jul 14, 2018

💔 Build Failed

elasticmachine commented Jul 14, 2018

💔 Build Failed

elasticmachine commented Jul 14, 2018

💔 Build Failed

elasticmachine commented Jul 16, 2018

💔 Build Failed

cjcenizal commented Jul 16, 2018

elasticmachine commented Jul 16, 2018

💔 Build Failed

elasticmachine commented Jul 17, 2018

💔 Build Failed

elasticmachine commented Jul 17, 2018

💚 Build Succeeded

jen-huang left a comment

Choose a reason for hiding this comment

jen-huang Jul 17, 2018

Choose a reason for hiding this comment

cjcenizal Jul 18, 2018

Choose a reason for hiding this comment

nreese commented Jul 17, 2018

jen-huang commented Jul 17, 2018 • edited Loading

elasticmachine commented Jul 17, 2018

💚 Build Succeeded

jen-huang Jul 18, 2018

Choose a reason for hiding this comment

elasticmachine commented Jul 18, 2018

💔 Build Failed

cjcenizal left a comment • edited Loading

Choose a reason for hiding this comment

cjcenizal Jul 18, 2018

Choose a reason for hiding this comment

cjcenizal Jul 18, 2018

Choose a reason for hiding this comment

nreese commented Jul 18, 2018

cjcenizal Jul 18, 2018

Choose a reason for hiding this comment

cjcenizal Jul 18, 2018

Choose a reason for hiding this comment

elasticmachine commented Jul 18, 2018

💚 Build Succeeded

nreese commented Jul 18, 2018

elasticmachine commented Jul 18, 2018

💚 Build Succeeded

malcolmm83 commented Dec 23, 2019

nreese commented Dec 26, 2019

nreese commented Jul 12, 2018 •

edited

Loading

jen-huang commented Jul 17, 2018 •

edited

Loading

cjcenizal left a comment •

edited

Loading