Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Security Solution][Detections] New Rule Execution Log UI #119599

Closed
Tracked by #101014
banderror opened this issue Nov 24, 2021 · 2 comments
Closed
Tracked by #101014

[Security Solution][Detections] New Rule Execution Log UI #119599

banderror opened this issue Nov 24, 2021 · 2 comments
Assignees
@spong
Labels
8.2 candidate considered, but not committed, for 8.2 release Feature:Rule Monitoring Security Solution Detection Rule Monitoring SecuritySolution:QAAssist Part of QA testing process for release Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.2.0

Comments

@banderror
Copy link
Contributor

banderror commented Nov 24, 2021 •
edited
Loading

Parent ticket: #101014

Summary

Implement a basic Rule Execution Log UI. A simple log view with search, basic filters and pagination. No visualized view, no grouping by the rule execution UUID.
@botelastic botelastic bot added the needs-team Issues missing a team label label Nov 24, 2021
@banderror banderror added Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.1.0 Team:Detection Rule Management Security Detection Rule Management Team labels Nov 24, 2021
@botelastic botelastic bot removed the needs-team Issues missing a team label label Nov 24, 2021
@banderror banderror mentioned this issue Nov 24, 2021
Closed
2 tasks
@banderror banderror added the Feature:Rule Monitoring Security Solution Detection Rule Monitoring label Nov 24, 2021
@spong spong self-assigned this Dec 13, 2021
@banderror banderror mentioned this issue Jan 19, 2022
Merged
28 tasks
@spong spong mentioned this issue Feb 1, 2022
Closed
13 tasks
@spong
Copy link
Member

spong commented Feb 1, 2022

Discussed #124198 with team, holding for 8.2 to further iterate on implementation and UX.

@spong spong added v8.2.0 and removed v8.1.0 labels Feb 1, 2022
@banderror banderror added the 8.2 candidate considered, but not committed, for 8.2 release label Feb 15, 2022
@spong spong mentioned this issue Feb 23, 2022
Merged
7 tasks
@banderror banderror added the SecuritySolution:QAAssist Part of QA testing process for release label Mar 10, 2022
spong added a commit that referenced this issue Mar 28, 2022
@spong
[Security Solution][Detections] Adds rule execution log table (#126215)
9bc4c0c 
## Summary

Resolves #119598, #119599, #101014

Test plan ([internal doc](https://docs.google.com/document/d/1-prIUGYaPHiwGA79CgSdw1926lxIPKGWWkYOUD2BM1U/edit#heading=h.womzsfdt6zt8))

Adds `Rule Execution Log` table to Rule Details page:

<p align="center">
  <img width="700" src="https://user-images.githubusercontent.com/2946766/158540840-e9cddb9b-f33d-4b95-86ad-cb3e0a00cf39.gif" />
</p>


### Implementation notes

The useful metrics within `event-log` for a given rule execution are spread between a few different platform (`execute-start`, `execute`) and security (`execution-metrics`, `status-change`) events. In effort to provide consolidated metrics per rule execution (and avoiding a lot of empty cells and mis-matched statuses like in the image below)

<p align="center">
  <img width="700" src="https://user-images.githubusercontent.com/2946766/151933881-2e58f4d7-4cda-4528-9d44-37cb7bd5de9c.png" />
</p>



these rule execution events are aggregated by their `executionId`, and then fields are merged from each different event. This PR was re-worked to take advantage of the new event-log aggregation support added in #126948, and is no longer implemented as an in-memory aggregation server side.

* Due to restrictions around supplying search filters that may match multiple sub-agg buckets and missing data ([see discussion here](https://github.com/elastic/kibana/pull/127339/files#r825240516)), it was decided that we'd disable the search bar for the time being. We have both a near-term (writing single rollup event) and long-term (ES|QL) solution that will allow us to re-enable this functionality.

* Note, since a `terms` agg is used to fetch all execution events, an upper bound must be set. See [this discussion](https://github.com/elastic/kibana/pull/127339/files#r823035420) for more details, but setting this max to `1000` events for the time being, and returning total cardinality of execution events back within `total` to allow the UI to inform the user that they should narrow their search further to better isolate and find possible issues. This should be a be a reasonable constraint for most all rules as a rule executing every 5 minutes, 1000 executions would cover over 3 days of execution time.

<p align="center">
  <img width="700" src="https://user-images.githubusercontent.com/2946766/159045563-966896b4-3cd1-475d-9f0e-c2d300683546.png" />
</p>


The `Filter for alerts` action will be available on all `Succeeded`/`Partial Failure` executions even if there weren't alerts generated until #126210 is merged and we can start returning the alert count, at which point we can programmatically enabled/disable this action based on alert count.



<p align="center">
  <img width="300" src="https://user-images.githubusercontent.com/2946766/159051762-e2f97ba4-4ce1-4f67-8ae1-395e4b191cab.png" />
</p>
@spong
Copy link
Member

spong commented Mar 30, 2022

Resolved in #126215, closing...

@spong spong closed this as completed Mar 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@spong spong

Labels
8.2 candidate considered, but not committed, for 8.2 release Feature:Rule Monitoring Security Solution Detection Rule Monitoring SecuritySolution:QAAssist Part of QA testing process for release Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v8.2.0
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@spong @banderror