Filtering with alert creation (index threshold)

[arisonl]

As an alerting user I want the ability to add filters when creating an alert. This way I can specify/narrow down the scope of my alerting trigger further, based on fields additional to the ones determined in the alerting condition. For example, I want to be able to specify that an alert instance is generated when a threshold on field-X is exceeded, only for those documents where fields Y, Z, W etc. satisfy other user-specified conditions. See for example the metrics alert type, Discover filters etc.

[elasticmachine]

Pinging @elastic/kibana-alerting-services (Team:Alerting Services)

[bvader]

Please add the "Create alert per" feature as well.

Here is a simple but commonly requested use case.

As a user I want to be notified when my log.level="ERROR" count exceed a threshold. I want to be alerted by host.

E.g.
INDEX filebeat-* when count OVER all documents IS ABOVE 10 FOR THE LAST 1 minutes Filter log.level : "ERROR" Create alert per host.name

NOTE: I am unclear how the Group By and Create Alert Per features would interact / are related. As in the example above.

Perhaps Top-N / Group by already handles the Per Alert use case, the filtering is definitely high value.

[pmuellr]

Perhaps Top-N / Group by already handles the Per Alert use case, the filtering is definitely high value.

Yes, I believe the top-n grouping we already support handles the per-alert use case, and there's an issue open to allow multiple "nested" groupings - #66052

Seems like we need a new array of "filter" specs here. You'd pick a term, and a value, do we need a comparator there? In Stephen's example, it would be "log.level" "=" "ERROR". Do all the other comparators for the threshold value apply? Or maybe a subset, like = and !=. One difference is the threshold value is always numeric, would the filter values always be strings?

[pmuellr]

see also: #33931 and #18463

[pmuellr]

We'll also need to know the boolean operators at play here. We could go with ANDing all the filters, but beyond that, and perhaps an "OR all the filters", any combination of AND or OR may require grouping semantics within the filters (x AND (y OR z))

[cpmoore]

Are there plans to implement this soon?
I believe you should be able to implement the filter from the new metrics alerts relatively easily.

[mikecote]

@arisonl I'm trying to understand the scope of this issue. Is this for index threshold alert?

[pmuellr]

per #66046 (comment) , we should look at copying the API/design from metrics, to have a more consistent experience across all our alert types. I'm curious about what types of filtering - numeric comparison, term matching/not matching, text search matching/not matching, etc. Copying the metrics alert will be a good start here, without too much thinking about such things :-)

[pmuellr]

per #66046 (comment)

I'm trying to understand the scope of this issue. Is this for index threshold alert?

My understanding is that this is for the index threshold alert, however talking about the metrics alert brings up a good point.

As we start to see common patterns across alerts - and filters is a great example - we should look to see if we can modularize/standardize as much as we can here for all the alerts. Ideally, metrics and index threshold alerts could reuse some common "filtering for index-based alerts" code - maybe just UI, perhaps some shared schema/validators for those parameters, etc.

[eedugon]

@mikecote : This could be applied to threshold alerts, yes, but even to other types of alerts if delivered in the future.

The goal makes all sense in my view. Besides all the nice functionality that we have now, defining the when, the group (aggregations), the index, etc... it would be great to provide a way (like a where for the SQL analogy) in order to filter out the data before doing the calculations, instead of working always with the entire index. As @arisonl said: only for those documents where fields Y, Z, W etc. satisfy other user-specified conditions.

The fields to filter could be of any type in my view (the most common will always be numeric and text / keyword, but if possible it would be great to not restrict only to those types).

And in an advanced way we could also offer the possibility of including the payload of a query to be included as a filter, like:

        { "term":  { "status": "published" }},
        {
                  "match": {
                    "uri_path": "/api/cancellation"
                  }
          },
          {
             "match": {
               "status_code": "500"
             }
           }

[mikecote]

Thanks. We should be able to make the components developed for index threshold filtering be re-usable by other alert types. From a framework perspective, there is no control over the query / code done within an alert type so the best we can do is provide libraries / components to make things easier.

[Piskous]

Hi @mikecote,
please can you define the release date for this issue?

[mikecote]

Hi @Piskous, thank you very much for reaching out. Unfortunately, we cannot share release dates of future features. As I am sure you appreciate this is aligned with the industry’s practices as roadmaps are dynamic.

[mikecote]

From: #129596 (closed in favour of this issue).

Describe the feature:

The alert rule 'Index Threshold' should have a box like the 'Elastic Search Query' that lets you specify a query that scopes the aggregation being done.

Describe a specific use case for the feature:

I have an index with logs from multiple deployments being dumped in it that all have the field "deployment". I would like to specify different rules for certain deployments while using the 'Index Threshold' alerting rule. There is currently no way to filter your results before Grouping.

[andystroz]

It would be great to have this feature, this type of alert is limited in use until it is implemented.