[Security Solution] Re-validate rule exceptions when updating pre-packaged rules

[spong]

In v7.10 some pre-packaged rules were updated to change their type from query to eql (#82214). While the system supports this rule-type change, and user rules can be upgraded without error, one thing that can occur is that any existing exception list could now no longer be valid for the new rule type.

For example, if in 7.9 a user installed the Network Connection via Certutil pre-packaged rule (of type query), then added an exception list item using a value list, once the user upgrades to 7.10 and updates their rule (now of type eql), the value list exception list item on the rule will no longer be valid since EQL rules do not support value lists (#79871).

Another touch point (though less likely) would be if the rule-type was changed to a type that didn't support exceptions at all, like a Machine Learning rule. In this instance the updated ML rule would then reference an exception list, which is not valid for machine learning rules.

To mitigate this, we should add API validation for adding value list exceptions to EQL rules (similar as we did with endpoint exceptions #71791), and ensure this validation is performed when a rule's type is updated.

For the above example, the expected UX once this is in place would be that when the user attempts to update their pre-packaged rules, if there are any validation errors they would then be displayed to the user within an error toast, and so directing them to remove any invalid exception list items and perform the upgrade again. All other rules should still update.

[spong]

In his recent work with adding exception list support to ML Rules (#84006) @marshallmain mentioned adding value-list support to EQL would be a similar LoE -- just needing to determine behavior around sequence EQL queries. Once that effort is completed though, Threshold Rules will be the only rule type to not support exceptions, and so the only remaining touch point for this bug.

[spong]

@jmikell821 @Donnater for 7.11, the below pre-packaged rules have been updated from query type to eql, and so if the user had configured a value-list exception previously, those exception items will not be processed once the rule is updated to an eql rule until this issue is resolved.

Rules updated from `query` type to `eql`

7.11: c25e9c87-95e1-4368-bfab-9fd34cf867ec - Microsoft IIS Connection Strings Decryption -> type: eql, language: eql
7.11: 0564fb9d-90b9-4234-a411-82a546dc1343 - Microsoft IIS Service Account Password Dumped -> type: eql, language: eql
7.11: 7405ddf1-6c8e-41ce-818f-48bea6bcaed8 - Potential Modification of Accessibility Binaries -> type: eql, language: eql
7.11: 5aee924b-6ceb-4633-980e-1bde8cdb40c5 - Potential Secure File Deletion via SDelete Utility -> type: eql, language: eql
7.11: 6ea41894-66c3-4df7-ad6b-2c5074eb3df8 - Potential Windows Error Manager Masquerading -> type: eql, language: eql
7.11: 2e1e835d-01e5-48ca-b9fc-7a61f7f11902 - Renamed AutoIt Scripts Interpreter -> type: eql, language: eql
7.11: 201200f1-a99b-43fb-88ed-f65a45c4972c - Suspicious .NET Code Compilation -> type: eql, language: eql
7.11: b41a13c6-ba45-4bab-a534-df53d0cfed6a - Suspicious Endpoint Security Parent Process -> type: eql, language: eql
7.11: a624863f-a70d-417f-a7d2-7a404638d47f - Suspicious MS Office Child Process -> type: eql, language: eql
7.11: e2f9fdf5-8076-45ad-9427-41e0e03dc9c2 - Suspicious Process Execution via Renamed PsExec Executable -> type: eql, language: eql
7.11: 97aba1ef-6034-4bd3-8c1a-1e0996b27afa - Suspicious Zoom Child Process -> type: eql, language: eql
7.11: 1dcc51f6-ba26-49e7-9ef4-2655abb2361e - UAC Bypass via DiskCleanup Scheduled Task Hijack -> type: eql, language: eql
7.11: 35df0dd8-092d-4a83-88c1-5151a804f31b - Unusual Parent-Child Relationship -> type: eql, language: eql

[marshallmain]

Update on the status of rule types and supported exceptions:

Rule type	Non-value list exceptions	Value list exceptions
KQL	✔️	✔️
Machine Learning	✔️	✔️
Threshold	✔️	❌
EQL	✔️	❌
Indicator Match	✔️	✔️

With the current status, we still have a potential problem with value list exceptions being added to EQL or Threshold rules. Updates to the detection engine that converted value list exception evaluation to use fields instead of _source blocked the implementation of value list exceptions for EQL rules until we start using the newly added fields param in the EQL search API.

Value list exceptions for threshold rules are more difficult, and may not be implemented soon since the nature of threshold aggregations means we don't get all the matching documents back in the search response. This makes it impossible to evaluate each document against the value list exception in the current implementation. Therefore we still have a long term need for providing some kind of validation that prevents (or at least warns) users when value list exceptions are attached to a threshold rule.

As for implementing validation that prevents value list exceptions from being added to Threshold and EQL rules, the main problem is that exception lists exist independent of specific rules, instead, rules reference one or more exception lists by id. In addition, it's possible for multiple rules, even rules of different types, to reference the same exception list. If we prevent value list exceptions from being added to an exception list referenced by any threshold or EQL rule, this leads to potential poor UX if a user wants to, for example, add a value list exception to a list referenced by a KQL rule but the value list is also referenced by a threshold rule. In addition, to prevent this we would have to query for all rules that reference an exception list any time we add an item to the list.

If we're willing to do a migration, a solution would be to separate value list and non-value list exceptions into different exception list types and prevent threshold and EQL rules from referencing the value-list exception lists. This could be done at rule create/update time, which is also simpler than attempting to retrieve all rules that reference an exception list at exception item create time. This migration work would not be achievable for 7.12.

A near term mitigation would be to add a check in Threshold and EQL rules at rule execution time that checks if the exception list(s) the rule uses contain any value list exception items. If they do, then we would write a Warning to the rule execution status, warning users that some exceptions are not being evaluated for the rule. This is a minor change that I think could be included in 7.12.

@spong @peluja1012 Does adding a warning on rule execution if the exception list contains value list exception items that can't be evaluated by the rule sound like a good approach to mitigate this issue in 7.12?

[MadameSheema]

@deepikakeshav-qasource can you please validate this ticket on 7.12.0 BC4? Taking a look to the PR can be useful :)

[ghost]

Hi @MadameSheema

We have validated this issue on 7.12.0 BC4 and Below are the observations:

Build Details:

Version: 7.12.0-BC4
Platform: Production
Commit:99ac38d70e426f589bb61a034c96e602d759cfab
Build:39242
https://staging.elastic.co/7.12.0-336ff10d/summary-7.12.0.html

Observation 1:
Depolyed the 7.10.0 build and add the "Rule Exception" with value list in Elastic rule Ex. "Microsoft IIS Connection Strings Decryption" with custom query. Pass 🟢

Observation 2:
Upgrade the build from 7.10.0 to 7.12.0. Pass 🟢

Observation 3:
"Microsoft IIS Connection Strings Decryption" Elastic Rule with custom query is displaying before update the rule with value list exception. Pass 🟢


Observation 4:
After update the Elastic rules. "Microsoft IIS Connection Strings Decryption" Elastic Rule is update from custom query to EQL query. Pass 🟢

Observation 5:
After update the Elastic rule "Microsoft IIS Connection Strings Decryption". No notification is displayed when EQL rule contains the exception value list. Fail🔴

Please let us know if we are missing something.

Thanks!!

[MadameSheema]

@marshallmain @spong can you please check Deepika's observations? Thanks :)

[spong]

Hey there @deepikakeshav-qasource! 👋

So based off of #92914, it looks like the notification that is displayed is in a partial warning thrown during rule execution. Those will show up not under the exceptions tab, but as a banner at the top of Rule Details (don't believe they're shown in failure history, but I could be incorrect here so good to check there too :).

The message that should be display is:

'Exceptions that use "is in list" or "is not in list" operators are not applied to EQL rules' (source)

There's definitely more we can do for UX here when viewing the exceptions tab (like have an error callout within the exception entry), but the root issue here should be solved when we break each rule out into their own type as either an SO migration will need to be performed (or a delete-and-add back with resources sorta flow).

[ghost]

Hi @spong,

Thank you for provide the information.

We have validated this issue on the 7.12.0 BC4 build and Please find our below observations:

Build Details:

Version:7.12.0 BC4
Platform: Production
Commit:99ac38d70e426f589bb61a034c96e602d759cfab
Build:39242
Artifact Page: https://staging.elastic.co/7.12.0-336ff10d/summary-7.12.0.html

observation: 1
Waring message "Exceptions that use "is in list" or "is not in list" operators are not applied to EQL rules" is displaying for "Elastic Rule" when update from custom query to Eql query.

observation: 2
Waring message "Exceptions that use "is in list" or "is not in list" operators are not applied to EQL rules" is displaying for "Custom Rule" when update the custom Rule through API to Eql Rule.

observation: 3
Waring message "Exceptions that use "is in list" or "is not in list" operators are not applied to Threshold rules" is displaying for "Custom Rule" when update the custom Rule through API to Threshold Rule.

Moreover, No item is displyed in "Failure History"

Please let us known if something more to be checked for this issue. Else we are good to add "QA Validated" to it.

Thanks !!

[spong]

Since fix has been validated for Threshold/EQL rules going to go ahead and close this as we'll be re-visiting what it means to 'update a pre-packaged rule' from the RAC perspective.

[ghost]

Bug Conversion:

Created 01 Test-Case for this Ticket
https://elastic.testrail.io/index.php?/cases/view/76930

Thanks!!