[FEATURE] Alerts in Correlations

[riysaxen-amzn]

Is your feature request related to a problem?
Currently, Security Analytics operates its correlation engine to correlate findings generated by detectors based on customer-created correlation rules. However, a significant challenge arises: while the correlation engine actively listens to findings and triggers the correlation generation process, customers lack a mechanism to receive notifications once a correlation is successfully generated.
To address this challenge, we propose implementing a feature to support alert triggers based on correlation rules. These triggers will enable customers to receive notifications when a correlation is generated and matches any of the correlation rules they have created. This document outlines the functionality and implementation details of this feature, aiming to enhance user experience and provide timely alerts regarding correlation activities. Initially, the correlation alert will be rule-based, meaning alerts will not be generated for auto-correlations, i.e., correlated findings generated without any rules.

What solution would you like?

• Implementing a correlation rule-based alert trigger where alerts are triggered within specified time windows defined in correlation rules. Within each rule's time window, only one correlation alert should be generated as soon as the first match occurs. Furthermore, if the same rule is matched again within the time window, the existing alert should be updated rather than generating a new one.
• 2 new APIs for correlationAlerts i.e getCorrelationAlerts and acknowledgeCorrelationAlerts
• A super alert data model at later stages which is generic and abstract. And there can be a monitor-alert which alerting and SAP uses and the correlation-alert that is owned by SAP
  As per the above:

1. A new index named .opensearch-sap-correlations-alerts will be created to store correlation alerts.

2. CorrelationsAlertService will have all the CRUD related operations to the index .opensearch-sap-correlations-alerts

3. When end user creates/updates a correlation rule with an Alert Trigger (optional). In case of a trigger, .opensearch-sap-correlation-rules-config will be updated with a trigger field.

4. The correlation engine is configured to subscribe and monitor Findings, generating correlations whenever a finding matches a predefined correlation rule. Triggers are sourced from the correlation rule configuration, initiating the generation of alerts.

5. In the CorrelationAlertScheduler, since each correlation rule has its own time window, which determines when the rule is active and when alerts can be generated. The startTime marks the beginning of this window, and the endTime marks the end. During this time window, if the rule is triggered multiple times, we only want to send one alert and notification. If the same rule is triggered again within the same time window, we simply update the existing alert instead of creating a new one.

   1. As part of generation of alerts, .opensearch-sap-correlations-alerts will be populated with Alert Trigger info and state of the Alert as ACTIVE
   2. An API of NotificationPlugin to sendNotification will be called to send a notification message to a customer configured channel.

6. When end user switches to Correlations Alerts view, getCorrelationAlerts API at the backend will be called to fetch the list of correlation alerts.

7. End user can acknowledge the alert

What alternatives have you considered?

• Alternative Idea: The idea is to create a document-level monitor when a correlation rule is created, with the monitor responsible for triggering and sending notifications. This approach aimed to avoid redundant work with the proposed designed approach above. The monitor's role would be to monitor correlations and generate alerts and send notifications. Monitors in Alerts can be created with queries resolved from the Correlation rules configured by customer. These queries are run periodically on the source data indices [i.e .opensearch-sap-correlation-history* index] and would trigger alerts based on query criteria. The SAP Correlation and the Alerting Plugin communicate via a transport layer interaction
  This design is discarded due to issues with handling upgrade scenarios and operations load after introducing a monitor to generate alerts.

Do you have any additional context?
Add any other context or screenshots about the feature request here.