[RAC][Security Solution][Detections] Implement a Rule Execution Log abstraction for use in Security Solution

[banderror]

Parent ticket: #101013

Summary

Implement a rule execution log abstraction that would provide a simple api for writing to the log and executing queries, hiding non-important details from the rest of security_solution.

• Use this proposal as a foundation: [Security Solution][Detections] Proposal: building Rule Execution Log on top of Event Log and ECS #94143.
• Define a schema for rule execution events and metrics.
  • if the resulting schema will differ from the one proposed in [Security Solution][Detections] Proposal: building Rule Execution Log on top of Event Log and ECS #94143, please explicitly show the differences in the PR description
• Implement the following methods:
  • several methods for logging specific events and metrics
  • a method for fetching a list of last N events of 1 rule
  • a method for fetching a table of last N events and metrics of M rules, total N*M values; here we should use Elasticsearch aggregations
• Make sure to include the current Kibana space id in the documents when logging events and metrics. Filter by space id when reading the log.
• Make sure to use event.sequence to ensure deterministic ordering in the log.
• Use a separate feature switch for Rule Execution Log to switch between the old (based on custom Saved Objects) and the new (so far, based on rule_registry) implementations. Consider using a constant in the code instead of a feature switch in Kibana config to keep it safer and simpler.
• Consider hiding both the old and the new implementations (and so switching between them as well) under a single abstraction, e.g. integrating RuleStatusService (implementation based on SOs) into the Rule Execution Log client.

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[xcrzx]

Implemented in #103463