From efb9038e9e40d06774df415dbf0709fd5db3cd12 Mon Sep 17 00:00:00 2001 From: lcawl Date: Mon, 13 Jul 2020 16:10:44 -0700 Subject: [PATCH 1/2] [DOCS] Adds siem_cloudtrail module --- .../ootb-ml-jobs-siem.asciidoc | 109 ++++++++++++++++++ 1 file changed, 109 insertions(+) diff --git a/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-siem.asciidoc b/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-siem.asciidoc index 66e264f1b..4286323bb 100644 --- a/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-siem.asciidoc +++ b/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-siem.asciidoc @@ -16,6 +16,28 @@ For more details, see the {dfeed} and job definitions in the `siem_*` folders in https://github.com/elastic/kibana/tree/{branch}/x-pack/plugins/ml/server/models/data_recognizer/modules[GitHub]. +high_distinct_count_error_message:: +Looks for a spike in the rate of an error message. These spikes might simply +indicate an impending service failure but they can also be byproducts of +attempted or successful persistence, privilege escalation, defense evasion, +discovery, lateral movement, or collection activity by a threat actor. + +Job details::: + +* Analyzes Cloudtrail logs +* Detects anomalies where the number of distinct values in +the `aws.cloudtrail.error_message` field is unusual +(using the {ml-docs}/ml-count-functions.html#ml-distinct-count[`high_distinct_count` function]). + +Required {beats}::: + +* {filebeat} + +Required ECS fields when not using {beats}::: + +* `source.ip` +* `source.geo.city_name` + linux_anomalous_network_activity_ecs:: + windows_anomalous_network_activity_ecs:: @@ -361,6 +383,93 @@ Required ECS fields when not using {beats}::: * `user_agent.original` * `agent.type` +rare_error_code:: +Looks for unusual errors. Rare and unusual errors might simply indicate an +impending service failure but they can also be byproducts of attempted or +successful persistence, privilege escalation, defence evasion, discovery, +lateral movement, or collection activity by a threat actor. + +Job details::: + +* Analyzes Cloudtrail logs +* Detects `aws.cloudtrail.error_code` values that have never or rarely occurred +before (using the {ml-docs}/ml-rare-functions.html#ml-rare[`rare` function]). + +Required {beats}::: + +* {filebeat} + +Required ECS fields when not using {beats}::: + +* `source.ip` +* `source.geo.city_name` + +rare_method_for_a_city:: +Looks for AWS API calls that--while not inherently suspicious or abnormal--are +sourcing from a geolocation (city) that is unusual. These calls can be the +result of compromised credentials or keys. + +Job details::: + +* Analyzes Cloudtrail logs +* Detects unusually rare `event.action` values compared to other cities (using +the {ml-docs}/ml-rare-functions.html#ml-rare[`rare` function]). + +Required {beats}::: + +* {filebeat} + +Required ECS fields when not using {beats}::: + +* `event.action` +* `source.geo.city_name` +* `source.ip` + +rare_method_for_a_country:: +Looks for AWS API calls that--while not inherently suspicious or abnormal--are +sourcing from a geolocation (country) that is unusual. These calls can be the +result of compromised credentials or keys. + +Job details::: + +* Analyzes Cloudtrail logs +* Detects unusually rare `event.action` values compared to other countries +(using the {ml-docs}/ml-rare-functions.html#ml-rare[`rare` function]). + +Required {beats}::: + +* {filebeat} + +Required ECS fields when not using {beats}::: + +* `event.action` +* `source.geo.country_iso_code` +* `source.ip` +* `source.geo.country_iso_code` + +rare_method_for_a_username:: +Looks for AWS API calls that--while not inherently suspicious or abnormal--are +sourcing from a user context that does not normally call the method. These calls +can be the result of compromised credentials or keys as someone uses a valid +account to persist, move laterally, or exfil data. + +Job details::: + +* Analyzes Cloudtrail logs +* Detects unusually rare `event.action` values compared to other users (using +the {ml-docs}/ml-rare-functions.html#ml-rare[`rare` function]). + +Required {beats}::: + +* {filebeat} + +Required ECS fields when not using {beats}::: + +* `event.action` +* `user.name` +* `source.geo.city_name` +* `source.ip` + rare_process_by_host_linux_ecs:: + rare_process_by_host_windows_ecs:: From 76a0f86f0b1ccf702e1e741f09559abdd76c66c4 Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 15 Jul 2020 14:21:09 -0700 Subject: [PATCH 2/2] [DOCS] Removes unnecessary text --- .../anomaly-detection/ootb-ml-jobs-siem.asciidoc | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-siem.asciidoc b/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-siem.asciidoc index a68853859..b6ac995cc 100644 --- a/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-siem.asciidoc +++ b/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-siem.asciidoc @@ -270,8 +270,7 @@ attempted or successful persistence, privilege escalation, defense evasion, discovery, lateral movement, or collection activity by a threat actor. Job details::: - -* Analyzes CloudTrail logs + * Detects anomalies where the number of distinct values in the `aws.cloudtrail.error_message` field is unusual (using the {ml-docs}/ml-count-functions.html#ml-distinct-count[`high_distinct_count` function]). @@ -282,8 +281,8 @@ Required {beats}::: Required ECS fields when not using {beats}::: -* `source.ip` * `source.geo.city_name` +* `source.ip` rare_error_code:: Looks for unusual errors. Rare and unusual errors might simply indicate an @@ -293,7 +292,6 @@ lateral movement, or collection activity by a threat actor. Job details::: -* Analyzes CloudTrail logs * Detects `aws.cloudtrail.error_code` values that have never or rarely occurred before (using the {ml-docs}/ml-rare-functions.html#ml-rare[`rare` function]). @@ -303,8 +301,8 @@ Required {beats}::: Required ECS fields when not using {beats}::: -* `source.ip` * `source.geo.city_name` +* `source.ip` rare_method_for_a_city:: Looks for AWS API calls that--while not inherently suspicious or abnormal--are @@ -313,7 +311,6 @@ result of compromised credentials or keys. Job details::: -* Analyzes CloudTrail logs * Detects unusually rare `event.action` values compared to other cities (using the {ml-docs}/ml-rare-functions.html#ml-rare[`rare` function]). @@ -334,7 +331,6 @@ result of compromised credentials or keys. Job details::: -* Analyzes CloudTrail logs * Detects unusually rare `event.action` values compared to other countries (using the {ml-docs}/ml-rare-functions.html#ml-rare[`rare` function]). @@ -347,7 +343,6 @@ Required ECS fields when not using {beats}::: * `event.action` * `source.geo.country_iso_code` * `source.ip` -* `source.geo.country_iso_code` rare_method_for_a_username:: Looks for AWS API calls that--while not inherently suspicious or abnormal--are @@ -357,7 +352,6 @@ account to persist, move laterally, or exfil data. Job details::: -* Analyzes CloudTrail logs * Detects unusually rare `event.action` values compared to other users (using the {ml-docs}/ml-rare-functions.html#ml-rare[`rare` function]). @@ -368,9 +362,9 @@ Required {beats}::: Required ECS fields when not using {beats}::: * `event.action` -* `user.name` * `source.geo.city_name` * `source.ip` +* `user.name` // end::security-cloudtrail-jobs[]