Add testcase to check for related_integrations based on index #4096

shashank-elastic · 2024-09-23T15:18:18Z

Pull Request

Issue link(s): #4046

Summary - What I changed

Added a test case to check for related_integrations based on index
Refactored declaration of ignore_ids in definitions.py
Added ignore_indexes and required_integrations_map in definitions.py needed for the new testcase
Updated right integration metadata for failing rules.

How To Test

Unit Test case should pass.
The new test case has identified rules that needs update

Failing Rules

>           self.fail(err_msg + '\n'.join(failures))
E           AssertionError: 
E                           The following rules have missing or invalid integrations tags.
E                           Try updating the integrations manifest file:
E                               - `python -m detection_rules dev integrations build-manifests`
E           
E                           c371e9fc-6a10-11ef-a0ac-f661ea17fbcc - AWS SSM `SendCommand` with Run Shell Command Parameters -> Missing integrations: auditd_manager
E           6e1a2cc4-d260-11ed-8829-f661ea17fbcc - First Time Seen Commonly Abused Remote Access Tool Execution -> Missing integrations: system
E           76fd43b7-3480-4dd9-8ad7-8bd36bfad92f - Potential Remote Desktop Tunneling Detected -> Missing integrations: system
E           78de1aeb-5225-4067-b8cc-f4a1de8a8546 - Suspicious ScreenConnect Client Child Process -> Missing integrations: system
E           0b96dfd8-5b8c-4485-9a1c-69ff7839786a - Attempt to Establish VScode Remote Tunnel -> Missing integrations: system
E           00140285-b827-4aee-aa09-8113f58a08f3 - Potential Credential Access via Windows Utilities -> Missing integrations: system
E           5c6f4c58-b381-452a-8976-f1b1c6aa0def - FirstTime Seen Account Performing DCSync -> Missing integrations: system
E           54c3d186-0461-4dc3-9b33-2dc5c7473936 - Network Logon Provider Registry Modification -> Missing integrations: windows
E           4682fd2c-cfae-47ed-a543-9bed37657aa6 - Potential Local NTLM Relay via HTTP -> Missing integrations: system
E           be8afaed-4bcd-4e0a-b5f9-5562003dde81 - Searching for Saved Credentials via VaultCmd -> Missing integrations: system
E           d117cbb4-7d56-41b4-b999-bdf8c25648a0 - Symbolic Link to Shadow Copy Created -> Missing integrations: system
E           a16612dd-b30e-4d41-86a0-ebe70974ec00 - Potential LSASS Clone Creation via PssCaptureSnapShot -> Missing integrations: system
E           56557cde-d923-4b88-adee-c61b3f3b5dc3 - Windows CryptoAPI Spoofing Vulnerability (CVE-2020-0601 - CurveBall) -> Missing integrations: system
E           2ffa1f1e-b6db-47fa-994b-1512743847eb - Windows Defender Disabled via Registry Modification -> Missing integrations: windows
E           c8cccb06-faf2-4cd5-886e-2c9636cfcb87 - Disabling Windows Defender Security Settings via PowerShell -> Missing integrations: system
E           201200f1-a99b-43fb-88ed-f65a45c4972c - Suspicious .NET Code Compilation -> Missing integrations: system
E           8b4f0816-6a65-4630-86a6-c21c179c0d09 - Enable Host Network Discovery via Netsh -> Missing integrations: system
E           c5dc3223-13a2-44a2-946c-e9dc0aa0449c - Microsoft Build Engine Started by an Office Application -> Missing integrations: system
E           9d110cb3-5f4b-4c9a-b9f5-53f0a1707ae3 - Microsoft Build Engine Started by a System Process -> Missing integrations: system
E           ebfe1448-7fac-4d59-acea-181bd89b1f7f - Process Execution from an Unusual Directory -> Missing integrations: system
E           b41a13c6-ba45-4bab-a534-df53d0cfed6a - Suspicious Endpoint Security Parent Process -> Missing integrations: system
E           32c5cf9c-2ef8-4e87-819e-5ccb7cd18b14 - Program Files Directory Masquerading -> Missing integrations: system
E           f2c7b914-eda3-40c2-96ac-d23ef91776ca - SIP Provider Modification -> Missing integrations: windows
E           97aba1ef-6034-4bd3-8c1a-1e0996b27afa - Suspicious Zoom Child Process -> Missing integrations: system
E           de9bd7e0-49e9-4e92-a64d-53ade2e66af1 - Unusual Child Process from a System Virtual Process -> Missing integrations: system
E           06dceabf-adca-48af-ac79-ffdf4c3b1e9a - Potential Evasion via Filter Manager -> Missing integrations: system
E           94a401ba-4fa2-455c-b7ae-b6e037afc0b7 - Group Policy Discovery via Microsoft GPResult Utility -> Missing integrations: system
E           0c7ca5c2-728d-4ad9-b1c5-bbba83ecb1f4 - Peripheral Device Discovery -> Missing integrations: system
E           1a6075b0-7479-450e-8fe7-b8b8438ac570 - Execution of COM object via Xwizard -> Missing integrations: system
E           53a26770-9cbd-40c5-8b57-61d01a325e14 - Suspicious PDF Reader Child Process -> Missing integrations: system
E           e3343ab9-4245-4715-b344-e11c56b0a47f - Process Activity via Compiled HTML File -> Missing integrations: system
E           7e23dfef-da2c-4d64-b11d-5f285b638853 - Microsoft Management Console File from Unusual Path -> Missing integrations: system
E           69c251fb-a5d6-4035-b5ec-40438bd829ff - Modification of Boot Configuration -> Missing integrations: system
E           035889c4-2686-4583-a7df-67f89c292f2c - High Number of Process and/or Service Terminations -> Missing integrations: system
E           dc9c1f74-dac3-48e3-b47f-eb79db358f57 - Volume Shadow Copy Deletion via WMIC -> Missing integrations: system
E           3d00feab-e203-4acc-a463-c3e15b7e9a73 - ScreenConnect Server Spawning Suspicious Processes -> Missing integrations: system
E           ddab1f5f-7089-44f5-9fda-de5b11322e77 - NullSessionPipe Registry Modification -> Missing integrations: windows
E           4fe9d835-40e1-452d-8230-17c147cafad8 - Execution via TSClient Mountpoint -> Missing integrations: system
E           c4210e1c-64f2-4f48-b67e-b5a8ffe3aa14 - Mounting Hidden or WebDav Remote Shares -> Missing integrations: system
E           fa01341d-6662-426b-9d0c-6d81e33c8a9d - Remote File Copy to a Hidden Share -> Missing integrations: system
E           8c37dc0e-e3ac-4c97-8aa0-cf6a9122de45 - Unusual Child Process of dns.exe -> Missing integrations: system
E           6839c821-011d-43bd-bd5b-acff00257226 - Image File Execution Options Injection -> Missing integrations: windows
E           c8b150f0-0164-475b-a75e-74b47800a9ff - Suspicious Startup Shell Folder Modification -> Missing integrations: windows
E           ce64d965-6cb0-466d-b74f-8d2c76f47f05 - New ActiveSyncAllowedDeviceID Added via PowerShell -> Missing integrations: system
E           54902e45-3467-49a4-8abc-529f2c8cfb80 - Uncommon Registry Persistence Change -> Missing integrations: windows
E           403ef0d3-8259-40c9-a5b6-d48354712e49 - Unusual Persistence via Services Registry -> Missing integrations: windows
E           36a8e048-d888-4f61-a8b9-0f9e2e40f317 - Suspicious ImagePath Service Creation -> Missing integrations: windows
E           14ed1aa9-ebfd-4cf9-a463-0ac59ec55204 - Potential Persistence via Time Provider Modification -> Missing integrations: windows
E           1aa9181a-492b-4c01-8b16-fa0735786b2b - User Account Creation -> Missing integrations: system
E           5d676480-9655-4507-adc6-4eec311efff8 - Unsigned DLL loaded by DNS Service -> Missing integrations: windows
E           043d80a3-c49e-43ef-9c72-1088f0c7b278 - Potential Escalation via Vulnerable MSI Repair -> Missing integrations: windows
E           bd7eefee-f671-494e-98df-f01daf9e5f17 - Suspicious Print Spooler Point and Print DLL -> Missing integrations: windows
E           d563aaba-2e72-462b-8658-3e5ea22db3a6 - Privilege Escalation via Windir Environment Variable -> Missing integrations: windows
E           57bccf1d-daf5-4e1a-9049-ff79b5254704 - File Staged in Root Folder of Recycle Bin -> Missing integrations: windows
E           2e311539-cd88-4a85-a301-04f38795007c - Accessing Outlook Data Files -> Missing integrations: system, windows
E           8eec4df1-4b4b-4502-b6c3-c788714604c9 - Bitsadmin Activity -> Missing integrations: system, windows
E           c55badd3-3e61-4292-836f-56209dc8a601 - Attempted Private Key Access -> Missing integrations: system, windows
E           53dedd83-1be7-430f-8026-363256395c8b - Binary Content Copy via Cmd.exe -> Missing integrations: system, windows
E           bd3d058d-5405-4cee-b890-337f09366ba2 - Potential Defense Evasion via CMSTP.exe -> Missing integrations: system, windows
E           98843d35-645e-4e66-9d6a-5049acd96ce1 - Indirect Command Execution via Forfiles/Pcalua -> Missing integrations: system, windows
E           90babaa8-5216-4568-992d-d4a01a105d98 - InstallUtil Activity -> Missing integrations: system, windows
E           808291d3-e918-4a3a-86cd-73052a0c9bdc - Suspicious Troubleshooting Pack Cabinet Execution -> Missing integrations: system, windows
E           f243fe39-83a4-46f3-a3b6-707557a102df - Service Path Modification -> Missing integrations: windows
E           c5677997-f75b-4cda-b830-a75920514096 - Service Path Modification via sc.exe -> Missing integrations: system, windows
E           708c9d92-22a3-4fe0-b6b9-1f861c55502d - Suspicious Execution via MSIEXEC -> Missing integrations: windows
E           1f460f12-a3cf-4105-9ebb-f788cc63f365 - Unusual Process Execution on WBEM Path -> Missing integrations: system, windows
E           d68e95ad-1c82-4074-a12a-125fe10ac8ba - System Information Discovery via Windows Command Shell -> Missing integrations: system
E           4982ac3e-d0ee-4818-b95d-d9522d689259 - Process Discovery Using Built-in Tools -> Missing integrations: system, windows
E           6ea55c81-e2ba-42f2-a134-bccf857ba922 - Security Software Discovery using WMIC -> Missing integrations: system
E           e0881d20-54ac-457f-8733-fe0bc5d44c55 - System Service Discovery through built-in Windows Utilities -> Missing integrations: system
E           06568a02-af29-4f20-929c-f3af281e41aa - System Time Discovery -> Missing integrations: system
E           51176ed2-2d90-49f2-9f3d-17196428b169 - Windows System Information Discovery -> Missing integrations: system
E           1e6363a6-3af5-41d4-b7ea-d475389c0ceb - Creation of SettingContent-ms Files -> Missing integrations: windows
E           d3551433-782f-4e22-bbea-c816af2d41c6 - WMI WBEMTEST Utility Execution -> Missing integrations: system, windows
E           b483365c-98a8-40c0-92d8-0458ca25058a - At.exe Command Lateral Movement -> Missing integrations: system, windows
E           f59668de-caa0-4b84-94c1-3a1549e1e798 - WMIC Remote Command -> Missing integrations: system, windows

tests/test_all_rules.py:831: AssertionError
=========================== short test summary info ============================
FAILED tests/test_all_rules.py::TestRuleMetadata::test_integration_tag - Asse...
============================== 1 failed in 50.67s ==============================
Finished running tests!

With the new pattern derivation, we had warned on each index kind for determining the integration tag

Dynamic Integration Mapping

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for apm-*-transaction* is ['apm']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for traces-apm* is ['apm']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-endpoint* is ['endpoint']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-system.security* is ['system']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-endpoint.events.* is ['endpoint']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-windows.* is ['windows']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-endpoint.events.network-* is ['endpoint']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-system.security-* is ['system']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-windows.sysmon_operational-* is ['windows']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-auditd_manager.auditd-* is ['auditd_manager']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-endpoint.events.process-* is ['endpoint']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-endpoint.events.file-* is ['endpoint']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-aws.cloudtrail-* is ['aws']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-aws.cloudtrail* is ['aws']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-azure* is ['azure']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-azure.signinlogs-* is ['azure']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-azure.activitylogs-* is ['azure']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-o365* is ['o365']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-cloud_defend.alerts-* is ['cloud_defend']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-cloud_defend* is ['cloud_defend']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-cyberarkpas.audit* is ['cyberarkpas']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-endpoint.alerts-* is ['endpoint']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-fim.event-* is ['fim']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-gcp* is ['gcp']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-github.audit-* is ['github']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-google_workspace* is ['google_workspace']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-kubernetes.* is ['kubernetes']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-o365.audit-* is ['o365']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-okta* is ['okta']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-okta.system* is ['okta']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-system.auth-* is ['system']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-endpoint.events.process* is ['endpoint']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-endpoint.events.file* is ['endpoint']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-endpoint.events.network* is ['endpoint']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-system.syslog-* is ['system']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-jamf_protect* is ['jamf_protect']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-network_traffic.* is ['network_traffic']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-panw.panos* is ['panw']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-panw.* is ['panw']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-sentinel_one_cloud_funnel.* is ['sentinel_one_cloud_funnel']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-m365_defender.event-* is ['m365_defender']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-windows.powershell* is ['windows']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-endpoint.events.library-* is ['endpoint']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-endpoint.events.registry-* is ['endpoint']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-system.* is ['system']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-windows.forwarded* is ['windows']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-endpoint.events.api-* is ['endpoint']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-windows.network-* is ['windows']
    if not index_map:

tests/test_all_rules.py::TestRuleMetadata::test_integration_tag
  /Users/shashankks/elastic_workspace/detection-rules/tests/test_all_rules.py:823: UserWarning: Integration map for logs-endpoint.events.library* is ['endpoint']
    if not index_map:

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

We simulated a failure as well

Failure Check

E           AssertionError: 
E                           The following rules have missing or invalid integrations tags.
E                           Try updating the integrations manifest file:
E                               - `python -m detection_rules dev integrations build-manifests`
E           
E                           c371e9fc-6a10-11ef-a0ac-f661ea17fbcc - AWS SSM `SendCommand` with Run Shell Command Parameters -> Missing integration metadata: auditd_manager

tests/test_all_rules.py:836: AssertionError
=============================================================================== short test summary info ================================================================================
FAILED tests/test_all_rules.py::TestRuleMetadata::test_integration_tag - AssertionError: 
========================================================================== 1 failed, 49 deselected in 54.33s ===========================================================================
(.venv) 
detection-rules on  issue-4046 [$!+] is 📦 v0.1.0 via 🐍 v3.12.5 (.venv) on ☁️  [email protected] took 55s 
❯

Once these rules are tuned, the unit test should pass.

Checklist

Added a label for the type of pr: bug, enhancement, schema, Rule: New, Rule: Deprecation, Rule: Tuning, Hunt: New, or Hunt: Tuning so guidelines can be generated
Added the meta:rapid-merge label if planning to merge within 24 hours
Secret and sensitive material has been managed correctly
Automated testing was updated or added to match the most common scenarios
Documentation and comments were added for features that require explanation

Contributor checklist

Have you signed the contributor license agreement?
Have you followed the contributor guidelines?

protectionsmachine · 2024-09-23T15:18:34Z

shashank-elastic · 2024-09-24T12:59:36Z

Discussion Point from @Mikaayenson

Rule changes with min-stack wont port back and will likely break Unit Test in older protected branches.
Re discuss how this can be merged , probably note those rules and commit on older branches.
Maintenance Window merge

terrancedejesus · 2024-09-25T13:51:59Z

tests/test_all_rules.py

+                    index_map = [key for key, value in
+                                 definitions.INTEGRATION_TO_INDEX_MAP.items() if re.search(value, index)]


The challenge here is that we are checking against TRADE definitions and will need to manage these overtime, plus they are subjective to what we define them as versus actually comparing with the datastreams available in each package per version. I'd investigate if we can pull expected datastream patterns from packages and store along our manifests or schemas, then check against these.

Based on definitions.INTEGRATION_TO_INDEX_MAP, it seems like the key is already in the value, where the key represents what we already have defined in each rule. So can do dynamically define the value to check against by concatenating f"logs-{rule.contents.meta.integration}"

Not exactly this way, because this would again give us a partial data what the authors would have updated. Keeping in the idea of dynamic mapping and datastream patterns, we have a new way to generate this mapping!
Testing is also updated.

shashank-elastic added 2 commits September 23, 2024 20:28

Add testcase to check for related_integrations based on index

0f8b6b5

Refine apm index search

8172997

shashank-elastic requested a review from traut September 23, 2024 15:18

shashank-elastic self-assigned this Sep 23, 2024

shashank-elastic requested review from Mikaayenson, eric-forte-elastic and terrancedejesus as code owners September 23, 2024 15:18

botelastic bot added python Internal python for the repository schema labels Sep 23, 2024

github-actions bot added the backport: auto label Sep 23, 2024

shashank-elastic linked an issue Sep 23, 2024 that may be closed by this pull request

[FR] Unit test to check for related_integrations based on index #4046

Open

Fix failing rules

837d23e

botelastic bot added bbr Building Block Rules Domain: Endpoint OS: Windows windows related rules labels Sep 23, 2024

shashank-elastic requested review from w0rk3r and Samirbous September 23, 2024 16:36

Refactor Definitions declarations

42b4ba8

shashank-elastic added enhancement New feature or request Rule: Tuning tweaking or tuning an existing rule labels Sep 23, 2024

Fix Lint errors

8c2407b

terrancedejesus reviewed Sep 25, 2024

View reviewed changes

Dynamic mapping of integration from index pattern

331f4c6

shashank-elastic requested a review from terrancedejesus September 27, 2024 11:17

shashank-elastic and others added 3 commits September 27, 2024 16:50

Fix Lint errors

c0598a0

Merge branch 'main' into issue-4046

7b10168

Fix updated date

094ac2f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add testcase to check for related_integrations based on index #4096

Add testcase to check for related_integrations based on index #4096

shashank-elastic commented Sep 23, 2024 •

edited

Loading

protectionsmachine commented Sep 23, 2024 •

edited

Loading

shashank-elastic commented Sep 24, 2024

terrancedejesus Sep 25, 2024 •

edited

Loading

terrancedejesus Sep 25, 2024

shashank-elastic Sep 27, 2024

		index_map = [key for key, value in
		definitions.INTEGRATION_TO_INDEX_MAP.items() if re.search(value, index)]

Add testcase to check for related_integrations based on index #4096

Are you sure you want to change the base?

Add testcase to check for related_integrations based on index #4096

Conversation

shashank-elastic commented Sep 23, 2024 • edited Loading

Pull Request

Summary - What I changed

How To Test

Checklist

Contributor checklist

protectionsmachine commented Sep 23, 2024 • edited Loading

Enhancement - Guidelines

Documentation and Context

Code Standards and Practices

Testing

Additional Checks

shashank-elastic commented Sep 24, 2024

terrancedejesus Sep 25, 2024 • edited Loading

Choose a reason for hiding this comment

terrancedejesus Sep 25, 2024

Choose a reason for hiding this comment

shashank-elastic Sep 27, 2024

Choose a reason for hiding this comment

shashank-elastic commented Sep 23, 2024 •

edited

Loading

protectionsmachine commented Sep 23, 2024 •

edited

Loading

terrancedejesus Sep 25, 2024 •

edited

Loading