Failures in Vulnerability Detection E2E Tests Beta 6

[Rebits]

Description

This issue tracks all known issues responsible for failures in Vulnerability Detection tests. The specified report is utilized to validate the Additional Vulnerability Detection End-to-End tests.

Initial Scans

• test_first_syscollector_scan[vd_disabled_when_agents_registration]
  • Database errors: Broken database during Vulnerability Detector tests wazuh#22847
  • centOS arm64 agent did not trigger any vulnerability in E2E VD tests. After removing and registering the agent again, the agent's vulnerabilities seem to appear. This was not possible to reproduce. Following discussions with @davidjiglesias, further investigation is planned during the next 4.8.0 stage.

---

• test_consistency_initial_scans
  • Different vulnerabilities between vd_enabled_when_agents_registration and vd_disabled_when_agents_registration. Related to the vd_disabled_when_agents_registration error. Following discussions with @davidjiglesias, further investigation is planned during the next 4.8.0 stage.

Installation of a vulnerable package when agents are down

• test_install_vulnerable_package_when_agent_down[install_package]
  • Missing vulnerabilities: https://github.com/wazuh/intelligence-platform/issues/1467
  • Unexpected vulnerabilities:
    • Unexpected CVE-2023-34111 vulnerability: Treatment of vendor field when using NVD data wazuh#23078
    • Missing CVE-2023-4822 in test package metadata: Missing CVE-2023-4822 in grafana metadata package #5320

Installation of a vulnerable package after agents's manager change

• test_change_agent_manager[install_package]
  • Missing vulnerabilities: Missing Nodejs CVE-2022-0778 vulnerability for Windows agent  wazuh#23192
    • Unexpected CVE-2023-34111 vulnerability. Treatment of vendor field when using NVD data wazuh#23078
    • Missing CVE-2023-4822 in test package metadata: Missing CVE-2023-4822 in grafana metadata package #5320
    • Missing expected alerts:
      • Missing Nodejs CVE-2022-0778 vulnerability for Windows agent  wazuh#23192
      • macOS vulnerability alerts are not correctly collected #5321

Basic Vulnerability Detection cases

• test_vulnerability_detector_scans_cases[install_package]
  • Missing vulnerabilities: Missing Nodejs CVE-2022-0778 vulnerability for Windows agent  wazuh#23192
  • Unexpected vulnerabilities:
    • Unexpected CVE-2023-34111 vulnerability. Treatment of vendor field when using NVD data wazuh#23078
    • Missing CVE-2023-4822 in test package metadata: Missing CVE-2023-4822 in grafana metadata package #5320
  • Missing expected alerts:
    • Missing Nodejs CVE-2022-0778 vulnerability for Windows agent  wazuh#23192
    • macOS vulnerability alerts are not correctly collected #5321

---

• test_vulnerability_detector_scans_cases[remove_package]
  • Missing expected mitigated alerts:
    • Missing Nodejs CVE-2022-0778 vulnerability for Windows agent  wazuh#23192
    • macOS vulnerability alerts are not correctly collected #5321

---

• test_vulnerability_detector_scans_cases[upgrade_package_maintain_vulnerability]
  • Duplicated vulnerabilities when changing agent manager wazuh#22867
  • Missing vulnerabilities: Missing Nodejs CVE-2022-0778 vulnerability for Windows agent  wazuh#23192
  • Unexpected vulnerabilities:
    • Unexpected CVE-2023-34111 vulnerability. Treatment of vendor field when using NVD data wazuh#23078
    • Missing CVE-2023-4822 in test package metadata: Missing CVE-2023-4822 in grafana metadata package #5320
  • Missing expected alerts:
    • Missing Nodejs CVE-2022-0778 vulnerability for Windows agent  wazuh#23192
    • macOS vulnerability alerts are not correctly collected #5321
  • Missing expected mitigated alerts:
    • Missing Nodejs CVE-2022-0778 vulnerability for Windows agent  wazuh#23192
    • macOS vulnerability alerts are not correctly collected #5321
    • Upgrade macOS package cases for Vulnerability Scanner E2E are not properly configured #5312
  • Failed setup operations. Due to previous failed checks

---

• test_vulnerability_detector_scans_cases[upgrade_package_add_vulnerability]
  • Duplicated vulnerabilities when changing agent manager wazuh#22867
  • Missing vulnerabilities
    • https://github.com/wazuh/intelligence-platform/issues/1467
  • Unexpected vulnerabilities:
    • Unexpected CVE-2023-34111 vulnerability. Treatment of vendor field when using NVD data wazuh#23078
    • Missing CVE-2023-4822 in test package metadata: Missing CVE-2023-4822 in grafana metadata package #5320
  • Missing expected alerts:
    • Missing Nodejs CVE-2022-0778 vulnerability for Windows agent  wazuh#23192
    • macOS vulnerability alerts are not correctly collected #5321
  • Missing expected mitigated alerts:
    • Missing Nodejs CVE-2022-0778 vulnerability for Windows agent  wazuh#23192
    • macOS vulnerability alerts are not correctly collected #5321
    • Upgrade macOS package cases for Vulnerability Scanner E2E are not properly configured #5312

---

• test_vulnerability_detector_scans_cases[upgrade_package_maintain_add_vulnerability]
  • Duplicated vulnerabilities when changing agent manager wazuh#22867
  • upgrade_package_maintain_add_vulnerability and upgrade_package_add_vulnerability cases use the same packages for macOS agent. #5333
  • Missing vulnerabilities
    • https://github.com/wazuh/intelligence-platform/issues/1467
  • Unexpected vulnerabilities:
    • Unexpected CVE-2023-34111 vulnerability. Treatment of vendor field when using NVD data wazuh#23078
    • Missing CVE-2023-4822 in test package metadata: Missing CVE-2023-4822 in grafana metadata package #5320
  • Missing expected alerts:
    • Missing Nodejs CVE-2022-0778 vulnerability for Windows agent  wazuh#23192
    • macOS vulnerability alerts are not correctly collected #5321
  • Missing expected mitigated alerts:
    • Missing Nodejs CVE-2022-0778 vulnerability for Windows agent  wazuh#23192
    • macOS vulnerability alerts are not correctly collected #5321
    • Upgrade macOS package cases for Vulnerability Scanner E2E are not properly configured #5312

---

• test_vulnerability_detector_scans_cases[upgrade_package_remove_vulnerability]
  • Duplicated vulnerabilities when changing agent manager wazuh#22867
  • Unexpected vulnerabilities:
    • Unexpected CVE-2023-34111 vulnerability. Treatment of vendor field when using NVD data wazuh#23078
  • Missing expected mitigated alerts:
    • macOS vulnerability alerts are not correctly collected #5321
    • Upgrade macOS package cases for Vulnerability Scanner E2E are not properly configured #5312

---

• test_vulnerability_detector_scans_cases[upgrade_package_nonvulnerable_to_nonvulnerable]
  • Duplicated vulnerabilities when changing agent manager wazuh#22867
  • Unexpected vulnerabilities:
    • Unexpected CVE-2023-34111 vulnerability. Treatment of vendor field when using NVD data wazuh#23078

---

• test_vulnerability_detector_scans_cases[upgrade_package_nonvulnerable_to_vulnerable]
  • Duplicated vulnerabilities when changing agent manager wazuh#22867
  • Unexpected vulnerabilities:
    • Unexpected CVE-2023-34111 vulnerability. Treatment of vendor field when using NVD data wazuh#23078
  • Missing expected alerts:
    • macOS vulnerability alerts are not correctly collected #5321
    • Upgrade macOS package cases for Vulnerability Scanner E2E are not properly configured #5312

---

• test_vulnerability_detector_scans_cases[install_package_non_vulnerable]
  • Unexpected vulnerabilities:
    • Unexpected CVE-2023-34111 vulnerability. Treatment of vendor field when using NVD data wazuh#23078

Summary

Product

• https://github.com/wazuh/intelligence-platform/issues/1467
• Broken database during Vulnerability Detector tests wazuh#22847
• Treatment of vendor field when using NVD data wazuh#23078
• Duplicated vulnerabilities when changing agent manager wazuh#22867

Tests

• Missing CVE-2023-4822 in grafana metadata package #5320
• upgrade_package_maintain_add_vulnerability and upgrade_package_add_vulnerability cases use the same packages for macOS agent. #5333
• macOS vulnerability alerts are not correctly collected #5321
• Upgrade macOS package cases for Vulnerability Scanner E2E are not properly configured #5312

Automation

• https://github.com/wazuh/wazuh-jenkins/issues/6445

[MARCOSD4]

LGTM