chore: decouple unit and integration tests on CI

[mdelapenya]

What does this PR do?

We are keeping Pipeline2.0-consistency across Beats calling the Python system-tests, using OSS metricbeat's as the canonical reference: run unit tests, then Go integration tests, and finally Python integration tests when needed. This change applies to filebeat and metricbeat in both OSS and xpack flavours.

I finally revisited each Beat and made sure all of them run the unit, goInteg and pythonInteg tests in a consistent manner: instead of calling the obscure mage test command, we separate it in three different stages when needed: mage unitTest, mage goIntegTest and mage pythonIntegTest. I've verified one by one that each of those three commands is present in the build for the related Beat, adding or removing them from the CI descriptor (Jenkinsfile.yml) when/if needed, creating the consistent experience of running atomic commands for each test type.

Why is it important?

First is consistency: metricbeat and filebeat are running system-tests, and we want to operate all Beats on CI in the same manner. That's why we are making the descriptors very similar.

And second, separation of concers: we want to run unit > integration > e2e tests in different stages, so that we acknowledge the power of not building/running code that could have been broken after a possible failure in a previous stage.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

• [ ]

Related issues

• Supersedes chore: standardise how unit/integration tests is run on CI, adding a build command to compress the system-tests files #26209

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2021-07-27T10:51:18.218+0000

• Duration: 153 min 37 sec

• Commit: 88e9027

Test stats 🧪

Test	Results
Failed	0
Passed	49331
Skipped	5320
Total	54651

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	49331
Skipped	5320
Total	54651

[andrewkroh]

LGTM. I grepped the code to see if any projects were implementing there own mage test that does something other than unitTest + integTest and I didn't find anything.