feat: Otel pytest plugin

[kuisathaverat]

What does this PR do?

• Reimplementation of our pytest_apm plugin to work with OpenTelemetry

Why is it important?

With this plugin we can report OpenTelemetry data related with the execution of tests implemented with pytest

Test Locally

1. Run the OpenTelemetry Collector in debug mode

git clone [email protected]:kuisathaverat/apm-pipeline-library.git 
cd apm-pipeline-library/resources/scripts/pytest_otel
git checkout pytest_otel
make run-otel-collector

2. In another terminal in the same folder run the sample tests

make test

3. Check the OpenTelemetry Collector console output to see the transactions and span logs.

InstrumentationLibrary Test Suit 
Span #0
    Trace ID       : d2e52557b4865e2f3c19ec9e3f3c779b
    Parent ID      : 
    ID             : 200590773ff3bf30
    Name           : Running test_xfail
    Kind           : SPAN_KIND_INTERNAL
    Start time     : 2021-07-22 17:27:04.595932 +0000 UTC
    End time       : 2021-07-22 17:27:12.602354 +0000 UTC
    Status code    : STATUS_CODE_ERROR
    Status message : 
Attributes:
     -> test.name: STRING(test_xfail)
     -> test.stack_trace: STRING(['Traceback (most recent call last):\n', '  File "/home/myuser/src/apm-pipeline-library/resources/scripts/pytest_otel/.venv/lib/python3.9/site-packages/_pytest/python.py", line 1641, in runtest\n    self.ihook.pytest_pyfunc_call(pyfuncitem=self)\n', '  File "/home/myuser/src/apm-pipeline-library/resources/scripts/pytest_otel/.venv/lib/python3.9/site-packages/pluggy/hooks.py", line 286, in __call__\n    return self._hookexec(self, self.get_hookimpls(), kwargs)\n', '  File "/home/myuser/src/apm-pipeline-library/resources/scripts/pytest_otel/.venv/lib/python3.9/site-packages/pluggy/manager.py", line 93, in _hookexec\n    return self._inner_hookexec(hook, methods, kwargs)\n', '  File "/home/myuser/src/apm-pipeline-library/resources/scripts/pytest_otel/.venv/lib/python3.9/site-packages/pluggy/manager.py", line 84, in <lambda>\n    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(\n', '  File "/home/myuser/src/apm-pipeline-library/resources/scripts/pytest_otel/.venv/lib/python3.9/site-packages/pluggy/callers.py", line 208, in _multicall\n    return outcome.get_result()\n', '  File "/home/myuser/src/apm-pipeline-library/resources/scripts/pytest_otel/.venv/lib/python3.9/site-packages/pluggy/callers.py", line 80, in get_result\n    raise ex[1].with_traceback(ex[2])\n', '  File "/home/myuser/src/apm-pipeline-library/resources/scripts/pytest_otel/.venv/lib/python3.9/site-packages/pluggy/callers.py", line 187, in _multicall\n    res = hook_impl.function(*args)\n', '  File "/home/myuser/src/apm-pipeline-library/resources/scripts/pytest_otel/.venv/lib/python3.9/site-packages/_pytest/python.py", line 183, in pytest_pyfunc_call\n    result = testfunction(**testargs)\n', '  File "/home/myuser/src/apm-pipeline-library/resources/scripts/pytest_otel/test_pytest_otel.py", line 56, in test_xfail\n    assert False\n', 'AssertionError: assert False\n'])
     -> test.error: STRING(assert False)
     -> test.last_value: STRING(assert False)
     -> test.last_type: STRING(<class 'AssertionError'>)
     -> test.xfailed: STRING()
     -> test.outcome: STRING(failed)
Span #1
    Trace ID       : 4b3603762fe8cbb8e829f284f1dd4e93
    Parent ID      : 
    ID             : 1554f2907ae6edb1
    Name           : Test Suit
    Kind           : SPAN_KIND_INTERNAL
    Start time     : 2021-07-22 17:26:38.497736 +0000 UTC
    End time       : 2021-07-22 17:27:12.635571 +0000 UTC
    Status code    : STATUS_CODE_OK
    Status message : 
Attributes:
     -> test.outcome: STRING(passed)

Test with APM Server

OTEL_EXPORTER_OTLP_ENDPOINT=https://apm.example.com OTEL_EXPORTER_OTLP_HEADERS="authorization=Bearer ${APM_TOKEN}" make test

[apmmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2021-07-27T09:48:13.072+0000

• Duration: 11 min 28 sec

• Commit: ec43df6

Test stats 🧪

Test	Results
Failed	0
Passed	1181
Skipped	2
Total	1183

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	1181
Skipped	2
Total	1183

[cyrille-leclerc]

It's very exciting, would it deserve a dedicated github repository pytest-otel that could eventually end up being donated to https://github.com/pytest-dev ?

[kuisathaverat]

It's very exciting, would it deserve a dedicated github repository pytest-otel that could eventually end up being donated to https://github.com/pytest-dev ?

sure let's tested in a couple of our projects tune it and then we would create a new repo for it

[v1v]

Outstanding!!! 🤟