Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarks to every type of package in build workflow #1468

Open
NathanielRN opened this issue Dec 11, 2020 · 2 comments
Open

Add benchmarks to every type of package in build workflow #1468

NathanielRN opened this issue Dec 11, 2020 · 2 comments
Labels
build & infra Issues related to build & infrastructure. triaged

Comments

@NathanielRN
Copy link
Contributor

In #1443 we add performance tests for the SDK.

The SDK falls under the category core of tests matched in the build workflow. For this reason, when the jq command looks through the entire repo after the workflow for *-benchmark.json files we are guaranteed to find at least ` file. This file is then given to the subsequent Github action to report on benchmarking results.

However, the build workflow also has instrumentation and exporter packages. These don't have performance tests yet so the jq command will fail. Even if we suppress the failure, the subsequent GitHub action will fail if no valid output.json file is created.

A temporary solution was to only run those 2 steps of the workflow if: matrix.package == 'core'. A more long-term solution will be to add benchmarks to instrumentation and exporter.

We know only these packages are required because as long as subsequent tests fit under these categories we won't have to keep adding if statements.

@NathanielRN
Copy link
Contributor Author

Related Contrib Issue: open-telemetry/opentelemetry-python-contrib#249

@github-actions
Copy link

github-actions bot commented Apr 9, 2021

This issue was marked stale due to lack of activity. It will be closed in 30 days.

@lzchen lzchen added triaged build & infra Issues related to build & infrastructure. and removed backlog labels May 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build & infra Issues related to build & infrastructure. triaged
Projects
None yet
Development

No branches or pull requests

2 participants