Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add template for tests on new platform/distribution environments #3600

Open
sxa opened this issue Jun 14, 2024 · 1 comment
Open

Add template for tests on new platform/distribution environments #3600

sxa opened this issue Jun 14, 2024 · 1 comment
Labels

Comments

@sxa
Copy link
Member

sxa commented Jun 14, 2024

As per https://github.com/adoptium/infrastructure/blob/master/FAQ.md#adding-new-systems we have a processs whereby new types of machines (New OS versions, new distributions etc.) Are run through an AQA_Test_Pipeline run.

It would be good if we had a standard template for logging the results of this e.g.

Key:

  • ✅ - All test cases pass
  • ⚠️ - Test suite had failures, but they are all covered by existing issues
  • ❌ - New unexpected (or untracked) failures which are of concern as they may indicate issues with this combination

Ubuntu 24.04 / JDK17 / aarch64

Test suite Result ✅⚠️
sanity.functional
extended.functional
special.functional ❌ (List of failing cases + link to new issue)
sanity.openjdk ⚠️ (IssueLink1, Issuelink2)
extended.openjdk ❌ (List of:failing cases + link to new issue/s)
sanity.system
extended.system
sanity.perf
extended.perf

The table could be extended with other columns if desired / appropriate e.g.

Ubuntu 24.04

Test suite JDK Arch Result ✅⚠️
sanity.openjdk 17 riscv64
extended.openjdk 17 riscv64 ❌ (New failing issue links)
sanity.openjdk 21 riscv64
extended.openjdk 21 riscv64 ❌ (New failing issue links)
sanity.openjdk 8 riscv64 ⚠️ (IssueLink1, Issuelink2)
extended.openjdk 8 riscv64 ❌ (New failing issue links)
@sxa sxa changed the title Add template for testin new platform/distribution environments Add template for tests on new platform/distribution environments Jun 14, 2024
@sxa
Copy link
Member Author

sxa commented Jun 15, 2024

Another option could be to have the AQA_Test_Pipeline job disaplay a suitable markdown table at the end that can be cut and pasted with the links and job status into the relevant issue.

Noting that if we do this it should be in a defined order so the results across tables are easier to compare. The order in which they are initiated in the AQA_Test_Pipeline log does not always map to the order that jenkins schedules them on the machines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Todo
Development

No branches or pull requests

2 participants