Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Discrepency in Worker ID info between logs and XML report #145

Open
nickrobinson251 opened this issue Jan 29, 2024 · 1 comment · May be fixed by #171
Open

Discrepency in Worker ID info between logs and XML report #145

nickrobinson251 opened this issue Jan 29, 2024 · 1 comment · May be fixed by #171
Labels
bug Something isn't working

Comments

@nickrobinson251
Copy link
Collaborator

We have seen a single error in the logs that are output while running tests (with runtests(...; nworkers=1, report=true)), like:

Error in testset "Foo" on worker 10469:
Error During Test at /test/foo_test.jl:4
  Got exception outside of a @test
  Worker process aborted (signal=11) running test item "Foo" (run=1)
No Captured Logs for test item "Foo" at test/jet_test.jl:4 on worker 10469

(note worker 10469)

but then in the resulting XML report:

No Captured Logs for test item "Foo" at test/foo_test.jl:4 on worker 10447
┌ Error: Worker(pid=10447, terminated=true, termsignal=11) died running test item "Foo". Recording test error.
└ @ ReTestItems /nix/store/wmnl0wbwlq710087qchdfq2zjxgbm9kp-raicode-pkg-build/packages/ReTestItems/rzEcY/src/ReTestItems.jl:561

(note worker 10447)

We have inconsistent information about which worker process terminated.

The logs are correct and the report is wrong.

Possibly because we reset the workerid on each retry (

ReTestItems.jl/src/ReTestItems.jl

Line 504 in a3388ad

testitem.workerid[] = worker.pid
), but we don't write the XML report til after all tests have run, meaning the workerid for the test-item is out of date (we call print_errors_and_captured_logs which calls _on_worker(ti) and gets incorrect info)
@nickrobinson251 nickrobinson251 added the bug Something isn't working label Jan 29, 2024
@nickrobinson251
Copy link
Collaborator Author

We might want to change TestItem to have a field that tracks all the run-specific information, including (workerid, testset, stats), so that these pieces of data are always kept together, e.g. a runs::Vector{TestRun} field with

mutable struct TestRun
    workerid::Int
    testset::DefaultTestSet
    stats::PerfStats
end

@nickrobinson251 nickrobinson251 linked a pull request Aug 7, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Store the worker id used for each run of a testitem JuliaTesting/ReTestItems.jl
1 participant
@nickrobinson251