Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_services_cpp__rmw_cyclonedds_cpp tests are flakey #185

Closed
tfoote opened this issue May 12, 2020 · 1 comment
Closed

test_services_cpp__rmw_cyclonedds_cpp tests are flakey #185

tfoote opened this issue May 12, 2020 · 1 comment
Labels
bug Something isn't working Linux Linux support macOS macOS support tests Failing or missing tests Windows Windows support

Comments

@tfoote
Copy link
Contributor

tfoote commented May 12, 2020

Reviewing the failing tests on CI test_services_cpp__rmw_cyclonedds_cpp is failing regularly on multiple platforms of CI

linux_repeated: https://ci.ros2.org/job/nightly_linux_repeated/1869/testReport/projectroot/test/test_services__rmw_cyclonedds_cpp/history/

Screenshot from 2020-05-12 17-26-06

linux_release: https://ci.ros2.org/job/nightly_linux_release/1540/testReport/(root)/projectroot/test_services_cpp__rmw_cyclonedds_cpp/history/

Linux aarch64 repeated: https://ci.ros2.org/job/nightly_linux-aarch64_repeated/1190/testReport/projectroot/test/test_services__rmw_cyclonedds_cpp/history/

osx_repeated: https://ci.ros2.org/job/nightly_osx_repeated/1951/testReport/projectroot/test/test_services__rmw_cyclonedds_cpp/history/

windows repeated: https://ci.ros2.org/job/nightly_win_rep/1846/testReport/(root)/projectroot/test_services_cpp__rmw_cyclonedds_cpp/history/

Example stack trace: https://ci.ros2.org/job/nightly_osx_repeated/1951/testReport/projectroot/test/test_services__rmw_cyclonedds_cpp/

-- run_test.py: extra environment variables:
 - RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
-- run_test.py: extra environment variables to append:
 - DYLD_LIBRARY_PATH+=/Users/osrf/jenkins-agent/workspace/nightly_osx_repeated/ws/build/rcl
-- run_test.py: invoking following command in '/Users/osrf/jenkins-agent/workspace/nightly_osx_repeated/ws/build/rcl/test':
 - /Users/osrf/jenkins-agent/workspace/nightly_osx_repeated/venv/bin/python3 -m launch_testing.launch_test /Users/osrf/jenkins-agent/workspace/nightly_osx_repeated/ws/build/rcl/test/test/test_services__rmw_cyclonedds_cpp_.py --junit-xml=/Users/osrf/jenkins-agent/workspace/nightly_osx_repeated/ws/build/rcl/test_results/rcl/test_services__rmw_cyclonedds_cpp.xunit.xml --package-name=rcl
[INFO] [launch]: All log files can be found below /Users/osrf/.ros/log/2020-05-12-03-39-21-129212-mini1-86109
[INFO] [launch]: Default logging verbosity is set to INFO
test_services (rcl.TestTwoExecutables)
Test that the executable under test terminates after a finite amount of time. ... [INFO] [service_fixture-1]: process started with pid [86111]
[INFO] [client_fixture-2]: process started with pid [86112]
[INFO] [client_fixture-2]: sending signal 'SIGINT' to process[client_fixture-2]
[INFO] [service_fixture-1]: sending signal 'SIGINT' to process[service_fixture-1]
[ERROR] [client_fixture-2]: process has died [pid 86112, exit code -2, cmd '/Users/osrf/jenkins-agent/workspace/nightly_osx_repeated/ws/build/rcl/test/client_fixture__rmw_cyclonedds_cpp service_fixture'].
[ERROR] [service_fixture-1]: process has died [pid 86111, exit code -2, cmd '/Users/osrf/jenkins-agent/workspace/nightly_osx_repeated/ws/build/rcl/test/service_fixture__rmw_cyclonedds_cpp'].
FAIL

======================================================================
FAIL: test_services (rcl.TestTwoExecutables)
Test that the executable under test terminates after a finite amount of time.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/osrf/jenkins-agent/workspace/nightly_osx_repeated/ws/build/rcl/test/test/test_services__rmw_cyclonedds_cpp_.py", line 42, in test_services
    proc_info.assertWaitForShutdown(process=executable_under_test, timeout=10)
  File "/Users/osrf/jenkins-agent/workspace/nightly_osx_repeated/ws/install/launch_testing/lib/python3.8/site-packages/launch_testing/proc_info_handler.py", line 144, in assertWaitForShutdown
    assert success, "Timed out waiting for process '{}' to finish".format(process)
AssertionError: Timed out waiting for process '<launch.actions.execute_process.ExecuteProcess object at 0x1039075b0>' to finish

----------------------------------------------------------------------
Ran 1 test in 10.003s

FAILED (failures=1)
test_services (rcl.TestTwoExecutablesAfterShutdown)
Test that the executable under test finished cleanly. ... FAIL

======================================================================
FAIL: test_services (rcl.TestTwoExecutablesAfterShutdown)
Test that the executable under test finished cleanly.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/osrf/jenkins-agent/workspace/nightly_osx_repeated/ws/build/rcl/test/test/test_services__rmw_cyclonedds_cpp_.py", line 50, in test_services
    launch_testing.asserts.assertExitCodes(
  File "/Users/osrf/jenkins-agent/workspace/nightly_osx_repeated/ws/install/launch_testing/lib/python3.8/site-packages/launch_testing/asserts/assert_exit_codes.py", line 62, in assertExitCodes
    assert info.returncode in allowable_exit_codes, 'Proc {} exited with code {}'.format(
AssertionError: Proc client_fixture-2 exited with code -2

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (failures=1)
-- run_test.py: return code 1
-- run_test.py: verify result file '/Users/osrf/jenkins-agent/workspace/nightly_osx_repeated/ws/build/rcl/test_results/rcl/test_services__rmw_cyclonedds_cpp.xunit.xml'
tfoote added a commit to ros2/rcl that referenced this issue May 13, 2020
tfoote added a commit to ros2/rcl that referenced this issue May 14, 2020
@ivanpauno ivanpauno added the bug Something isn't working label Nov 3, 2020
@chapulina chapulina added Linux Linux support macOS macOS support tests Failing or missing tests Windows Windows support labels Jan 21, 2021
@sloretz
Copy link
Contributor

sloretz commented Apr 2, 2021

It looks this issue has been solved. The failing test is no longer marked as xfail: ros2/rcl#904

I'll close this for now.

@sloretz sloretz closed this as completed Apr 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Linux Linux support macOS macOS support tests Failing or missing tests Windows Windows support
Projects
None yet
Development

No branches or pull requests

4 participants