Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deflake X-display tests on GitHub actions #58

Open
chapulina opened this issue May 7, 2020 · 8 comments
Open

Deflake X-display tests on GitHub actions #58

chapulina opened this issue May 7, 2020 · 8 comments
Assignees
Labels
tests Broken or missing tests / testing infra

Comments

@chapulina
Copy link
Contributor

Many ign-gui tests are failing like this on GitHub actions:

qt.qpa.screen: QXcbConnection: Could not connect to display 
Could not connect to any X display.

We should prevent these tests from running when no display is detected. As a reference, this is how Gazebo-classic detects it: https://github.com/osrf/gazebo/blob/6fd426b3949c4ca73fa126cde68f5cc4a59522eb/cmake/CheckDRIDisplay.cmake

@chapulina chapulina added the tests Broken or missing tests / testing infra label May 7, 2020
This was referenced May 7, 2020
@j-rivero
Copy link
Contributor

We should prevent these tests from running when no display is detected

I would prefer to use a different approach that the one in Gazebo and add have an option in the build/test system to indicate that GUI tests needs to be compiled/executed or not. This way is easier to detect failures when the display is not working well since the build will fail instead of silently report success while hiding errors.

@chapulina
Copy link
Contributor Author

X display tests have been fixed on GitHub actions on #98, but they're still flaky. This is the new error:

  [GUI] [Wrn] [Application.cc:649] [QT] could not connect to display :1.0
  [GUI] [Err] [Application.cc:653] [QT] This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
  
  Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.

@chapulina chapulina changed the title Disable X-display tests on GitHub actions Deflake X-display tests on GitHub actions Feb 24, 2021
@chapulina
Copy link
Contributor Author

As a reference, ign-rendering also uses Xvfb and suffers from the same flakiness. Here's an example error message:

   [ RUN      ] Camera/CameraTest.RenderTexture/ogre2
  [Err] [Ogre2RenderEngine.cc:338] Unable to open display: :1.0
  [Err] [Ogre2RenderEngine.cc:735]  Unable to create the rendering window
  [Err] [Ogre2RenderEngine.cc:735]  Unable to create the rendering window
  [Err] [Ogre2RenderEngine.cc:735]  Unable to create the rendering window
  [Err] [Ogre2RenderEngine.cc:735]  Unable to create the rendering window
  [Err] [Ogre2RenderEngine.cc:735]  Unable to create the rendering window
  [Err] [Ogre2RenderEngine.cc:735]  Unable to create the rendering window
  [Err] [Ogre2RenderEngine.cc:735]  Unable to create the rendering window
  [Err] [Ogre2RenderEngine.cc:735]  Unable to create the rendering window
  [Err] [Ogre2RenderEngine.cc:735]  Unable to create the rendering window
  [Err] [Ogre2RenderEngine.cc:735]  Unable to create the rendering window
  [Err] [Ogre2RenderEngine.cc:742] Unable to create the rendering window
  
  
        Start  2: check_UNIT_Camera_TEST
   2/67 Test  #2: check_UNIT_Camera_TEST ................***Failed    0.03 sec

It doesn't always happen to the same test. I've not been able to identify a pattern (i.e. it's always the first test, etc). It looks like the display just can't be found for one test, but then it's found again.

I'm trying out different Xvfb arguments, and also trying to make the failure more verbose. The thing is that this failure isn't very common, so so it's hard to reproduce.


https://github.com/osrf/buildfarmer/issues/161

@chapulina
Copy link
Contributor Author

chapulina commented Jun 25, 2021

Got a new error today that may help debug this a bit more:

   [ RUN      ] Scene3DTest.Events
  [GUI] [Wrn] [Application.cc:657] [QT] The X11 connection broke: Unknown error (code 80)
  XIO:  fatal IO error 2 (No such file or directory) on X server "�0��LV"
        after 520 requests (520 known processed) with 0 events remaining.
  [GUI] [Wrn] [Application.cc:657] [QT] QObject::~QObject: Timers cannot be stopped from another thread
  [GUI] [Wrn] [Application.cc:657] [QT] QObject::~QObject: Timers cannot be stopped from another thread

It's possible that Xvfb is being killed due to high memory usage.

@chapulina
Copy link
Contributor Author

Using EGL may solve this issue.

@chapulina
Copy link
Contributor Author

have an option in the build/test system to indicate that GUI tests needs to be compiled/executed or not

We could revisit this idea and expose a CMake argument that lets us disable the tests which require a display on GitHub actions, but leave them enabled on Jenkins.

Another alternative that @mjcarroll brought up was to try using one of the other platform plugins suggested in one of the errors above:

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, xcb.

@Blast545 Blast545 self-assigned this Jun 14, 2022
@mjcarroll
Copy link
Contributor

Another option potentially? https://github.com/uwerat/qpagbm

@Blast545
Copy link
Contributor

This should be significantly improved after #419. Won't close this issue until the PR is forward ported and we have some confidence that there are no remaining odd failures on github actions.

Specific failures are to be tracked as this one: #421

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tests Broken or missing tests / testing infra
Projects
None yet
Development

No branches or pull requests

4 participants