Testing backend

[MuhammadMuradG]

Hi all,
This is a technical question about how or what to do when some backend widget doesn't provide a way to test/check its property or functionality? For example, asking the widget to show itself but the widget doesn't provide a way to check if it's being shown or not, like using a method is_visible().

[freakboy3742]

To clarify - are we asking about functionality you would use in a test harness, or functionality to generally evaluate the state of some window/widget property?

If it's the former: you're looking for the Dummy backend. It's a fully compliant Toga backend, but without any actual GUI elements - but it does have tooling to log and test what the GUI has done. It allows you to test functionality independent of an actual GUI, and it's a lot easier to run in a test harness, because you don't need any GUI to be present.

Toga's core uses the dummy backend to verify that the API works as expected. It's not very well documented at present, but the functionality exists.

However, if you're looking for a way to evaluate widget properties at runtime - it depends on what you're looking to check. Visibility of widgets is managed by the style; so if the style has visibility=hidden (ie mywidget.style.visibility == "hidden") then the widget won't be visible. Non-style widget properties (eg enabled) and window properties will be exposed as properties on the actual widget/window.

[MuhammadMuradG]

I didn't understand clearly what did you mean by "test harness".

Perhaps I was not clear enough, or this is not the right place to ask such question, or I cannot understand you clearly. But I mean, How does test toga widget backend (like Gtk, Cocca and winform) to verify the behavior of the backend in testbed? For example, if I asked Gtk.Window to be shown but Gtk.Window doesn't provide the method get_visible() to get current visibility state, how I can check its current visibility state? Moreover, I need to interacting with the internals of widget, verifying that the implementation is in the correct state as a result of invoking a public API, like what happen in widget probe.

[freakboy3742]

My "test harness" comment was trying to work out whether we were talking about a test suite you would write for your own app, or something that your app is using at runtime. From your follow up comments, it sounds like it's a third case - the testbed app, and the support that a platform backend provides for a specific feature.

The piece you're missing is the tests_backends folder in each backend (e.g., this folder for GTK). The testbed app defines a test suite; that test suite assumes that, at runtime, the deployed app will contain a module named tests_backend that contains a number of "probe" classes - one for each widget, plus one for each notable feature that needs to be tested (like dialogs). The appropriate backend-specific tests_backend module is added by Briefcase, by specifying platform-specific directories as a dependency.

When it runs, the testbed test suite will construct a widget instance (say, button) as a fixture, then create a probe for that instance passing in the widget (e.g, by creating an instance of toga_backends.widgets.button.ButtonProbe, passing in the button instance). The ButtonProbe exposes an API that allows the test case to evaluate runtime properties of the widget; every backend implements all the common API for each of the probe classes.

For example, the testbed test cases need to be able to evaluate whether widgets are visible; the probe API has an is_visible property; and the GTK widget probe base class implements that API by calling get_visible() on the native widget.

In general, the probe API tries to be as "close to the metal" as possible - that is, it will call the native API on the widget to establish ground truth, rather than going through the public widget API (after all - we're trying to evaluate that the underlying widget is doing the right thing).

The probe APIs also implement some methods to "poke" the widgets - so, the since button needs to be pressed, there's a probe method to press the button, so that the testbed can evaluate that the callback is invoked in response to pressing the button. Again, as much as possible, this is done with primitives, triggering native system events - but sometimes that isn't possible (you can't "fake" a press event on iOS, for example); so sometimes, we make do by faking the response to the event, rather than the event itself.

[MuhammadMuradG]

Thanks for details.

Now, our hand on the problem. The problem that I face, in this PR, is in the implementation of Dialog in GTK backend. Gtk4 provide us with the Gtk.AlertDialog that can be used to implement Toga Dialog but it doesn't provide us with a way to be used in probe API to help us test if it's doing the right thing or not. For example, it is just allow us to show the Gtk.AlertDialog using choose(...) method, but it doesn't provide us with a way to check if the widget being shown or not.

So, my question was what is the proper way to deal with this issue?

My suggested solutions are:

• Just check if the choose(...) returned an exception when it's called or not and guarantee, if it's not, it will show the AlertDialog correctly.
• Use another way to implement Dialog in GTK which requires reinvent the wheel, i.e. Create an AlertDialog with an additional method is_visible().

What do you think?

[freakboy3742]

We can only test the features that the underlying toolkit allows us to test. If GTK doesn't provide a way to evaluate if the dialog has been displayed, then there's not much we can do. As long as we can retain code coverage, exercising all possible code paths, that's sufficient. We definitely don't want to get into the game of re-implementing core GTK features - at some level, we have to be able to assume that that "gtk_show_dialog()" will, in fact, show a dialog :-)

That said - one thing that might be worth investigating is whether you can get an indirect signal from the window. The dialog is attached to the window, which suggests there might be some property on the window that has one more child while the dialog is displayed. However, if there aren't any obvious candidates, I wouldn't consider that a blocker.

[MuhammadMuradG]

Good!

That said - one thing that might be worth investigating is whether you can get an indirect signal from the window.

I will check it.