Port gnome3 like tests to python and udisks2 #72860

worldofpeace · 2019-11-05T21:25:41Z

Motivation for this change

#72828

Things done

Notify maintainers

cc @

jtojnar

It would be nice to have a function for gnome-desktop-testing-runner too.

nixos/tests/fwupd.nix

jtojnar · 2019-11-05T23:21:05Z

nixos/tests/packagekit.nix


    # send a dbus message to activate the service
-    $machine->succeed("dbus-send --system --type=method_call --print-reply --dest=org.freedesktop.PackageKit /org/freedesktop/PackageKit org.freedesktop.DBus.Introspectable.Introspect");
+    machine.succeed("dbus-send --system --type=method_call --print-reply --dest=org.freedesktop.PackageKit /org/freedesktop/PackageKit org.freedesktop.DBus.Introspectable.Introspect")


Was the dbus API added or will it come later?

Do you mean in the testing driver? I don't think so, but I have some great ideas of all the things that should be added there. Currently they've tried to go for a 1:1 port from the perl one.

nixos/tests/packagekit.nix

nixos/tests/udisks2.nix

worldofpeace · 2019-11-05T23:47:48Z

It would be nice to have a function for gnome-desktop-testing-runner too.

I can make it similar to #34987 (comment) but maybe makeGnomeInstalledTest and have them in one flat file like what I did for flashback

https://github.com/NixOS/nixpkgs/pull/71212/files#diff-f7f7b8188e959c78a48bd33b4b7723d9R78

(description is Make NixOS test for gnome-desktop-testing-runner using software)

worldofpeace · 2019-11-06T01:01:17Z

@jtojnar I've added a gnome-installed-tests.nix with the function and the tests.
That does make all the tests under nixosTests.gnome-installed-tests.*.

Would you prefer having everything in one file like I've done, or perhaps the colord = makeGnomeInstalledTests could be split out?

jtojnar · 2019-11-06T01:42:30Z

I think one file per test is still a good idea, some of them are slightly more complicated, recall also python-packages.nix. The really trivial ones could perhaps be created directly in all-tests.nix.

I am also not sure about the GnomeInstalledTests name. While it originated in GNOME project, it spread to various freedesktop projects as well.

worldofpeace · 2019-11-06T01:48:45Z

I think one file per test is still a good idea, some of them are slightly more complicated, recall also python-packages.nix. The really trivial ones could perhaps be created directly in all-tests.nix.

I am also not sure about the GnomeInstalledTests name. While it originated in GNOME project, it spread to various freedesktop projects as well.

I just wanted the function name to express that it's just for projects that use gnome-desktop-testing.

I will change the structure though, python-packages.nix was kinda a mess and all-tests.nix is structured that way for a similar reason.

FRidh · 2019-11-06T12:07:39Z

I am a bit confused by the moving of the tests. In the package set, I thought the plan was to get rid of the gnome3 namespace. Why not with tests then?

jtojnar · 2019-11-06T12:59:21Z

gnome-desktop-testing is just a single implementation of the test runner but the installed spec itself is not specific to GNOME. There is at least ibus-desktop-testing-runner. I plan to add that in #71442

worldofpeace · 2019-11-06T17:20:14Z

From IRC:

@worldofpeace
jtojnar: what are the requested changes in #72860 (comment)? I just wanted a good overview of the installed tests specifically, and having them under an attr nixosTests.gnome-installed-tests.* just made that easy. No relation to the gnome3 attr deprecation

@jtojnar
worldofpeace I would just use nixosTests.installed-tests, and maybe pester upstream to drop gnome from its name when I have some time

@worldofpeace
jtojnar: did that 👍️

nixos/tests/udisks2.nix

jtojnar · 2019-11-06T17:33:43Z

nixos/tests/udisks2.nix


-      $machine->succeed("udisksctl info -b /dev/vda >&2");
-      $machine->fail("udisksctl info -b /dev/sda1");
+      with lzma.open(


Why space this around, "${stick}" is much shorter than the line below.

black fails on line length, as this becomes a nix store path so it demands I format it this way. We should probably make that more relaxed. Similarly I've had to format makeInstalledTest testscript injection specifically with concatenating multi-line strings so it doesn't error out on it not being pretty.

nixos/tests/installed-tests/gdk-pixbuf.nix

nixos/tests/installed-tests/gnome-photos.nix

nixos/tests/installed-tests/default.nix

jtojnar · 2019-11-06T18:56:41Z

nixos/tests/installed-tests/gnome-photos.nix

+  };
+
+  extraTestScript = ''
+    # dogtail needs accessibility enabled


In the future, we should probably have a module for this. Depends on #54150

nixos/tests/installed-tests/ostree.nix

nixos/tests/installed-tests/default.nix

jtojnar · 2019-11-06T19:03:32Z

nixos/tests/installed-tests/default.nix

+    , testConfig ? {}      # Config to inject into machine
+    , extraTestScript ? "" # Test script snippet to inject before gnome-desktop-testing-runnner begins
+    , withX11 ? false      # Does test need X11?
+    , testRunnerFlags ? "" # Extra flags to pass to gnome-desktop-testing-runner


Maybe we should make this an array in line with structuredAttrs proposal. I can imagine python test driver to accept list of args like Python’s subprocess module does.

nixos/tests/installed-tests/default.nix

nixos/tests/installed-tests/gnome-photos.nix

worldofpeace · 2019-11-06T22:56:08Z

@GrahamcOfBorg test udisks2 fontconfig-default-fonts packagekit

The test script is also ported to python.

optionalPackages are deprecated.

ofborg bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Nov 5, 2019

worldofpeace requested review from jtojnar and flokli November 5, 2019 22:01

jtojnar reviewed Nov 5, 2019

View reviewed changes

nixosTests.fontconfig-default-fonts: port to python

5efe49c

worldofpeace force-pushed the port-gnome3y-tests branch from 88244d9 to 3be01be Compare November 6, 2019 00:56

worldofpeace force-pushed the port-gnome3y-tests branch from 3be01be to 2aaf62a Compare November 6, 2019 03:08

worldofpeace force-pushed the port-gnome3y-tests branch from 2aaf62a to 189ff39 Compare November 6, 2019 17:18

jtojnar reviewed Nov 6, 2019

View reviewed changes

nixos/tests/udisks2.nix Outdated Show resolved Hide resolved

jtojnar reviewed Nov 6, 2019

View reviewed changes

worldofpeace force-pushed the port-gnome3y-tests branch from 189ff39 to e4bf4cd Compare November 6, 2019 17:45

jtojnar reviewed Nov 6, 2019

View reviewed changes

jtojnar mentioned this pull request Nov 6, 2019

gcab: 1.2 -> 1.3 #72525

Merged

worldofpeace force-pushed the port-gnome3y-tests branch from e4bf4cd to 6c1bd19 Compare November 6, 2019 22:06

jtojnar reviewed Nov 6, 2019

View reviewed changes

nixos/tests/installed-tests/gnome-photos.nix Outdated Show resolved Hide resolved

worldofpeace force-pushed the port-gnome3y-tests branch from 6c1bd19 to e237fd2 Compare November 6, 2019 22:23

worldofpeace added 4 commits November 6, 2019 19:18

nixosTests.packagekit: port to python

7d331ea

nixosTests.udisks2: port to python

ceec19f

nixos/tests: add gnome-installed-tests with builder function

eee5986

The test script is also ported to python.

nixosTests.flatpak: fix eval

d7c8ad7

optionalPackages are deprecated.

worldofpeace force-pushed the port-gnome3y-tests branch from e237fd2 to d7c8ad7 Compare November 7, 2019 00:19

worldofpeace merged commit a46c781 into NixOS:master Nov 7, 2019

worldofpeace deleted the port-gnome3y-tests branch November 7, 2019 00:44

worldofpeace mentioned this pull request Nov 7, 2019

Port tests to Python #72935

Merged

10 tasks

tfc mentioned this pull request Nov 7, 2019

nixos/tests: Leftover tasks towards unified Python integration tests #72828

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Port gnome3 like tests to python and udisks2 #72860

Port gnome3 like tests to python and udisks2 #72860

worldofpeace commented Nov 5, 2019 •

edited

Loading

jtojnar left a comment

jtojnar Nov 5, 2019

worldofpeace Nov 5, 2019

worldofpeace commented Nov 5, 2019 •

edited

Loading

worldofpeace commented Nov 6, 2019

jtojnar commented Nov 6, 2019

worldofpeace commented Nov 6, 2019

FRidh commented Nov 6, 2019

jtojnar commented Nov 6, 2019

worldofpeace commented Nov 6, 2019

jtojnar Nov 6, 2019

worldofpeace Nov 6, 2019

jtojnar Nov 6, 2019

jtojnar Nov 6, 2019

worldofpeace commented Nov 6, 2019

Port gnome3 like tests to python and udisks2 #72860

Port gnome3 like tests to python and udisks2 #72860

Conversation

worldofpeace commented Nov 5, 2019 • edited Loading

Motivation for this change

Things done

Notify maintainers

jtojnar left a comment

Choose a reason for hiding this comment

jtojnar Nov 5, 2019

Choose a reason for hiding this comment

worldofpeace Nov 5, 2019

Choose a reason for hiding this comment

worldofpeace commented Nov 5, 2019 • edited Loading

worldofpeace commented Nov 6, 2019

jtojnar commented Nov 6, 2019

worldofpeace commented Nov 6, 2019

FRidh commented Nov 6, 2019

jtojnar commented Nov 6, 2019

worldofpeace commented Nov 6, 2019

jtojnar Nov 6, 2019

Choose a reason for hiding this comment

worldofpeace Nov 6, 2019

Choose a reason for hiding this comment

jtojnar Nov 6, 2019

Choose a reason for hiding this comment

jtojnar Nov 6, 2019

Choose a reason for hiding this comment

worldofpeace commented Nov 6, 2019

worldofpeace commented Nov 5, 2019 •

edited

Loading

worldofpeace commented Nov 5, 2019 •

edited

Loading