feat(e2e): add e2e tests for stream disconnect, vhal methods [WD-11812]

[lorumic]

Done

• Determine architecture automatically by querying the /nodes endpoint of AMS.
• Add e2e tests for stream disconnect + done callback.
• Add e2e tests for VHAL: vhalReady callback, getAllVhalPropConfigs, getVhalProperties, setVhalProperties methods.

Until #37 is merged, you can see the test passing here.

Fixes WD-11812

[ajanon]

Just one small question: with this change, we would only be running e2e tests against AAOS 13, right?
Should we run these tests against multiple LXD images versions?
I'm not sure if it's too useful, but I would like to have other opinions.

[lorumic]

Just one small question: with this change, we would only be running e2e tests against AAOS 13, right? Should we run these tests against multiple LXD images versions? I'm not sure if it's too useful, but I would like to have other opinions.

For now, I'm focusing on having a single image downloaded, and running tests on a session created from that one only. This is to make the tests run reasonably fast. If there is an actual need of running on different images, technically we could do that, however I don't know if it's in the scope of the e2e tests, as it seems more about "testing the images" rather than "testing the features of the Anbox streaming SDK end-to-end". However, this is just my point of view, so I'm happy to hear other opinions and change the workflow accordingly. :)

[ajanon]

LGTM then, thanks!

[morphis]

For now, I'm focusing on having a single image downloaded, and running tests on a session created from that one only. This is to make the tests run reasonably fast. If there is an actual need of running on different images, technically we could do that, however I don't know if it's in the scope of the e2e tests, as it seems more about "testing the images" rather than "testing the features of the Anbox streaming SDK end-to-end". However, this is just my point of view, so I'm happy to hear other opinions and change the workflow accordingly. :)

There are quite a few nuances we do not necessarily can test with every image (e.g. I doubt we have virtual keyboard support for AAOS atm). I think you definitely want one per image variant (android, aaos and generic as well once we start officially supporting that).

[morphis]

@lorumic I've merged #37, if you update the PR we should be able to get tests running now.

[lorumic]

@morphis @ajanon

I'm laying the ground for when we'll need to test different features on different images. For this purpose, I tested locally the parallel creation of an AOSP-based app + session and AAOS-based ones. Thanks to the parallel execution, it should not take too much additional time - the test is still running, though, and we'll have to see how it behaves on the runner, since we'd also need to download both images there, unlike locally where they are already downloaded.

If that works well, I will run the join + disconnect stream test on the AOSP session, and the VHAL test (which includes join + disconnect anyway) on the AAOS one. Anything else, I'd like to keep as a future addition so as not to grow too much the scope of this task, which was planned to comprise only what is listed in the "Done" section of the PR description above.

Does that sound reasonable to you?

EDIT: I have pushed the changes already, but for some reason GitHub is not reflecting them on the PR after almost 30 minutes:

Anyway, I ran the job manually and you can compare the difference here:

• Before (AAOS image only)
• After (AOSP and AAOS)

With AOSP + AAOS it takes just 3 minutes more than with AAOS only. Not bad. :)

[morphis]

LGTM

[adglkh]

LGTM

[ajanon]

LGTM

[morphis]

@lorumic I think we're blocked on #43 before we can merge this one?

[lorumic]

@lorumic I think we're blocked on #43 before we can merge this one?

Yes indeed, I was hoping to test the outcome of #43 on this one before merging it.

[lorumic]

As discussed with @edlerd, there are changes to the workflow in this PR which won't get picked up, making the tests still fail. I have another addition of e2e tests in the works (based on this one) which doesn't make changes to the workflow, so we can test the CI changes on that one once I open it. You can check that the tests for this branch are passing by looking at the fork run. So I'm merging this now.