Add workflow to run e2e tests from lxd-ui

[edlerd]

Added a step to the GitHub test workflow, executing the e2e test suite from lxd-ui with a lxd backend built from the current branch from a PR or main branch.

We might extend this on the stable-5.21 and stable-5.0 branches, as we have dedicated test suites in lxd-ui for those. For older versions, we should continue skipping the e2e tests. I'd keep this as a second step in a followup PR.

This was successful on a fork, i.e. see this run.

[tomponline]

Hi @edlerd please can you work with @simondeziel to update this to use the pre-built LXD binaries we now have (that are compiled during the Code Tests phase) and to use the re-usable workflows for performance tuning and dependency installation.

Also i'd like to avoid building the docs twice, but they are built into the tarball i believe during the Code Tests phase so may be available already now.

[edlerd]

@simondeziel @tomponline Updated the workflow to consume the build artefacts from previous steps. Please give it another review.

[edlerd]

Also using the docs artefact by depending and downloading on that build step.

[tomponline]

Is this a secret key?

[edlerd]

Same as the password over there

[tomponline]

Can we use our mini oidc service that we use for testing oidc in LXD? cc @markylaing

[edlerd]

That is a good hint, I'll check how to adopt it.

[edlerd]

Using the mini OIDC implementation is not trivial. It is only implementing the device flow, which is incompatible with the web flow.

[tomponline]

OK so we can stick with the external service, but all identifiers should be GH secrets.

[tomponline]

This shouldnt be hardcoded into the test file, as it references an external service, and thus is a real password.

We should now change this password to something else and then securely transfer it to me so we can set it up as a GH secret.

[edlerd]

This is a dedicated free account, specially created for e2e tests. It is expected to be public. I don't see a reason for this to be kept secret.

Though once we start using the local OIDC service, we can remove this.

[tomponline]

Hrm, yeah, but we shouldnt be making the credentials public

[tomponline]

please can you move this as part of the system-tests: workflow and make it a step after name: "Run system tests (${{ matrix.suite }}, ${{ matrix.backend }})" and before name: Upload coverage data

Then we can re-use the dependency install deps, and by ensuring LXD is run with the code coverage env var, we can collect coverage data from the UI tests too.

[edlerd]

This will slow down LXD test runtime. With current changes from this PR, the UI tests run in parallel to the system tests, which will not affect your total runtime. Because system tests take longer than the UI tests. I think a better option is to also record coverage in the UI step and download both coverage reports in the TICS step.

[simondeziel]

Suggested change

	curl --head --fail --retry-delay 2 --retry 100 --retry-connrefused --insecure https://localhost:8407
	curl --head --fail --retry-delay 2 --retry 100 --retry-connrefused --insecure https://localhost:${{ env.PORT }}

[edlerd]

Good idea.

[simondeziel]

Could that be one of the loopback IPs instead?

[edlerd]

The ui is is running in a container, so we need to use this to reach out from the container that is running the ui. Loopback would be inside the container and not reach lxd which runs on the host.

[simondeziel]

Could we have those generated on the fly please? You can potentially reuse test/includes/certificates.sh that has a handy function called gen_cert_and_key.

[edlerd]

Generation of the keys is taking ~1 minute and burning cpu cycles on each ci run seems easy to resolve, hence we introduced this cache a while ago in the lxd-ui pipeline. It is just a few kb and I think the benefits outweigh here. wdyt?

[simondeziel]

Hmm, maybe that was due to RSA being slow as with our function, this should be even faster than fetching from the cache:

$ . test/includes/certificates.sh
$ export LXD_CONF=$(mktemp -d)
$ time gen_cert_and_key lxd-ui
-----

real	0m0.007s
user	0m0.003s
sys	0m0.004s

So hard to beat :)

And you get such cert/key:

$ openssl x509 -noout -text -in $LXD_CONF/lxd-ui.crt
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            0b:d3:1c:7f:64:54:df:b9:34:af:1f:cc:40:14:bd:8b:b2:92:03:0f
        Signature Algorithm: ecdsa-with-SHA384
        Issuer: CN = lxd-ui.local
        Validity
            Not Before: Nov 14 23:13:44 2024 GMT
            Not After : Nov 15 23:13:44 2024 GMT
        Subject: CN = lxd-ui.local
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (384 bit)
                pub:
                    04:bd:8c:4e:94:7d:bb:66:de:f7:52:a8:05:f8:19:
                    d4:32:0b:4c:8a:b2:60:42:77:27:33:7e:74:cb:af:
                    39:66:70:3c:d9:f6:79:26:c8:5c:43:cf:1e:b9:e5:
                    ee:02:42:55:ec:04:15:c8:4f:9a:93:ad:69:9c:87:
                    dd:95:c4:52:69:66:93:22:5a:e0:77:65:51:2e:fd:
                    76:21:28:87:23:82:8d:55:f6:8d:1e:2c:54:0e:1a:
                    98:1f:8f:4f:fd:44:98
                ASN1 OID: secp384r1
                NIST CURVE: P-384
        X509v3 extensions:
            X509v3 Subject Key Identifier: 
                C8:FC:A2:AC:F1:1C:85:0E:E8:7C:FD:A3:A3:7B:27:AE:D0:8C:54:BB
            X509v3 Authority Key Identifier: 
                C8:FC:A2:AC:F1:1C:85:0E:E8:7C:FD:A3:A3:7B:27:AE:D0:8C:54:BB
            X509v3 Basic Constraints: critical
                CA:TRUE
    Signature Algorithm: ecdsa-with-SHA384
    Signature Value:
        30:65:02:31:00:82:02:fc:69:6f:ec:88:f6:9f:53:94:b2:68:
        76:1c:b6:f4:33:7a:0f:62:49:df:7e:95:a7:37:b5:9a:df:78:
        08:a2:14:d4:57:4e:7e:ec:db:75:eb:7c:5d:c3:75:ac:cf:02:
        30:4f:b3:c5:22:63:a3:6b:c6:00:7c:98:80:cd:91:1e:62:11:
        5a:2c:e0:72:d0:6c:db:d7:d4:93:85:79:0c:8b:02:5f:0c:2b:
        bc:47:ab:06:b0:c3:d4:64:1d:ad:76:20:9f

[simondeziel]

Please use SHA pinning for external actions.

[simondeziel]

No need to launch, init will do. Also, we tend to use minimal-daily elsewhere:

Suggested change

	sudo -E LXD_DIR=/var/lib/lxd lxc launch ubuntu-minimal:22.04 my-instance
	sudo -E LXD_DIR=/var/lib/lxd lxc init ubuntu-minimal-daily:22.04 my-instance

But if you don't actually need a full fledged instance, we have a simple busybox testimage. See test/includes/setup.sh.

[edlerd]

This is a workaround to generate an image from an instance, before this was not possible in the UI. I think by now we can generate the image directly in our e2e test. So we might be able to get rid of this preparation step completely. I need to double-check if we can evolve our test, but I am optimistic it will work. Will keep this open for now.