Run tests as part of CI

[mrlacey]

Integrate running unit tests as part of the PR CI

For #48

[michael-hawker]

Looks like they ran great on the last past. Do we know how test results are bubbled up in GitHub actions? (Like if they fail I assume the build will fail, but if we want to search/look up the output for an individual test, can we do that easily somehow?)

[mrlacey]

Looks like they ran great on the last past. Do we know how test results are bubbled up in GitHub actions? (Like if they fail I assume the build will fail, but if we want to search/look up the output for an individual test, can we do that easily somehow?)

I'll push an update with a failing test so it can be seen.

[mrlacey]

Looks like they ran great on the last past. Do we know how test results are bubbled up in GitHub actions? (Like if they fail I assume the build will fail, but if we want to search/look up the output for an individual test, can we do that easily somehow?)

I'll push an update with a failing test so it can be seen.

So, good news, a failing test does fail the CI checks.

The details of why it failed aren't always going to be super obvious. (We don't even get coloring.)
But, at least we get the name of the failed test, next to the "Failed" keyword, and a stacktrace.
Where I've seen a small number of failures amongst a large number of tests, the failures do stand out more obviously than here.

Removing the failed test now.

[michael-hawker]

@mrlacey looks like we may need another action to push/bubble results. This one looked interesting: https://github.com/marketplace/actions/test-reporter, though there's an updated clone here in the marketplace that I'm looking to get info on: https://github.com/phoenix-actions/test-reporting

Once the repo is public, I think we could maybe just try this one as well: https://github.com/marketplace/testspace-com/plan/MDIyOk1hcmtldHBsYWNlTGlzdGluZ1BsYW41OTc0#pricing-and-setup

[mrlacey]

@mrlacey looks like we may need another action to push/bubble results. This one looked interesting: https://github.com/marketplace/actions/test-reporter, though there's an updated clone here in the marketplace that I'm looking to get info on: https://github.com/phoenix-actions/test-reporting

Once the repo is public, I think we could maybe just try this one as well: https://github.com/marketplace/testspace-com/plan/MDIyOk1hcmtldHBsYWNlTGlzdGluZ1BsYW41OTc0#pricing-and-setup

What's the desired output?
Or is it a case of see what we can get and then decide?

[mrlacey]

This is from a passing test, the output looks like this:

And the details:

I'll now rerun with a deliberately failing test.

[mrlacey]

And here's what we'll see when there's a failling test:

details:

and

Removing failing test now

[mrlacey]

The CI test reports are currently created with https://github.com/dorny/test-reporter
and could easily be switched to https://github.com/phoenix-actions/test-reporting if necessary.

It's also using a format that's supported by https://github.com/marketplace/testspace-com so switching to that should be painless if we eventually want to move to that.

[michael-hawker]

Looks great @mrlacey! Good to know everything looks fairly interchangeable. I'll look into the Testspace system a bit more after the repo goes public. Opened #72 for tracking.