Skip to content

Commit

Permalink
internal/ci: stop using txtar tests for GitHub actions workflows
Browse files Browse the repository at this point in the history
In the early days of the GitHub actions workflows, prior to unity, the
import from the JSON schema source of truth for GitHub actions
workflows, and subsequent evaluation of workflow declarations, provided
a very good source of test material for changes in the evaluator. At the
time, the evaluator and the workflow declarations were both changing a
lot. So it made sense to have the generation of the Yaml workflow
specifications be based on cmd/cue from the current commit as opposed to
a previous version (@master commit or a semantic version).

But having to remember to run go generate in order to validate the
workflow of updating GitHub actions workflows was a regular footgun.

As such we created txtar tests based on the current GitHub actions
workflow declarations that then ensured go test _did_ capture the entire
workflow:

  cue/testdata/eval/github.txtar
  encoding/jsonschema/testdata/github.txtar
  cmd/cue/cmd/testdata/script/cmd_github.txt

However, this is where things got a bit circular and burdensome. The
files in the txtar tests needed to be updated following a change to the
workflow declarations, just as the expectations needed to be updated
following a change to the workflow declarations or the evaluator (or any
related tooling).

The evaluator has now settled down significantly, taking away one major
source of changes. As such, the overhead associated with maintaining
txtar tests outweighs any benefits.

Instead, we can factor out the txtar tests to equivalents in a
unity-tested repository, cue-unit/cue-github-actions.

In this CL we therefore:

* Base the GitHub Actions workflow on a prior known good version of CUE,
  v0.4.3.
* Split the "vendor" and "import" process into two steps to simplify the
  process of vendoring these packages into cue-unit/cue-github-actions.
* Simplify the command declaration in internal/ci and
  internal/ci/vendor, most notably using injection vars for OS.
* Remove the step of updating txtar tests from internal/ci commands and
  the associated call from internal/ci/gen.go.
* Remove internal/ci/updatetxt which is no longer needed.
* Remove the txtar tests listed above.

The packages internal/ci/... can then be "vendored" into
cue-unit/cue-github-actions as the basis for external, unity-driven
tests of this workflow with no loss of code coverage, but a
significantly smoother development workflow.

Signed-off-by: Paul Jolly <[email protected]>
Change-Id: I13606d3bf74050d1dbeb87b16713e3a19a66de7e
Reviewed-on: https://review.gerrithub.io/c/cue-lang/cue/+/540682
Unity-Result: CUEcueckoo <[email protected]>
TryBot-Result: CUEcueckoo <[email protected]>
Reviewed-by: Roger Peppe <[email protected]>
Reviewed-by: Daniel Martí <[email protected]>
  • Loading branch information
myitcv committed Jul 1, 2022
1 parent b8703c8 commit 1876276
Show file tree
Hide file tree
Showing 7 changed files with 26 additions and 6,286 deletions.
Loading

0 comments on commit 1876276

Please sign in to comment.