A Dart builder that uses templates to generate HTML files for dart tests. Useful for projects with many tests that require custom HTML. Instead of having to replicate the custom HTML file for every test file that requires it, this builder can apply a template to any number of test files.
Add this package as a dev_dependency:
dart pub add --dev test_html_builderCreate as many HTML test templates in a test/templates/ directory as needed:
<!-- test/templates/example_template.html -->
<html>
<head>
<!-- The "testName" placeholder will be replaced with a unique name for
each test. -->
<title>{{testName}} Test</title>
<!-- Load custom assets needed by the test. -->
<script src="custom.js"></script>
<!-- Every template must include this placeholder. -->
{{testScript}}
<!-- It will be replaced by the builder with the required <link> tag:
<link rel="x-dart-test" href="...">
-->
<!-- Every template must include the test runner script. -->
<script src="packages/test/dart.js"></script>
</head>
</html>Tell the builder which templates should be applied to which files. For example:
targets:
$default:
builders:
test_html_builder:
options:
templates:
"test/react_with_styles_template.html":
- "test/components/styled/**_test.dart"
"test/react_template.html":
- "test/components/**_test.dart"To illustrate how this works, consider an example test directory structure:
test/foo_test.dartreact_template.htmlreact_with_styles_template.htmlcomponents/bar_test.dartstyled/baz_test.dart
Running tests via dart run build_runner test with the above configuration will
result in the following (hidden) generated outputs:
test/components/bar_test.html(fromreact_template.html)test/components/styled/baz_test.html(fromreact_with_styles_template.html)
-
If there is overlap between the globs defined for multiple templates, the builder will choose the first template that matches.
-
If none of the templates match for a given test file, no html file will be generated.
-
One-off custom html files for individual tests are still supported, but they must use the
.custom.htmlextension:test/example_test.darttest/example_test.custom.html
Some projects would like additional assurance in a more production-like
environment. This is typically done by running browser tests in release mode
(-r|--release), which compiles them with dart2js. However, in large projects
with a lot of browser tests, this can be prohibitively slow because dart2js is
not a modular compiler like the Dart Dev Compiler. This means that dart2js
treats each browser test as a full program that must be compiled together with
all of the files it imports transitively, which results in a lot of duplicate
work.
One option to workaround this slow process is to create an "aggregate" test that imports all of your browser tests and runs them. With this in place, dart2js only needs to compile one thing, but all of your browser tests still run. This approach has some caveats:
- An aggregate test can only have one HTML file associated with it. If your project uses multiple HTML templates, you'd need an aggregate test for each.
- You need to setup some config and customize the command used to run tests such that only the aggregate tests are compiled and run; otherwise, you might still be spending a bunch of time compiling all of the individual tests unnecessarily.
The builder provided by this package can automate all of this! First, enable
this functionality by setting browser_aggregation: true:
targets:
$default:
builders:
test_html_builder:
options:
templates:
"test/react_with_styles_template.html":
- "test/components/styled/**_test.dart"
"test/react_template.html":
- "test/components/**_test.dart"
browser_aggregation: trueOnce enabled, the builder will generate an aggregate test for each template that
imports and runs each test that uses the template. It will also generate a
default aggregate test for browser tests that don't match any of the templates.
Finally, it generates a dart_test.browser_aggregate.yaml file that can be
included in your project's dart_test.yaml so that the aggregate tests can be
easily selected with this test argument: --preset=browser-aggregate.
# dart_test.yaml
include: dart_test.browser_aggregate.yamlTo run these tests, you can use the executable provided by this package:
dart run test_html_builder:browser_aggregate_tests [--release]
Or, if you have your own test runner that you'd like to integrate this functionality into, you can run:
dart run test_html_builder:browser_aggregate_tests --mode=args [--release]
which will print the necessary build_runner and test args in this format:
<build args> -- <test args>
# For example:
--release --build-filter=test/templates/react_template.browser_aggregate_test.** -- --preset=browser-aggregate
You can parse these or pass them directly into a command to run tests, like so:
dart run build_runner test $(dart run test_html_builder:browser_aggregate_tests --mode=args [--release])With dart test, you can randomize the order in which test files are run by using the --test-randomize-ordering-seed flag. However, when using the this package's browser aggregation feature, this method of test order randomization doesn't work because there will be only be one test file for each template. In other words, most of the tests will continue to be run in the same order.
For that reason, test_html_builder also supports a randomize_ordering_seed option that can be set in your build.yaml. It behaves essentially the same as the --test-randomize-ordering-seed flag, meaning that you can set the value to a specific seed or "random" if you'd like the seed to be picked at random for you.
Here's an example:
targets:
$default:
builders:
test_html_builder:
options:
...
browser_aggregation: true
randomize_ordering_seed: randomIf you notice a test failure due to the specific shuffling that occurred during that test
run, update the randomize_ordering_seed option to be whatever was output in the test runners log.
When randomizing the test order it is recommended to ignore the aggregated test files from version control to avoid constant updates.
See the Contributing Guidelines.