Leverage test-scenarios from ember-auto-import

[thoov]

Problem

Currently our test suite is setup that each test package is an actual ember application (or addon). While this works, we incur a large overhead in maintenance, dryness, and complexity.

Solution

Instead of individual packages that are full blown applications, we can leverage a shared base "template" that we apply diffs on top. This base template can be maintained via ember-cli-update while all of the test scenarios don't need to care about the underlining template.

Future

This PR simply converts the fastboot-app test package over to the new format. Once this lands all the rest of the test pacakges can be converted one by one.

[ef4]

Thank you for fighting the good fight here.

[lifeart]

We had same problem for LS, and recently I spend some time on project serialization optimization, and now project could be stored as json file https://github.com/lifeart/ember-language-server/blob/component-context-info-origin/test/fixtures/batman/project.json found it's quite useful.

toObject normalizer for this format (https://github.com/lifeart/ember-language-server/blob/component-context-info-origin/test/test_helpers/integration-helpers.ts#L114)

serializer to flat format: https://github.com/lifeart/ember-language-server/blob/component-context-info-origin/test/test_helpers/integration-helpers.ts#L55

[thoov]

@ef4 / @rwjblue:

Ok this should be good to go now. The node_modules hoisting issue was just a bad yarn.lock (so I don't need to do the yarn hack). I also had to add @ember/string to a few packages to work around: emberjs/data#6791

This only adds release as scenarios so we can discuss what the full matrix should look like. Additionally, an improvement to scenario-tester would be to add matrix multiplication such that we can have environment variables that are different on each scenario (such as CLASSIC vs EMBROIDER)

[ef4]

Thanks, looking good.

an improvement to scenario-tester would be to add matrix multiplication such that we can have environment variables that are different on each scenario

Yeah the closest example is this plus this which lets you multiply out the scenarios, but not bake the environment variables right into the scenarios themselves.