Feature request: Support using Jest testing framework

[kuncevic]

This is a proposal/future request

Current Angular CLI testing tools based on karma and jasmine that we all get use to it. But would you consider to support Jest as well https://github.com/facebook/jest ?

Jest testing framework commonly used across React and Nodejs projects. It is supports unit tests, assertions, mocks, coverage, live test reload, JS File mocks and has a lots of features build in to it like 'Snapshot Testing' also is provided zero configuration experience and fully compatible with webpack.

Goal

Just thinking if it might help the Angular community to improve out testing story using Jest and it would of be great it it can be available with Angular CLI.

Any thoughts?

[dancancro]

We recently tried that out in a React project. It seemed pretty cool but we had trouble getting it to work on Windows. Has anyone had better luck?

[delasteve]

@kuncevic, I would suggest seeing if jest works with Angular, first. I know there has been issues in the past for mocha and jasmine users when using async. There had to be a specific patch put in place in the zone.js library.

https://github.com/angular/zone.js/blob/master/lib/mocha/mocha.ts
https://github.com/angular/zone.js/tree/master/lib/jasmine/jasmine.ts

I know @hansl has mentioned he would only like to support one testing framework, jasmine. If you can verify the async issue and/or get the patch in place for jest, I will do my best to get a PR in place that supports jasmine, mocha, and jest.

[mattlewis92]

I looked into this before as jest is really awesome, unfortunately it only runs on node so there isn't a way of getting it to run with karma in the browser. There's an open issue for it here: jestjs/jest#848

Due to the way angular abstracts the DOM it might be possible to get it working, but it would be a big difference from the current test setup.

[beeman]

I have set up a demo project here https://github.com/beeman/angular-cli-jest and the first impression is that it works fine. I did not explore any advanced feature yet though.

[thymikee]

So it's currently possible to use Jest with Angular, and it's pretty forward to add/migrate thanks to jest-preset-angular. I've made a blog post about this very topic, thought you might be interested.

What @beeman proposed is half a solution. Jest also needs to run in Zone.js context, otherwise testing components will fail.

Although it would be great, if we could bake it into the cli and make it an option while bootstrapping somehow.

[beeman]

@thymikee your preset looks great, I'm about to test it.

I don't think Jest will find it's way into the CLI any time soon. It might be a nice thing to look at when the CLI starts supporting custom blueprints.

[deebloo]

This is a great candidate for a blueprint!

[intellix]

This blog post has just appeared which apparently goes over using Jest in CLI 1.0.0: https://www.xfive.co/blog/testing-angular-faster-jest/

Just ran some speed tests on ~400 tests:

Karma: 0m48.999s
Jest: 0m31.385s

Edit: Just tested watch mode and it's brilliant. You can filter by filename or test name via patterns and every time it detects a change, it'll only run the tests from those matched files.

Also, it's able to see what files were last changed via git and only run tests against changed files, so you know within seconds when you've broken a test as opposed to running the whole suite on any change.

ne file of tests: Time: 1.536s, estimated 2s:

[vecernik]

I eagerly tried ng-cli 1.0 default project with xfive.co and jest-preset-angular changes described, also tried https://github.com/beeman/angular-cli-jest and none was working on osx/node7.8. Did I miss anything?

[thymikee]

@vecernik please describe what went wrong on jest-preset-angular issue tracker, we'll see what can be done :)

[Meligy]

I tried Jest with instructions from @thymikee's post (which uses jest-preset-angular), and it worked just fine.

Let me share my findings:

Result Summary

• It works!
• It took some small changes.
• It seemed slower in my case.
• It caught more errors than Webpack/Karma/Jasmine, which I was very grateful for.
• The experience is quite different though that I won't make it the default my team yet.

Required Changes

• I started with all the steps in the post above.

• For some libraries I had to follow this github guidance, which made the jest property in my package.json look like:

  "jest": {
    "preset": "jest-preset-angular",
    "setupTestFrameworkScriptFile": "<rootDir>/src/setupJest.ts",
    "transformIgnorePatterns": [
      "^<rootDir>\\node_modules\\(?!ng2-bootstrap|angular2-json2csv|angular2-datatable)."
    ]
  }

  The transformIgnorePatterns is the most interesting here. I used \\ because I'm on Windows, and I used ^<rootDir>\\ to try to speed things, although it didn't make a big difference. There's also "allowJs": true in "compilerOptions" in src/tsconfig.spec.json.

• I also had to change the tests. The post mentions all the changes needed (mostly only changing jasmine. to expect. in a few places).

  But there's another change I had to do, which is removing any expectationFailOutput, which is when you give a custom error message for when a matcher fails to match. Seems Jest does not support that.

Gains

• Jest seems to run tests in more isolation than the current Webpack/Karma/Jasmine combo, which showed me some errors in my tests that somehow didn't show before.

• Jest is reporting which tests are taking too long, the slowness warnings were useful smells to identify not-greatly-written tests.

• The watch mode is very nice, even though it seems to work off git changes not file watching, which can be confusing at first when you see slightly more files than expected, but it's still very useful.

Issues

• As mentioned in Required Changes above, now I cannot use custom error messages in my test matchers (that said, it's true that default matcher errors are very beautiful and obvious, but still).

• At the moment, it seems the jest-preset-angular initialization code needs optimization. It doesn't call polyfills.ts and instead calls its own set, even though it's optimized for Angular CLI (for example, it uses src/tsconfig.spec.json).

  More importantly, it imports the entire rxjs library, which might hide errors when you forget to import some operators.

• It's surprisingly slower than the Webpack/Karma/Jasmine combo!

  For my 196 tests, the Angular CLI v1.0.0 default test runner (with Angular 4) takes ~ 55 seconds, while Jest takes ~100 seconds.

  Update: That conclusion might not be accurate. The time I quoted for each test runner is what the test runner reported. This might for example not include Webpack compilation and browser opening in Karma and might be total time in Jest. If so, then the total time would be the same.
  Thanks @hugoccampos for bringing this to my attention.

• No browser tests debugging apparently (It might be my ignorance here). A bit harder maybe for those starting testing.

  The Angular CLI does a great job at making testing easy for those not used to it.

• When Jest itself fails to run, for example if I put a badly formatted regex for transformIgnorePatterns shown above, or mess up something in my src/tsconfig.spec.json file, the errors it shows are very cryptic and tell you nothing that can lead to the real issue (unlike matching errors in tests, which are very nice).

[thymikee]

@Meligy thanks for your input, really appreciate it!

You should definitely time the execution of whole process (in my case it's much faster overall).
As of your required changes – they are described in the Troubleshooting section of project's readme.

It doesn't call polyfills.ts and instead calls its own set

That's right. It's because polyfills are also for browsers, and Jest runs through Node, and hence we need dist/zone-node.js instead of dist/zone.js

More importantly, it imports the entire rxjs library, which might hide errors when you forget to import some operators.

Working on it! thymikee/jest-preset-angular#17
Edit: done, released v1.0

No browser tests debugging apparently (It might be my ignorance here)

You're not being ignorant, as for now Jest unfortunately doesn't integrates well with browsers, but it may change in future (but not so soon)

[intellix]

Aye, when you use time to time the whole process (including compilation) then I found the tests took about the same amount of time between Jasmine/Jest.

I love that it shows my slow tests. Actually, I've noticed that components with large views and a lot of form elements are super slow and was wondering if there's any way to speed those up

Example, a registration-form component with about 10 Form Controls and 100 lines of HTML takes about 10 seconds to run through all my assertions.

• With 100 lines of HTML: 300ms per assertion
• After deleting all HTML within my component's view: 50ms per assertion

[Meligy]

@thymikee Thanks heaps man. I was planning to look into making the RxJS change in the project in a few days. You did it and released in a few hours. Super!

@intellix I haven't timed fully yet (after easter, as it's on a different machine), but yes, they feel overall the same total time. Also, I had the same results around which tests take longer time.

Container components test seem to be the slowest. I'm not sure yet whether it's compiling all the child components or interacting with the debug DOM that causes most time, but it's quite slow. Some tests go for 15 seconds or more. I suspect it's the DOM, you know, the jsdom emulation layer, is much slower than the real thing (at least Chrome).

[victornoel]

Also, be careful with the fact that protractor is still using jasmine, so when using you favourite IDE, it's possible that you either end up with jest assertions functions being proposed by the IDE in e2e tests, or if you leave @types/jasmine, jasmine assertions functions being proposed by the IDEA in your unit tests…

This is more a limitation of the IDE than angular, but the problem exists still :)

[victornoel]

Also, be careful with the fact that protractor is still using jasmine, so when using you favourite IDE, it's possible that you either end up with jest assertions functions being proposed by the IDE in e2e tests, or if you leave @types/jasmine, jasmine assertions functions being proposed by the IDEA in your unit tests…

FYI, it is easily workaroundable by specifying tsconfig.json in e2e/ and src/ directories with the correct types declared.

[intellix]

Generated an Angular CLI 1.0.0 project and created a commit to show the required changes to get Jest working in your projects for anyone who hasn't made the leap yet :) intellix/angular-cli-jest@07829f5

[victornoel]

@intellix you can also remove karma and related dependencies I think.

This is how I did it in the end: https://gitlab.com/linagora/petals-cockpit/commit/91955c8d5d2209c2bcd19cb1d3050eef23b68f55

It is quite similar, but also update the tsconfig.json files (because you get jasmine bindings if not) as well as introduce some for nice IDE experience as I noted in a previous comment here.
Also I remove karma stuffs, and have a few different configurations for jest to ignore some things or include others for coverage :)

[filipesilva]

I had a go myself and there is a big blocker unfortunately... there doesn't seem to be a way to add the the webpack build chain we use to Jest, instead it uses a custom transform to inline templates.

This is a problem because a lot of functionality we enable in the CLI depends heavily on webpack processing. For instance, correct scss compilation.

Having a different build system for test mean it might work on some cases, maybe it doesn't on others, and it needs double the work for changes. We had this for a while with Karma actually, and it was the source of a lot of insidious bugs.

So for now I'm sorry to say Jest is not really a viable option to fully integrate into the CLI.

[victornoel]

@filipesilva thanks for the detailed feedbadk!

I'm not clear when it is problematic to use jest, when testing the angular component that uses scss? Is that it?
I use jest but it's only for non-angular-specific code, so it's ok for now, but I wonder if we will have to go back to karma then…

[filipesilva]

It's problematic whenever your code uses functionality that depends on the CLI build system, rather than anything that's just part of TypeScript+html/css inlining (which the Jest preset does in a regex based way). SCSS compilation would be problematic, yes.

[cgatian]

I also ran into issues attempting to get this setup. Some quick notes on roadblocks.

• Issues getting angular/material since they are examining window.matchMedia. While you can mock this object I feel like this is just the tip of the iceburg. I want to be able to write tests quickly and not have to continue to mock each missing piece of jsdom. You quickly find yourself refactoring your component tests to use NO_ERRORS_SCHEMA more than you would like to.
• using require statements for asset resources (e.g. png). I couldn't get this to work, and while the documentation states there's a workaround by specifying moduleNameMapper in the config, all of this just feels hacky.
• Had to throw accessibility tests out the window. We are using axe for accessibility testing and these simply failed with jsdom. Granted, I didn't look further into the issue.

Jest is a really great idea, and the feedback loop for writing tests is awesome. Hopefully with headless Chrome hitting mainstream there will be some advances in Jasmine to get closer to the parity of Jest.

I agree with @filipesilva. I don't think its ready for the masses.

[davidanaya]

Ey, any update on this? Is it possible to use Jest with angular-cli?

[4kochi]

@davidanaya You can use the jest preset from @thymikee which is described here: https://github.com/thymikee/jest-preset-angular

Works like a charm for us.

[maxime1992]

@4kochi no doubt it's working great for unit test. But out of curiosity, is it working well with integration tests and compiled templates? Last time I tried (a long ago), it was not doing well especially with the fact that I'm using scss and for some reasons it was just crashing.

Can you give a bit more of information on the app you use jest? Is it medium, large?
What kind of tests are you doing? Any particular difficulty that you've encountered?

[4kochi]

@maxime1992 We have a medium size project I would say. But we are only doing unit test with jest. So no angular helper methods from the TestBed class. Just calling the component constructor and testing the ts code. So I can’t say if this is fixed now. But you can make an issue in the jest-preset-angular if it’s still the case. They normally respond really fast.

[GoGoris]

@filipesilva Can this be achieved now that Angular moves to Bazel?

[benjamincharity]

@maxime1992 we are using Jest for both unit and integration tests (via jest-preset-angular) and it's been working just fine for us.

In case it's helpful, this repo is using Jest for all library code (~80% unit / 20% integration).

[maxime1992]

https://twitter.com/JesseS_BrieBug/status/1011989104416808960

https://github.com/briebug/jest-schematic

[cgatian]

Even better is the builder!

https://github.com/angular-builders/jest

[sinedied]

@filipesilva maybe this issue can be reopened since the angular builder is there?

[clydin]

@sinedied You can use the above mentioned builder directly in a project. There are no changes to the CLI code required.

[checketts]

@cgatian Looks like the angular builder url changed to https://github.com/meltedspark/angular-builders/tree/master/packages/jest

[maxime1992]

Just to have people aware before deciding to move on Jest, it seems that it's not possible to use fakeAsync with Jest... Which in my case is a huge issue.

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}