Setup test suite for type definitions

[SimenB]

Jest is written in TypeScript, but that doesn't mean that the different APIs Jest injects into a test are particularly well typed. Having good type tests for expect and the different methods available on the jest object (jest.spyOn, jest.mock etc) would be great to assert that they provide the DX they should. Especially since we hope to include global typings with Jest 25, we should ensure we are safe to move quickly if they are painful for people coming from the DefinitelyTyped types. Especially assertions that invalid things are reported as type errors would be great.

Current test suite in DT: https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/jest/jest-tests.ts

I'm not sure what tool would be best for this, but something like https://github.com/microsoft/dtslint or https://github.com/SamVerschueren/tsd maybe? It should run as part of Jest's test suite, so we need a custom runner for them (or some other tool that does the same job).

This runner could then of course be used by other people who want to test their type defs, and run those tests with Jest.

/cc @orta if you have any thoughts you wanna share on this

[orta]

We chatted in discord: I feel like tsd is a good spot for this and there are techniques like const a: import("jest").Expect which would allow types to be passed around internally without too much churn.

[jevakallio]

Working with @MLH-Fellowship on this, summarising some of the internal fact-finding here.

• I built a quick prototype for checking typescript definitions at type level using tsd, and it seems like for many users, combination of ts-jest + tsd + // @ts-expect-error would allow to test their types.
• In Jest itself, we don't run tsc as part of the test run, we just compile the typescript source to javascript with babel, so we'd need to bring typescript errors as part of the runtime
• Therefore we need a method for testing typescript types that doesn't rely on running tsc as part of the jest run
• We should be able to achieve this by create a jest-runner that can run tsd and pass the diagnostics to jest

Solution

Requirements:

• Should be a test runner
• Should support using tsd assertions (expectType etc.)
• Should pass results to reporters
• Should work as part of Jest's own test suite

Ideal solution:

• Can run tests anywhere in the test tree (but acceptable if in a specific directory, as that's how TSD works internally)
• Allows writing tests in "Jest style" (describe, it) and reports test success/fails as part of the right test (this might be tricky, TSD test format is acceptable)
• Uses TSD directly (but forking or inlining is an option, if needed)

[raijinsetsu]

This seems huge for making sure that public APIs do not change across versions and that expectations of how Typescript resolves the types remains true.

I am able to use the expectType functions (those work without any Jest changes) but the expectError and similar just will not. I also have to mock up the implementation because, unlike when it's run under "tsd", running the tests under Jest DOES mean that the code will be executed and that means that it must, at least, not throw an error.

[SimenB]

This landed in #10407. While we don't test that much yet, the infrastructure is now in place 🎉

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.