Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert breaking changes to mocks (restoreAllMocks, spy changes) since 29.3.0 #14429

Merged
merged 7 commits into from
Aug 21, 2023

Conversation

robhogan
Copy link
Contributor

@robhogan robhogan commented Aug 18, 2023

Summary

A series of commits to jest-mock that landed in Jest 29.4.0-29.4.3 introduced breaking behaviour changes. Per semver, and especially because Jest's multi-package architecture make it difficult to "pin" to an old version, these kind of changes should be held until at least Jest 30.

Changes reverted:

In practice, we found this broke a few things, but the biggest by far was the change to restoreAllMocks such that it became equivalent to calling mockRestore on every mocked function. Note that this contradicts the docs:

Beware that jest.restoreAllMocks() only works for mocks created with jest.spyOn() and properties replaced with jest.replaceProperty(); other mocks will require you to manually restore them.

Mocks created "manually" by assigning jest.fn(), eg const myMock = jest.fn().mockReturnValue('foo') or global.setImmediate = jest.fn(cb => cb()) lost their implementation when restoreAllMocks was called. (Note that they're not really "restored" to any state they had previously, and they're still mocks). In particular this breaks legacy fake timers, and mocks created during setup to emulate an environment (popular for requestAnimationFrame, etc).

(Suggestion: If we do bring these changes back for Jest 30, it'd be great to have a restoreAllSpies() that does what the current restoreAllMocks() does - that'd make the migration much easier)

Related / fixes

#13229 should be reopened if this is merged

Test plan

  • Ran this (replacing jest-mock in an otherwise Jest 29.6.2 installation) against ~28k tests at FB, and everything passes that passed in 29.2.1
  • Preserved the unit tests that were added in the reverted changes, with expectations reflecting the pre-existing behaviour

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Aug 18, 2023

CLA Signed

The committers listed above are authorized under a signed CLA.

@netlify
Copy link

netlify bot commented Aug 18, 2023

Deploy Preview for jestjs ready!

Built without sensitive environment variables

Name Link
🔨 Latest commit abced97
🔍 Latest deploy log https://app.netlify.com/sites/jestjs/deploys/64e33599a0fe2f000737883c
😎 Deploy Preview https://deploy-preview-14429--jestjs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@@ -130,7 +130,7 @@ Beware that `mockFn.mockClear()` will replace `mockFn.mock`, not just reset the

Does everything that [`mockFn.mockClear()`](#mockfnmockclear) does, and also removes any mocked return values or implementations.

This is useful when you want to completely reset a _mock_ back to its initial state.
This is useful when you want to completely reset a _mock_ back to its initial state. (Note that resetting a _spy_ will result in a function with no return value).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should it be re-added to older versions of the docs as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe. I'd reconsider the wording though, I think. The behaviour (pre 29.4.0 / after this PR) is to replace all spy mock implementations with empty functions - that is, it's not just about the return value.

test('mocks, spies, resetAllMocks', () => {
  jest.spyOn(console, 'log');
  jest.spyOn(globalThis, 'setImmediate').mockImplementation(cb => cb());

  const mockCb = jest.fn();
  setImmediate(mockCb); // Calls mockCb synchronously
  expect(mockCb).toHaveBeenCalled();

  // spyOn wraps original implementation, foo printed to console
  console.log('foo');
  expect(console.log).toHaveBeenCalledWith('foo');

  jest.resetAllMocks();

  // Replaces default spy implementation with a stub - nothing is printed.
  console.log('bar');
  expect(console.log.mock.calls).toEqual([['bar']]);

  setImmediate(mockCb); // Never calls mockCb
  expect(mockCb).not.toHaveBeenCalled();
});

This is in fact what the paragraph above the one in question already says it should do, and IMO the confusion only arises from "Back to its initial state", which leads to people expecting it to do what mockRestore does (and possibly led to #13808).

Instead of:

Does everything that mockFn.mockClear() does, and also removes any mocked return values or implementations.

This is useful when you want to completely reset a mock back to its initial state.

How about:

Does everything that mockFn.mockClear() does, and also replaces the mock implementation with an empty function, returning undefined.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah good point. I think your suggestion sounds good 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated and rebased 👍

@robhogan robhogan force-pushed the robhogan/revert-mock-changes branch from a41537b to abced97 Compare August 21, 2023 09:59
@mrazauskas
Copy link
Contributor

This PR is somewhat puzzling for me.

Those aren’t just "our tests are not passing" problems. These are in the level "Jest does not run for us anymore". So would a revert PRs be considered as well?

@robhogan
Copy link
Contributor Author

robhogan commented Aug 21, 2023

Yeah, I'd consider those revert candidates. Semver is pretty clear on this:

Major version X (X.y.z | X > 0) MUST be incremented if any backward incompatible changes are introduced to the public API.
https://semver.org/#spec-item-8

As soon as you realize that you’ve broken the Semantic Versioning spec, fix the problem and release a new minor version that corrects the problem and restores backward compatibility.
https://semver.org/#what-do-i-do-if-i-accidentally-release-a-backward-incompatible-change-as-a-minor-version

Maybe it'd be worth considering moving development of a major onto branches as soon as a major is cut (eg, any commits that are 29.x candidates are made to a 29.x branch) rather than releasing off main, to avoid this kind of thing?

We will do our best to keep main in good shape, with tests passing at all times. But in order to move fast, we will make API changes that your application might not be compatible with. We will do our best to communicate these changes and always version appropriately so you can lock into a specific version if need be.
https://github.com/jestjs/jest/blob/main/CONTRIBUTING.md#main-is-unsafe

@mrazauskas
Copy link
Contributor

Thanks for bringing in those lines from Semver spec here. They answered all questions I had.

Somehow I was sure that any bug fix can be released as a patch, but the spec talks about "backward compatible bug fixes" only. That sounds smarter than I was thinking.

@SimenB
Copy link
Member

SimenB commented Aug 21, 2023

I don't really think the steam locomotive case should warrant a revert (even though it's weird Sapling itself doesn't provide an alternative binary) - it's people trolling themselves. But the symbol thing sounds like it probably should be reverted.


IMO, any "backward incompatible changes" is too broad, but it's very good general advice. But if you rely on a bug, and change to its behaviour shouldn't necessarily lead to a new major.

Extreme example, but obligatory: https://xkcd.com/1172/

@SimenB SimenB merged commit 085f063 into jestjs:main Aug 21, 2023
89 checks passed
@SimenB
Copy link
Member

SimenB commented Aug 21, 2023

@mrazauskas would you be up for preparing a PR essentially reverting this one again so we remember to reland for v30?

@robhogan
Copy link
Contributor Author

robhogan commented Aug 21, 2023

Thanks both!

Re the restoreAllMocks() change - the biggest break in practice seems to be one that might've been accidental, and isn't clearly beneficial - just wanting to reiterate that in case re-reverting accidentally brings it back :)

085f063#diff-7ae0bd704c3c2789b19abe2bbf94aca3505e2a6b3823d85c7f6b316b216d37c9L1461-R1419

The change reverted above, and in particular this._mockConfigRegistry = new WeakMap(); makes restoreAllMocks reset non-spies. restoreAllMocks() is explicitly documented to have no effect on non-spies (I'd be in favour of keeping but renaming it in v30).

@SimenB
Copy link
Member

SimenB commented Aug 21, 2023

https://github.com/jestjs/jest/releases/tag/v29.6.3

@mrazauskas
Copy link
Contributor

The change reverted above, and in particular this._mockConfigRegistry = new WeakMap(); makes restoreAllMocks reset non-spies. restoreAllMocks() is explicitly documented to have no effect on non-spies (I'd be in favour of keeping but renaming it in v30).

Documentation starts with "Restores all mocks" and "Equivalent to calling .mockRestore() on every mocked function". Later adds "only works for mocks created with jest.spyOn()" and ends with "other mocks will require you to manually restore them" (bolds are mine). A code example could explain this, but it is missing here.

Just above in the jest.spyOn() entry these two examples are given:

jest.spyOn(object, methodName).mockImplementation(() => customImplementation)

object[methodName] = jest.fn(() => customImplementation)

For me it sounds like the second case is that "other mock" which has to be manually restored. The "other mocks" are "all mock" and .mockRestore() should be called, but the property mock by the "other mock" can not be restored to the original implementation. In other words: "Beware that jest.restoreAllMocks() is able to restore the original implementation only for mocks created with jest.spyOn()".

@robhogan robhogan deleted the robhogan/revert-mock-changes branch August 21, 2023 18:30
@robhogan
Copy link
Contributor Author

robhogan commented Aug 21, 2023

That's fair - I think we can agree that the docs on this are not unambiguous. The current behaviour does match "no effect on non-spies" though, and has done for a long time. If that's to change, it might be worth considering giving folks an easy migration path.

FWIW, this broke almost every test using restoreAllMocks in the codebase I look after, which represents different developers across different teams over several years - that suggests that actual use in the wild of restoreAllMocks a) relies on not resetting non-spies and b) finds its current behaviour useful.

My interpretation of restoreAllMocks, has always been "restore all mocks that can be restored" - ie, only those that wrapped a real implementation Jest knows about.

@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants