Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: resolve repeatable test suite breakage with ioredis 4.19.0+ #1933

Merged

Conversation

Rua-Yuki
Copy link
Contributor

This PR resolves failures in the repeatable job test suite brought about by upgrading ioredis to 4.19.0+, as mentioned in the comments for 6fe2928 and #758.

The precise change in ioredis which yields breakage is the introduction of a new script cleanup helper interval, intended to periodically purge a dictionary used for tracking scripts loaded for the purpose of pipelining. See here, or observe the excerpt below.

      this._addedScriptHashesCleanInterval = setInterval(() => {
        this._addedScriptHashes = {};
      }, this.options.maxScriptsCachingTime);

Nothing is intrinsically wrong with what ioredis has done here. We only see breakage within the Bull tests due to a chance interaction with the new cleanup interval, brought about by extremely taxing usage of sinon.

During the test setup phase, sinon is used to spoof timing functions, applying a base time equivalent to the Unix epoch.

this.clock = sinon.useFakeTimers();

Within some tests, the same sinon clock is advanced by massive amounts at once (over 47 years).

bull/test/test_repeat.js

Lines 200 to 201 in dbc08f2

const date = new Date('2017-02-07 9:24:00');
this.clock.tick(date.getTime());

These clock.tick(...) calls result in a punishing number of executions (around 25 million) of the aforementioned interval. This all takes quite a while, leading to timeouts in the end, and ultimately being the root of test failures.

The solution for this thankfully appears to be a simple one: merely swapping out clock.tick with clock.setSystemTime.

Update from pinned ioredis 4.18.0 to the latest version via ^4.19.2.

ioredis was previously pinned to 4.18.0 after tests began breaking with
the upgrade to 4.19.0.

Running tests with this version should result in a number of failures
in the repeatable job test suite.

These failures are a result of the manner in which `sinon` is used to
make massive advancements in time, thus triggering huge amounts of
interval handler executions as catch up is done.
This should resolve test errors in the repeatable job suite brought
about by upgrading ioredis to 4.19.0+.
@manast
Copy link
Member

manast commented Dec 13, 2020

Great!, thanks for the fix!.

@manast manast merged commit 6725f97 into OptimalBits:develop Dec 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants