perf: create index for Jobs queries #5137
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Impact: minor
Type: performance
Issue
In profiling DB ops locally, I noticed that the job-queue package does periodic queries of the
Jobs
collection with onlystatus
andexpiresAfter
as query fields. These queries do aCOLLSCAN
, which can be very slow as the number of jobs increases.Solution
Added a
{ status: 1 }
index onJobs
collection.Breaking changes
None
Testing
Check out
develop
branch.First run
db.setProfilingLevel(2)
in your local DB Mongo shell. This will tell it to log all queries.Now start the app, wait a minute, and run this query:
You should see
planSummary
field isCOLLSCAN
on these queries.Now check out this PR branch, start the app, wait a minute, and run the above query again. You should see
planSummary
field is "IXSCAN { status: 1 }" on the most recent one.Finally, run
db.setProfilingLevel(0)
to turn off profiling since it will slow down your local DB.