Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: create index for Jobs queries #5137

Merged
merged 1 commit into from
Apr 19, 2019
Merged

Conversation

aldeed
Copy link
Contributor

@aldeed aldeed commented Apr 19, 2019

Impact: minor
Type: performance

Issue

In profiling DB ops locally, I noticed that the job-queue package does periodic queries of the Jobs collection with only status and expiresAfter as query fields. These queries do a COLLSCAN, which can be very slow as the number of jobs increases.

Solution

Added a { status: 1 } index on Jobs collection.

Breaking changes

None

Testing

Check out develop branch.

First run db.setProfilingLevel(2) in your local DB Mongo shell. This will tell it to log all queries.

Now start the app, wait a minute, and run this query:

db.system.profile.find({ 
    "command.find" : "Jobs", 
    "command.filter.status" : "running"
}).sort({ 
    "ts" : -1.0
});

You should see planSummary field is COLLSCAN on these queries.

Now check out this PR branch, start the app, wait a minute, and run the above query again. You should see planSummary field is "IXSCAN { status: 1 }" on the most recent one.

Finally, run db.setProfilingLevel(0) to turn off profiling since it will slow down your local DB.

@aldeed aldeed requested a review from rosshadden April 19, 2019 00:32
@aldeed aldeed self-assigned this Apr 19, 2019
Copy link
Contributor

@rosshadden rosshadden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@aldeed aldeed merged commit dd49201 into develop Apr 19, 2019
@aldeed aldeed deleted the perf-aldeed-add-jobs-index branch April 19, 2019 11:17
@jeffcorpuz jeffcorpuz mentioned this pull request Jul 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants