Optimize redundant logic used in queries #7061
Conversation
Codecov Report
@@ Coverage Diff @@
## master #7061 +/- ##
==========================================
+ Coverage 85.41% 93.89% +8.47%
==========================================
Files 169 169
Lines 12425 12468 +43
==========================================
+ Hits 10613 11707 +1094
+ Misses 1812 761 -1051
Continue to review full report at Codecov.
|
|
Thanks for the PR! Can you write a test case for this? |
|
Would you please also open an issue for this to explain the proposed change? This allow others to join the discussion more easily and helps us to better trace changes in Parse Server. |
|
Have you tried your branch in production or tested it against your large database? |
|
Before merging, mind any ongoing discussion in #7065. |
Yes, after this optimization performance went back to the same range as before. |
|
I see that the test cases test the new reduction methods of the database controller, but there is no high-level test case that tests whether these methods are implemented correctly within the controller and in which scenarios they are actually applied to a query. Can we add some test cases that make a |
@mtrezza Good call! I previously tested only with two pointer permissions, so I added two extra test cases for 2 and 3 pointer permissions on a class and fixed and issue detected with the second case. |
@mtrezza No all, only on find operations related to pointer permissions and not the queries themselves. Update operations could also have unoptimized $and operations. A full optimizer for queries in general would require many more test cases. |
I was thinking into that direction. A Parse Query Optimizer would be an interesting feature, but maybe that's more a job for the guys at MongoDB. This PR looks good to me, ready for merge I'd say, just need to look into the failing tests. |
Leaving that task to MongoDB guys means that a MongoDB upgrade would be required for everybody and that would take even much longer. I've been already working on that idea in my own, it will take some time to be ready. This PR right now is already a good step forward. |
|
Checks passed. @dplewis any thoughts before merging? |
fix tests Postgres Support Update parse to 2.19.0 (#7060) Fix Prettier (#7066) Remove cache clear on validateObjects Improve add class if not exist Improve modifying schema instead of clearing Improve enforce class exists Fix flaky Test Release 4.5.0 (#7070) * Release 4.5.0 * Update CHANGELOG.md Co-authored-by: Tom Fox <[email protected]> * Improve braking change note * Create a breaking changes sub-section * Add release action Co-authored-by: Tom Fox <[email protected]> Improve issue templates & add PR template (#7051) * improved feature suggestion template * added test case chapter to bug report template * PR wording * added PR template * improved formatting in issue template * removed checkbox for concept due to new GH discussions process * improved wording * improved PR todo list * amended PR checklist; minor rewording * removed duplicate wording * add securtiy check section to contribution guide fix PR template file location (#7074) Optimize redundant logic used in queries (#7061) * Optimize redundant logic used in queries * Added CHANGELOG * Fixed comments and code style after recommendations. * Fixed code style after recommendation. * Improved explanation in comments * Added tests to for logic optimizations * Added two test cases more and some comments * Added extra test cases and fixed issue found with them. * Removed empty lines as requested. Co-authored-by: Pedro Diaz <[email protected]> FileUpload options for Server Config (#7071) * New: fileUpload options to restrict file uploads * review changes * update review * Update helper.js * added complete fileUpload values for tests * fixed config validation * allow file upload only for authenicated user by default * fixed inconsistent error messages * consolidated and extended tests * minor compacting * removed irregular whitespace * added changelog entry * always allow file upload with master key * fix lint * removed fit Co-authored-by: Manuel Trezza <[email protected]> Fix: context for afterFind (#7078) * Fix: context for afterFind * Update CHANGELOG.md Co-authored-by: Manuel <[email protected]> Fix max listener warning from livequery server (#7083) * fix max listner warning * fix * Clean test log Run definitions pg fix fix: upgrade ws from 7.4.0 to 7.4.1 (#7098) Snyk has created this PR to upgrade ws from 7.4.0 to 7.4.1. See this package in npm: https://www.npmjs.com/package/ws See this project in Snyk: https://app.snyk.io/org/acinader/project/8c1a9edb-c8f5-4dc1-b221-4d6030a323eb?utm_source=github&utm_medium=upgrade-pr fix: upgrade ldapjs from 2.2.2 to 2.2.3 (#7095) Snyk has created this PR to upgrade ldapjs from 2.2.2 to 2.2.3. See this package in npm: https://www.npmjs.com/package/ldapjs See this project in Snyk: https://app.snyk.io/org/acinader/project/8c1a9edb-c8f5-4dc1-b221-4d6030a323eb?utm_source=github&utm_medium=upgrade-pr fix: upgrade semver from 7.3.2 to 7.3.4 (#7092) Snyk has created this PR to upgrade semver from 7.3.2 to 7.3.4. See this package in npm: https://www.npmjs.com/package/semver See this project in Snyk: https://app.snyk.io/org/acinader/project/8c1a9edb-c8f5-4dc1-b221-4d6030a323eb?utm_source=github&utm_medium=upgrade-pr fix: upgrade uuid from 8.3.1 to 8.3.2 (#7101) Snyk has created this PR to upgrade uuid from 8.3.1 to 8.3.2. See this package in npm: https://www.npmjs.com/package/uuid See this project in Snyk: https://app.snyk.io/org/acinader/project/8c1a9edb-c8f5-4dc1-b221-4d6030a323eb?utm_source=github&utm_medium=upgrade-pr
|
🎉 This change has been released in version 5.0.0-beta.1 |
|
🎉 This change has been released in version 5.0.0 |
After upgrading parse-server from 2.7.4 to 4.4.0, MongoDB 4.0 maxed CPU usage from what used to be barely 20% usage.
This does reduce the redundant complexity on queries that involve pointer permissions.