Support filtering on nonindexed predicate #4531
Conversation
There was a problem hiding this comment.
Dismissed @golangcibot from 5 discussions.
Reviewable status: 0 of 4 files reviewed, all discussions resolved
There was a problem hiding this comment.
clean PR. Check with @manishrjain whether this could have performance impact.
Reviewed 3 of 4 files at r1, 1 of 1 files at r2.
Reviewable status:complete! all files reviewed, all discussions resolved
There was a problem hiding this comment.
Reviewed 3 of 4 files at r1, 1 of 1 files at r2.
Reviewable status: all files reviewed, 6 unresolved discussions (waiting on @animesh2049)
query/common_test.go, line 289 at r2 (raw file):
owner : [uid] . noconflict_pred : string @noconflict . noindex_age : int .
Move noindex_name here as well.
query/query0_test.go, line 2620 at r2 (raw file):
{ me(func: uid(0x01)) { friend @filter(ge(survival_rate, 30)) {
Can we have a value which returns a result?
query/query1_test.go, line 84 at r2 (raw file):
} func TestFilterNonIndexedPredicateFail(t *testing.T) {
Can we change this test to execute le function at root and then that should still return an error? If we already have a test, then we don't need it.
worker/task.go, line 858 at r2 (raw file):
if needsIndex(srcFn.fnType) && !schema.State().IsIndexed(q.Attr) { if q.UidList != nil && srcFn.fnType == compareAttrFn { // UidList is not nil means this is a filter. Filter predicate is not indexed, so
needsIndex can be modified to return false for compareAttrFn and q.UidList != nil
worker/task.go, line 1610 at r2 (raw file):
} fc.eqTokens = append(fc.eqTokens, fc.ineqValue) if !schema.State().IsIndexed(attr) {
Can be refactored a bit to make clear the distinction between the logic for eq which can have multiple arguments vs other functions.
worker/task.go, line 1632 at r2 (raw file):
// compare the value with all of them. Also eq would usually have less arguments, hence we // won't be fetching many index keys. if q.UidList != nil && fc.tokens == nil {
This should be !indexed so as to avoid the extra computation when it is it is indexed but doesn't have any tokens.
There was a problem hiding this comment.
Reviewable status: 0 of 4 files reviewed, 6 unresolved discussions (waiting on @animesh2049, @mangalaman93, and @pawanrawal)
query/common_test.go, line 289 at r2 (raw file):
Previously, pawanrawal (Pawan Rawal) wrote…
Move
noindex_namehere as well.
Done.
query/query0_test.go, line 2620 at r2 (raw file):
Previously, pawanrawal (Pawan Rawal) wrote…
Can we have a value which returns a result?
Done.
query/query1_test.go, line 84 at r2 (raw file):
Previously, pawanrawal (Pawan Rawal) wrote…
Can we change this test to execute
lefunction at root and then that should still return an error? If we already have a test, then we don't need it.
Done.
worker/task.go, line 1610 at r2 (raw file):
Previously, pawanrawal (Pawan Rawal) wrote…
Can be refactored a bit to make clear the distinction between the logic for
eqwhich can have multiple arguments vs other functions.
This requires larger change. I have added a todo. Probably in some other PR.
worker/task.go, line 1632 at r2 (raw file):
Previously, pawanrawal (Pawan Rawal) wrote…
This should be
!indexedso as to avoid the extra computation when it is it is indexed but doesn't have any tokens.
Done.
There was a problem hiding this comment.
Looks generally alright -- I don't see anything obvious in terms of perf impact, but good to double check that existing indices are still being used for queries -- do that manual testing. Defer to @pawanrawal for final review and approval.
Reviewed 4 of 4 files at r3.
Reviewable status: all files reviewed, 8 unresolved discussions (waiting on @animesh2049 and @pawanrawal)
query/query0_test.go, line 2578 at r3 (raw file):
} `, `{"data":{"me":[{"noindex_salary":501.230000},{"noindex_salary":589.040000},{"noindex_salary":459.470000}]}}`,
Look in the file. If the others are also overflowing 100 chars, then OK. Otherwise, bring it to 100 chars.
worker/task.go, line 858 at r2 (raw file):
Previously, pawanrawal (Pawan Rawal) wrote…
needsIndexcan be modified to return false forcompareAttrFnandq.UidList != nil
Yeah, needsIndex is probably the right place.
worker/task.go, line 1610 at r2 (raw file):
Previously, animesh2049 (Animesh Chandra Pathak) wrote…
This requires larger change. I have added a todo. Probably in some other PR.
Please add a TODO
worker/task.go, line 923 at r3 (raw file):
if srcFn.fnType == compareScalarFn && srcFn.isFuncAtRoot { span.Annotate(nil, "handleCompareScalarFunction") if err := qs.handleCompareScalarFunction(args); err != nil {
I'll get @pawanrawal to opine on this. Is this going to cause some race issues, perhaps?
There was a problem hiding this comment.
Reviewed 4 of 4 files at r3.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @animesh2049 and @manishrjain)
worker/task.go, line 923 at r3 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
I'll get @pawanrawal to opine on this. Is this going to cause some race issues, perhaps?
Should be ok as all the functions are being run sequentially in helpProcessTask and q, srcFn, out all are pointers.
animesh2049
changed the title
There was a problem hiding this comment.
Reviewable status: 1 of 4 files reviewed, 2 unresolved discussions (waiting on @manishrjain and @pawanrawal)
query/query0_test.go, line 2578 at r3 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
Look in the file. If the others are also overflowing 100 chars, then OK. Otherwise, bring it to 100 chars.
Done.
There was a problem hiding this comment.
Reviewable status: 1 of 4 files reviewed, 2 unresolved discussions (waiting on @manishrjain and @pawanrawal)
worker/task.go, line 923 at r3 (raw file):
Previously, pawanrawal (Pawan Rawal) wrote…
Should be ok as all the functions are being run sequentially in
helpProcessTaskandq,srcFn,outall are pointers.
Done.
Fixes #4305
This change is