Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Enable filtering doc by fields of JSON and Blob types #2841

Merged
merged 3 commits into from
Jul 17, 2024

Conversation

islamaliev
Copy link
Contributor

Relevant issue(s)

Resolves #2840

Description

Fixes the issue not being able to filter by JSON and Blob fields

@islamaliev islamaliev requested a review from a team July 16, 2024 14:06
@islamaliev islamaliev self-assigned this Jul 16, 2024
@islamaliev islamaliev added the area/query Related to the query component label Jul 16, 2024
@islamaliev islamaliev added this to the DefraDB v0.13 milestone Jul 16, 2024
Copy link

codecov bot commented Jul 16, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.31%. Comparing base (fe9fae6) to head (2e61ab0).

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2841      +/-   ##
===========================================
+ Coverage    79.22%   79.31%   +0.09%     
===========================================
  Files          323      323              
  Lines        24499    24661     +162     
===========================================
+ Hits         19408    19558     +150     
- Misses        3695     3701       +6     
- Partials      1396     1402       +6     
Flag Coverage Δ
all-tests 79.31% <100.00%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
internal/request/graphql/schema/manager.go 98.23% <100.00%> (+0.10%) ⬆️
internal/request/graphql/schema/types/base.go 100.00% <100.00%> (ø)

... and 12 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fe9fae6...2e61ab0. Read the comment docs.

@@ -33,8 +33,8 @@ var (
client.FieldKind_NILLABLE_STRING: gql.String,
client.FieldKind_STRING_ARRAY: gql.NewList(gql.NewNonNull(gql.String)),
client.FieldKind_NILLABLE_STRING_ARRAY: gql.NewList(gql.String),
client.FieldKind_NILLABLE_BLOB: schemaTypes.BlobScalarType(),
Copy link
Contributor

@AndrewSisley AndrewSisley Jul 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Why have you made this change? This was done quite recently as these are mutated by the gql library and making them singletons results in shared mutated state between database instances.

Is this change, and the one at the top of manager.go required for the fix?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is done to follow the same style that is used by github.com/sourcenetwork/graphql-go: to have 1 scalar definition instance instead of creating always a new.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is wrong (and graphql-go), and we should not do it. As well as being a production bug, it prevents us from using t.Parallel() in our tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have issues with t.Paralel() we should change how we use gql.String and others. I just made it consistent.

I tried to roll this part back and see if it works. It doesn't because there is a pointer comparison https://github.com/sourcenetwork/graphql-go/blob/master/schema.go#L295

Seems like the change is required for the fix.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not just about t.Parallel(). graphql-go is very singleton heavy, and it mutates the singleton types we provide it. This means multiple embedded Defra instances leak GQL types between them, instance1 will have bits of instance2's gql types in its GQL schema.

It doesn't because there is a pointer comparison

You need to only call schemaTypes.BlobScalarType() once per database instance, and fetch it from the type map where-ever else you require it. All our other types do this.

Copy link
Contributor

@AndrewSisley AndrewSisley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes, as I think you have introduced a regression.

@islamaliev
Copy link
Contributor Author

Requesting changes, as I think you have introduced a regression.

where? how?

Copy link
Member

@nasdf nasdf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Nice work!

},
},
testUtils.Request{
// the filtered-by JSON has no spaces, because this is now it's stored.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought: if we normalize JSON when parsing we could avoid the space issue and also match out of order properties.

Copy link
Contributor

@AndrewSisley AndrewSisley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for fixing this Islam :)

@islamaliev islamaliev merged commit f6b6485 into sourcenetwork:develop Jul 17, 2024
40 of 42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/query Related to the query component
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Filtering by JSON field returns "Unknown field" error
3 participants