Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Merge collection UpdateWith and DeleteWith #2531

Merged

Conversation

nasdf
Copy link
Member

@nasdf nasdf commented Apr 19, 2024

Relevant issue(s)

Resolves #2457

Description

This PR merges the DeleteWith and UpdateWith functions into a singular DeleteWithFilter and UpdateWithFilter.

Tasks

  • I made sure the code is well commented, particularly hard-to-understand areas.
  • I made sure the repository-held documentation is changed accordingly.
  • I made sure the pull request title adheres to the conventional commit style (the subset used in the project can be found in tools/configs/chglog/config.yml).
  • I made sure to discuss its limitations such as threats to validity, vulnerability to mistake and misuse, robustness to invalidation of assumptions, resource requirements, ...

How has this been tested?

make test

Specify the platform(s) on which this was tested:

  • MacOS

@nasdf nasdf added the area/api Related to the external API component label Apr 19, 2024
@nasdf nasdf added this to the DefraDB v0.11 milestone Apr 19, 2024
@nasdf nasdf self-assigned this Apr 19, 2024
@nasdf nasdf requested a review from a team April 19, 2024 00:33
Copy link
Contributor

@islamaliev islamaliev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

target any,
updater string,
) (*UpdateResult, error)

// UpdateWithFilter updates using a filter to target documents for update.
Copy link
Contributor

@AndrewSisley AndrewSisley Apr 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Just Update and Delete would be better names IMO, especially given that the filtering is optional - they might be calling UpdateWithFilter with no filter.

EDIT: Update and Delete were kept/already exist, ignore this :) Unless you want to unify them like GetCollections and GetSchemas do.

@@ -72,7 +72,15 @@ func (n *updateNode) Next() (bool, error) {
if err != nil {
return false, err
}
_, err = n.collection.UpdateWithDocID(n.p.ctx, docID, string(patch))
doc, err := n.collection.Get(n.p.ctx, docID, false)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo: Why are you fetching the value here? You already have it n.currentValue = n.results.Value()

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't worked with the planner much. Is there a way to create a client.Document from a core.Doc?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed that the types here were different, but yes you can - easiest way would probably be:

docMap := n.currentValue.ToMap()
clientDoc := client.NewDocFromMap(docMap, colDef)

There might be something nicer somewhere, but I couldn't spot anything in a 30sec scan. I prefer the double-cast over fetching twice.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a few tries but I think I found a good solution.

Copy link
Member Author

@nasdf nasdf Apr 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AndrewSisley It's not possible to update a document without fetching it first. Partial document updates break some parts of our indexing system.

It's also not possible to reconstruct the document from the planner node because it only selects the field values the user has requested be returned.

I've reverted back to fetching the document first and then updating with the patch fields.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partial document updates break some parts of our indexing system.

Thats not good, and a regression IMO. No worries though RE this PR, thanks for giving it a go :)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AndrewSisley That's exactly what UpdateWithDocID did so the pattern is the same. No regression in that sense.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The regression was introduced with sec. indexes, I'm pretty sure that up until that point we could do partial updates via any public func, so long as the docID was provided.

@nasdf nasdf changed the title refactor: Merge collection UpdateWith and DeleteWith functions refactor: Merge collection UpdateWith and DeleteWith Apr 19, 2024
Copy link
Collaborator

@fredcarle fredcarle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Another negative line contribution PR. Keep at it haha.

Just one minor todo before merge.

res, err := col.UpdateWithFilter(cmd.Context(), filter, updater)
if err != nil {
return err
}
return writeJSON(cmd, res)
case len(argDocIDs) == 1 && len(args) == 1:
docID, err := client.NewDocIDFromString(argDocIDs[0])
case len(argDocID) > 0 && len(args) == 1:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo: The use of len(argDocID) > 0 is misleading here as it suggests that argDocID could be an array of docIDs but then used as a string bellow. Please change this to argDocID != "". Same thing in the collection_delete.go file :)

}
patch, err := json.Marshal(n.input)
doc, err := client.NewDocFromMap(docMap, n.collection.Schema())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo: Now that the col vs schema PR is merged, this probably needs to change to:

doc, err := client.NewDocFromMap(docMap, n.collection)

Copy link
Contributor

@AndrewSisley AndrewSisley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Keenan :)

Copy link

codecov bot commented Apr 20, 2024

Codecov Report

Attention: Patch coverage is 58.82353% with 21 lines in your changes are missing coverage. Please review.

Project coverage is 76.20%. Comparing base (39286f1) to head (1b4c429).

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #2531      +/-   ##
===========================================
+ Coverage    75.80%   76.20%   +0.40%     
===========================================
  Files          292      292              
  Lines        28082    27755     -327     
===========================================
- Hits         21285    21149     -136     
+ Misses        5435     5248     -187     
+ Partials      1362     1358       -4     
Flag Coverage Δ
all-tests 76.20% <58.82%> (+0.40%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
cli/collection_delete.go 65.85% <100.00%> (+13.22%) ⬆️
cli/collection_update.go 64.29% <100.00%> (+18.72%) ⬆️
client/document.go 65.61% <ø> (+0.43%) ⬆️
db/collection_delete.go 37.10% <ø> (-1.53%) ⬇️
db/collection_update.go 73.50% <ø> (+0.50%) ⬆️
planner/delete.go 75.64% <100.00%> (ø)
planner/update.go 77.36% <100.00%> (+1.12%) ⬆️
http/client_collection.go 36.04% <0.00%> (+4.34%) ⬆️
http/handler_collection.go 69.37% <25.00%> (+8.75%) ⬆️

... and 7 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 39286f1...1b4c429. Read the comment docs.

@nasdf nasdf merged commit e65b164 into sourcenetwork:develop Apr 22, 2024
30 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/api Related to the external API component
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Merge UpdateWith / DeleteWith functions
4 participants