Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Refactor commit nodes #892

Merged
merged 18 commits into from
Oct 14, 2022

Conversation

AndrewSisley
Copy link
Contributor

@AndrewSisley AndrewSisley commented Oct 12, 2022

Relevant issue(s)

Resolves #858

Description

Refactors the commit nodes, reducing their number from 4, to 1.

Strongly recommend reviewing commit by commit, as there is quite a lot going on in this PR and I hope it is a bit easier to see the functional equivalence at commit-level, as well as the reasons for the change.

Was tempted to rename dagScan to commitNode or similar now, but laziness won out - let me know your thoughts!

Node no longer really does anything and only serves to misdirect and confuse
As well as being clearer IMO in it's own right, this will make life easier shortly
Is much easier if currentCid stays local.  Had to add a couple of checks for when cid is provided as otherwise it starts to take a different (and incorrect) code-path, will likely be removed/optimized shortly.
Also fixes an explain bug where it was incorrectly reporting field as 'C'
A bit of a nitpick, but it is also now located next to where it is used/returned
Doesnt actually do anything anymore
@AndrewSisley AndrewSisley added refactor This issue specific to or requires *notable* refactoring of existing codebases and components action/no-benchmark Skips the action that runs the benchmark. area/planner Related to the planner system labels Oct 12, 2022
@AndrewSisley AndrewSisley requested a review from a team October 12, 2022 23:13
@AndrewSisley AndrewSisley self-assigned this Oct 12, 2022
@AndrewSisley AndrewSisley added this to the DefraDB v0.4 milestone Oct 12, 2022
It no longer served a purpose
@codecov
Copy link

codecov bot commented Oct 12, 2022

Codecov Report

Merging #892 (f2d5e84) into develop (2ee445e) will decrease coverage by 0.09%.
The diff coverage is 84.97%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #892      +/-   ##
===========================================
- Coverage    59.78%   59.68%   -0.10%     
===========================================
  Files          157      156       -1     
  Lines        17423    17306     -117     
===========================================
- Hits         10416    10329      -87     
+ Misses        6066     6042      -24     
+ Partials       941      935       -6     
Impacted Files Coverage Δ
query/graphql/mapper/commitSelect.go 0.00% <ø> (ø)
query/graphql/mapper/select.go 37.50% <ø> (ø)
query/graphql/planner/explain.go 63.79% <ø> (-5.18%) ⬇️
query/graphql/planner/select.go 78.32% <75.00%> (+0.96%) ⬆️
query/graphql/planner/commit.go 84.44% <84.75%> (-6.57%) ⬇️
query/graphql/parser/commit.go 80.21% <100.00%> (ø)
query/graphql/parser/query.go 79.43% <100.00%> (ø)
query/graphql/planner/planner.go 77.21% <100.00%> (-0.15%) ⬇️
query/graphql/planner/scan.go 88.03% <100.00%> (ø)
... and 2 more

Bit weird the original way, not sure why I did that :)
Copy link
Collaborator

@fredcarle fredcarle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm quite happy with what I see here. Only one non blocking suggestion. You might want to wait on another review from someone who is more opinionated then me on that part of the code base. Up do you.

Comment on lines 229 to 231
newqueue := make([]*cid.Cid, len(heads)+len(n.queuedCids))
copy(newqueue[len(heads):], n.queuedCids)
n.queuedCids = newqueue
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (non-blocking): this can be simplified to

n.queuedCids = append(make([]*cid.Cid, len(heads)), n.queuedCids)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the suggestion looks incorrect to me - that would result in [a, b, c, nil, nil] instead of the (desired) [nil, nil, a, b, c] no?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

@AndrewSisley AndrewSisley Oct 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhh I see I misread - cheers!

  • do this

if len(n.queuedCids) > 0 {
currentCid = n.queuedCids[0]
n.queuedCids = n.queuedCids[1:(len(n.queuedCids))]
} else if n.parsed.Cid.HasValue() && n.parsed.DocKey == "" {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: I remember discussing the use of the parsed values instead of the duplicated fields and I'm happy to see that you made the change :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cheers, was a good suggestion and cuts a lot of the weirdness/effort out regarding some of the conversions.

@AndrewSisley
Copy link
Contributor Author

I'm quite happy with what I see here. Only one non blocking suggestion. You might want to wait on another review from someone who is more opinionated then me on that part of the code base. Up do you.

Cheers! Will try leave it for a few more hours, but with most people off/otherwise-occupied atm I'm not sure it will get anymore attention. (is also just a refactoring with reasonable and growing test coverage)

@AndrewSisley AndrewSisley merged commit 4202950 into develop Oct 14, 2022
@AndrewSisley AndrewSisley deleted the sisley/refactor/I858-refactor-commitNode branch October 14, 2022 21:32
shahzadlone pushed a commit to shahzadlone/defradb that referenced this pull request Feb 23, 2024
* Remove headsetScanNode

Node no longer really does anything and only serves to misdirect and confuse

* Make cid an option-type

As well as being clearer IMO in it's own right, this will make life easier shortly

* Use parsed.Cid over duplicate prop

* Dont store iteration state

Is much easier if currentCid stays local.  Had to add a couple of checks for when cid is provided as otherwise it starts to take a different (and incorrect) code-path, will likely be removed/optimized shortly.

* Remove legacy comment

* Use parsed.Depth over duplicate prop

* Use parsed.DocKey over duplicate prop

* Use parsed.Field over duplicate prop

Also fixes an explain bug where it was incorrectly reporting field as 'C'

* Simplify commit node constructor

* Set currentValue only when valid

A bit of a nitpick, but it is also now located next to where it is used/returned

* Replace legacy comment with correct comment

* Add cid related comment

* Remove type-unsafe list with slice magic

* Remove odd comment

* Remove commitSelectNode

Doesnt actually do anything anymore

* Remove commitSelectTopNode

It no longer served a purpose
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
action/no-benchmark Skips the action that runs the benchmark. area/planner Related to the planner system refactor This issue specific to or requires *notable* refactoring of existing codebases and components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Commits: Refactor commit node/dag-fetcher
2 participants