feat: Add Defra-Lens support for branching schema #2421

AndrewSisley · 2024-03-18T17:24:25Z

Relevant issue(s)

Resolves #2293

Description

Adds Defra-Lens support for branching schema.

Defra will now be able to handle migrations for a branching schema version DAG.

codecov · 2024-03-18T17:43:48Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.86%. Comparing base (d7931ff) to head (07c3167).

❗ Current head 07c3167 differs from pull request most recent head 915761b. Consider uploading reports for the commit 915761b to get more accurate results

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #2421      +/-   ##
===========================================
- Coverage    74.92%   74.86%   -0.06%     
===========================================
  Files          268      268              
  Lines        25888    25875      -13     
===========================================
- Hits         19395    19371      -24     
- Misses        5169     5178       +9     
- Partials      1324     1326       +2

Flag	Coverage Δ
all-tests	`74.86% <100.00%> (-0.06%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
lens/history.go	`92.86% <100.00%> (+5.23%)`	⬆️
lens/lens.go	`73.74% <100.00%> (ø)`

... and 8 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d7931ff...915761b. Read the comment docs.

fredcarle · 2024-03-19T21:34:16Z

lens/history.go

-		}
+	targetHistoryItem, ok := history[targetSchemaVersionID]
+	if !ok {
+		// If the target schema version is unknown then there are possible no migrations


typo: possibly

cheers, is actually in the wrong place, should be on the right side of no

Move word

fredcarle · 2024-03-19T21:56:09Z

lens/history.go

 	cols, err := description.GetCollectionsBySchemaRoot(ctx, txn, schemaRoot)
 	if err != nil {
 		return nil, err
 	}

-	history := map[collectionID]*schemaHistoryLink{}
+	history := map[schemaVersionID]*schemaHistoryLink{}
+	schemaVersionsByColID := map[uint32]schemaVersionID{}

 	for _, c := range cols {
 		col := c


question: (I know it's from a different PR but I might of missed it then) I don't think this necessary. Why did you decide to do this?

I have no idea, might have made sense at one point, or it might have been a brain-fart.

I'll remove it shortly before merge.

Remove odd var copy

I found out, it is because we want a pointer to it, and the way Go works means that without the copy col would always be the last col in cols.

Thanks for reminding me of this. Could you add a TODO remove when Go 1.22 comment above it please.

Ah nice, glad that will fix it - will do

Add too comment

I think I accidentally commited this in my last PR

fredcarle · 2024-03-20T18:45:33Z

tests/integration/schema/migrations/query/with_p2p_schema_branch_test.go

+	"github.com/sourcenetwork/defradb/tests/lenses"
+)
+
+func TestSchemaMigrationQueryWithP2PReplicatedDocOnOtherSchemaBranch(t *testing.T) {


thought: This test should propably be in the net replicator test no?

It fits in both directories just as well IMO - I probably prefer it here because I see it more from a schema migration perspective, and you probably see it in the net directory as you see it more from a P2P perspective.

In a sense, everything that is in the net directory also relate to some other test directory. In my mind, as soon as a test is interacting with the P2P system, it would be better placed in the net package.

tests/integration/schema/migrations/query/with_p2p_schema_branch_test.go

AndrewSisley · 2024-03-20T19:41:48Z

tests/integration/schema/migrations/query/with_p2p_schema_branch_test.go

+			},
+			testUtils.WaitForSync{},
+			testUtils.Request{
+				// Node 0 should yield results as they were defined, as the newer schema version is


todo: delete comment

delete comment

fredcarle

LGTM! Thanks for answering all my comments and questions.

AndrewSisley · 2024-03-20T20:05:36Z

LGTM! Thanks for answering all my comments and questions.

Thanks for finding a bunch of stuff (including the ones raised on discord) :)

AndrewSisley added feature New feature or request area/schema Related to the schema system labels Mar 18, 2024

AndrewSisley added this to the DefraDB v0.11 milestone Mar 18, 2024

AndrewSisley requested a review from a team March 18, 2024 17:24

AndrewSisley self-assigned this Mar 18, 2024

AndrewSisley force-pushed the 2293-schema-branch-migrations branch from 527d1e5 to 8d61158 Compare March 18, 2024 17:45

fredcarle reviewed Mar 19, 2024

View reviewed changes

AndrewSisley force-pushed the 2293-schema-branch-migrations branch 3 times, most recently from 4bc7c0c to d17eceb Compare March 19, 2024 22:24

AndrewSisley requested a review from fredcarle March 19, 2024 22:31

AndrewSisley added 4 commits March 20, 2024 10:46

Add Lens support for branching schema

36b6cd6

PR FIXUP - Fix comment wording

e054b00

PR FIXUP - Remove debugs statement

ecedd00

I think I accidentally commited this in my last PR

PR FIXUP - Add Go 1.22 todo

07c3167

AndrewSisley force-pushed the 2293-schema-branch-migrations branch from 7827019 to 07c3167 Compare March 20, 2024 14:46

fredcarle reviewed Mar 20, 2024

View reviewed changes

tests/integration/schema/migrations/query/with_p2p_schema_branch_test.go Show resolved Hide resolved

fredcarle reviewed Mar 20, 2024

View reviewed changes

tests/integration/schema/migrations/query/with_p2p_schema_branch_test.go Outdated Show resolved Hide resolved

AndrewSisley commented Mar 20, 2024

View reviewed changes

fredcarle approved these changes Mar 20, 2024

View reviewed changes

AndrewSisley added 3 commits March 20, 2024 16:21

PR FIXUP - Delete test comment

1376acf

PR FIXUP - Fix test query

915761b

PR FIXUP - Rename schema history to collection history

9270daf

AndrewSisley force-pushed the 2293-schema-branch-migrations branch from e7133c4 to 9270daf Compare March 20, 2024 20:48

AndrewSisley merged commit 839b505 into sourcenetwork:develop Mar 20, 2024
27 of 28 checks passed

AndrewSisley deleted the 2293-schema-branch-migrations branch March 20, 2024 20:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add Defra-Lens support for branching schema #2421

feat: Add Defra-Lens support for branching schema #2421

AndrewSisley commented Mar 18, 2024

codecov bot commented Mar 18, 2024 •

edited

Loading

fredcarle Mar 19, 2024

AndrewSisley Mar 19, 2024 •

edited

Loading

fredcarle Mar 19, 2024

AndrewSisley Mar 19, 2024 •

edited

Loading

AndrewSisley Mar 19, 2024

fredcarle Mar 20, 2024

AndrewSisley Mar 20, 2024 •

edited

Loading

fredcarle Mar 20, 2024

AndrewSisley Mar 20, 2024

fredcarle Mar 20, 2024

AndrewSisley Mar 20, 2024 •

edited

Loading

fredcarle left a comment

AndrewSisley commented Mar 20, 2024

feat: Add Defra-Lens support for branching schema #2421

feat: Add Defra-Lens support for branching schema #2421

Conversation

AndrewSisley commented Mar 18, 2024

Relevant issue(s)

Description

codecov bot commented Mar 18, 2024 • edited Loading

Codecov Report

fredcarle Mar 19, 2024

Choose a reason for hiding this comment

AndrewSisley Mar 19, 2024 • edited Loading

Choose a reason for hiding this comment

fredcarle Mar 19, 2024

Choose a reason for hiding this comment

AndrewSisley Mar 19, 2024 • edited Loading

Choose a reason for hiding this comment

AndrewSisley Mar 19, 2024

Choose a reason for hiding this comment

fredcarle Mar 20, 2024

Choose a reason for hiding this comment

AndrewSisley Mar 20, 2024 • edited Loading

Choose a reason for hiding this comment

fredcarle Mar 20, 2024

Choose a reason for hiding this comment

AndrewSisley Mar 20, 2024

Choose a reason for hiding this comment

fredcarle Mar 20, 2024

Choose a reason for hiding this comment

AndrewSisley Mar 20, 2024 • edited Loading

Choose a reason for hiding this comment

fredcarle left a comment

Choose a reason for hiding this comment

AndrewSisley commented Mar 20, 2024

codecov bot commented Mar 18, 2024 •

edited

Loading

AndrewSisley Mar 19, 2024 •

edited

Loading

AndrewSisley Mar 19, 2024 •

edited

Loading

AndrewSisley Mar 20, 2024 •

edited

Loading

AndrewSisley Mar 20, 2024 •

edited

Loading