Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Epic] Scalar array types #100

Closed
5 of 6 tasks
jsimnz opened this issue Dec 16, 2021 · 2 comments
Closed
5 of 6 tasks

[Epic] Scalar array types #100

jsimnz opened this issue Dec 16, 2021 · 2 comments
Labels
area/crdt Related to the (Merkle) CRDT system area/schema Related to the schema system epic Includes sub-issues feature New feature or request

Comments

@jsimnz
Copy link
Member

jsimnz commented Dec 16, 2021

At the moment Defra doesn't support scalar array/list types. Which is a notable missing feature.

The primary reason for this was waiting for a proper List/Array CRDT type to use for the array, otherwise it would need to be implemented using the Last-Write-Wins register, and that imposes some challenges for updating a list, like having to specify the entire array whenever changes are made.

We can get a PoC implementation going so there's a reference we can improve on, but the correct approach requires a List/Array MerkleCRDT.

@jsimnz jsimnz added feature New feature or request area/schema Related to the schema system area/crdt Related to the (Merkle) CRDT system labels Dec 16, 2021
@AndrewSisley
Copy link
Contributor

Some notes:

  • Currently updates stick to JSON merge patch, which doesn't appear to support recursive indexing - meaning to stick to it arrays would need to be fully specified. Or we will need to break from JSON merge patch.

I can see a few simple implementation options here, that are not exclusive and we could add one at a time:

  1. standard lwwr with full replace (JSON merge patch compliant) - horrible for quite a few use-cases though as noted in description, but much easier for others
  2. point based patching - specify an index and value and lww on that index
  3. range based patching - same as above but with a range
  4. resize operations (maybe end only for now, or things might get complicated?)

Suggest we implement (1) to start with, then add the others (plus some) later? Could be handy to know verse's use-cases to know what to prioritize/get-ideas-for-other-fanciness

@AndrewSisley AndrewSisley self-assigned this Jan 12, 2022
@AndrewSisley AndrewSisley added the epic Includes sub-issues label Jan 12, 2022
@AndrewSisley AndrewSisley changed the title Scalar array types [Epic] Scalar array types Jan 12, 2022
@AndrewSisley AndrewSisley added this to the DefraDB v0.3 milestone Feb 8, 2022
@AndrewSisley AndrewSisley removed their assignment Feb 8, 2022
@jsimnz jsimnz modified the milestones: DefraDB v0.3, DefraDB v0.4 May 17, 2022
@jsimnz jsimnz modified the milestones: DefraDB v0.4, DefraDB v0.5 Nov 1, 2022
@jsimnz jsimnz modified the milestones: DefraDB v0.5, DefraDB v0.6 Mar 3, 2023
@fredcarle fredcarle modified the milestones: DefraDB v0.6, DefraDB v0.7 Aug 24, 2023
@fredcarle fredcarle removed this from the DefraDB v0.7 milestone Jan 4, 2024
@AndrewSisley
Copy link
Contributor

Remaining item is hackathon item, closing epic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/crdt Related to the (Merkle) CRDT system area/schema Related to the schema system epic Includes sub-issues feature New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants