Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Reorganise client transaction related interfaces #1180

Merged
merged 4 commits into from
Mar 17, 2023

Conversation

AndrewSisley
Copy link
Contributor

@AndrewSisley AndrewSisley commented Mar 15, 2023

Relevant issue(s)

Resolves #1161

Description

Reorganises the client transaction related interfaces so that we are not doubling up the visible function count with Txn suffix variants.

Partly designed via discord thread dev-db.Client interface refactor https://discord.com/channels/427944769851752448/1083107751435173918

Out of scope

  • Further splitting up the interfaces is out of scope. There are two main reasons for this, firstly - I see it as much less important than verifying that Schema Updates actually works, and that it is accessible to users via the CLI. Secondly, I disagree with the proposed splits that I have seen, they all have their flaws that require ironing out and given the first point I do not wish to spend time on it now.
  • Worrying about the embedding of db in both explicit and implicit implementations. Relative to the other stuff I need to do before release, I just don't care enough about this. I'm moderately happy with the current design and think it is good enough. I really don't like the proposed idea of having db and coreDb, or directly exposing db and dropping either the explicit or implicit implementation, or a circular reference variant that relies on runtime checks. If people really want one of those, I suggest it is done later, by someone who thinks it is a nice change. This PR deals with the important externally visible stuff, and that is important for 0.5, the internals are less important, and it makes no sense to have me implement something I think is a bad idea.
  • I really dislike that both txn and client.Store are required for some use cases. I find that really really horrible - planner is a good example (highly visible via planner.New). We have two objects that represent the same transaction, yet that linkage is hidden and it makes it look like the Store is not respecting the sibling txn variable. I really really want to move MultiStore onto Store so that we don't have to do this, but I see that as out of scope for now, and whilst it is horrible IMO, Schema Updates is still more important.
  • This PR includes a few new features, not all implicit/explicit function variants existed previously, and as requested they have been added in this PR. Testing them is out of scope. Testing Schema Updates is seen as more important, and is more likely to be used by users in 0.5 than the new funcs.
  • The documentation for many of the (existing) functions in client that are affected by this PR is non-existent, I see fixing that as out of scope. Documentation that did exist, should however still exist, and should not be degraded by this PR.

@AndrewSisley AndrewSisley added area/api Related to the external API component refactor This issue specific to or requires *notable* refactoring of existing codebases and components action/no-benchmark Skips the action that runs the benchmark. labels Mar 15, 2023
@AndrewSisley AndrewSisley added this to the DefraDB v0.5 milestone Mar 15, 2023
@AndrewSisley AndrewSisley requested a review from a team March 15, 2023 17:56
@AndrewSisley AndrewSisley self-assigned this Mar 15, 2023
@codecov
Copy link

codecov bot commented Mar 15, 2023

Codecov Report

Merging #1180 (11e8312) into develop (08c4b8b) will decrease coverage by 0.14%.
The diff coverage is 56.42%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #1180      +/-   ##
===========================================
- Coverage    68.77%   68.64%   -0.14%     
===========================================
  Files          180      181       +1     
  Lines        17046    17126      +80     
===========================================
+ Hits         11724    11756      +32     
- Misses        4370     4415      +45     
- Partials       952      955       +3     
Impacted Files Coverage Δ
net/peer.go 46.16% <40.00%> (-0.13%) ⬇️
db/txn_db.go 46.59% <46.59%> (ø)
db/schema.go 64.93% <87.50%> (+0.41%) ⬆️
db/collection.go 68.73% <100.00%> (+1.07%) ⬆️
db/collection_update.go 72.17% <100.00%> (ø)
db/db.go 71.42% <100.00%> (+1.18%) ⬆️
db/p2p_collection.go 60.00% <100.00%> (+4.44%) ⬆️
db/replicator.go 78.82% <100.00%> (ø)
db/request.go 88.57% <100.00%> (+26.80%) ⬆️
db/subscriptions.go 71.42% <100.00%> (-8.58%) ⬇️
... and 5 more

... and 6 files with indirect coverage changes

Copy link
Collaborator

@fredcarle fredcarle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still trying to wrap my head around the idea of having *db embedded in a struct with nothing else for the purpose of scoping methods. At the moment I'm not convinced so I'll wait to see others' opinions.

I think we definitely need to improve the naming of the two new structs and the file that contains them and ensure it describes the content.

On the issue that you raise about planner.New, it can be easily fixed if WithTxn returned a TxnStore type that looked like this

type TxnStore interface {
  Store
  datastore.Txn
}

as well as changing explicitDB to

type explicitDB struct {
  *db
  datastore.Txn
}

with a few more trivial changes.

db/implicit.go Outdated
Comment on lines 25 to 32
type implicitDb struct {
*db
}

type explicitDb struct {
*db
txn datastore.Txn
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo: implicitDB and explicitDB are misleading names. It isn't the db that is implicit or explicit but rather it is the transaction. implicitTxn and explicitTxn would be more appropriate. If you don't like those two I'm open to suggestions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @fredcarle here that names seem misleading. Perhaps there is a way to avoid the use of implicit and explicit in filename and in source? don't have any super nice suggestions atm, maybe Db and DbTxn?

Copy link
Contributor Author

@AndrewSisley AndrewSisley Mar 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem I see is that both are transactional, and both are databases. implicit is still 100% transactional (bugs aside...), it is just not an externally exposed transaction.

Accounting for the fact that these types are internal only, and so we are only ever talking about internal 'stuff', one could actually argue that implicit is more transactional than explicit, as the volume of code relating to transactions is actually higher (in implicit compared to explicit).

Copy link
Contributor Author

@AndrewSisley AndrewSisley Mar 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed in discord, will rename these to implicitTxnDB and explicitTxnDB

  • Rename types (dont forget DB with two capitals...)

db/implicit.go Outdated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: The name of this file I find is at the same time to vague and too narrow for what it contains. To content could be put in db.go instead but if you really want it in it's own file, maybe we could try to find a better name for it.

Copy link
Contributor Author

@AndrewSisley AndrewSisley Mar 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed in discord, will rename to txn_db.go

  • Rename file

@AndrewSisley AndrewSisley force-pushed the sisley/refactor/I1161-client-txnality-2 branch from 5774180 to 20caa3e Compare March 16, 2023 23:39
Store will form the basis of the new transactional split, and will be the type returned from the soon-to-be WithTxn func
Copy link
Collaborator

@fredcarle fredcarle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to go!

Copy link
Member

@shahzadlone shahzadlone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@AndrewSisley AndrewSisley merged commit 62f8c83 into develop Mar 17, 2023
@AndrewSisley AndrewSisley deleted the sisley/refactor/I1161-client-txnality-2 branch March 17, 2023 00:23
shahzadlone pushed a commit that referenced this pull request Apr 13, 2023
* Extract Store functions to new interface

Store will form the basis of the new transactional split, and will be the type returned from the soon-to-be WithTxn func

* Wrap transactionality using WithTxn
shahzadlone pushed a commit to shahzadlone/defradb that referenced this pull request Feb 23, 2024
…work#1180)

* Extract Store functions to new interface

Store will form the basis of the new transactional split, and will be the type returned from the soon-to-be WithTxn func

* Wrap transactionality using WithTxn
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
action/no-benchmark Skips the action that runs the benchmark. area/api Related to the external API component refactor This issue specific to or requires *notable* refactoring of existing codebases and components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sort out client interfaces transactionality
3 participants