Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Opinionated persistence and exactly-once storage guarantee #222

Open
taqtiqa-mark opened this issue May 31, 2022 · 1 comment
Open
Labels
feature New feature or request

Comments

@taqtiqa-mark
Copy link

Is your feature request related to a problem? Please describe.

Guaranteeing that a message (state data) is persisted exactly once.

Describe the solution you'd like
A generic preexisting solution that does not involve having to write server/client code implementing the guarantee .... NATS-JetStream

Describe alternatives you've considered
Reimplementing NATS-JS functionality inside eventually and its storage adapters.

Additional context
The idea is that eventually only support delivering aggregate data to a persistence store that can guarantee an 'exactly-once' quality of service.

While there are other options, NATS-JS does appear to offer the simplest starting point. Having delivered the data to a time and space bounded persistence store, say NATS-JS, there is the question of persisting it for times and in volumes that exceed that initial store capacities. As best I can tell, every user will come to this point regardless of what persistence options eventually offers.

There is an additional benefit in terms of interfacing with a classical SQL store. Specifically, there are NAT-JS clients in multiple languages that provide the much of the difficult implementation required to write an 'exactly-once' consumer that populates, say PostgreSQL. This relieves eventually of the burden of a providing 'exactly-once' storage adapters.

An additional (albeit small?) advantage of specializing to NATS-JS would be a simplification of the eventually code by allowing the storage detail to move from a generic parameter to an associated type?

I'll note this wall of text was prompted by a gnarly couple of days trying to make what I thought would be a simple refactor in the example app, where the compiler introduced me to aE0277 error with the suggestion the path out was by adding a self referential trait bound - having spent some time wrestling with the compiler over the repository and store I did wonder if it be simpler just to have the store as an associated type. That could be all wrong but it explains what prompted this feature request :)

Thanks for all your effort that you have put into eventually - much appreciated.

@taqtiqa-mark taqtiqa-mark added the feature New feature or request label May 31, 2022
@ar3s3ru
Copy link
Collaborator

ar3s3ru commented Dec 14, 2022

Hey @taqtiqa-mark, thank you for opening the issue and so very sorry it took me so much time to answer 🙇🏻

Just to make sure I understand, the point from your side would be:

  1. Do not use a traditional ACID database to store the aggregate state,
  2. Use a messaging system like NATS and the JetStream capability on top of it as an aggregate store,
  3. Use an ACID database as a "sink database" from the data on JetStream.

Did I get that right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants