Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Projection rebuilding via Space.jobQueue #49

Open
rhyslbw opened this issue Jan 21, 2016 · 0 comments
Open

Projection rebuilding via Space.jobQueue #49

rhyslbw opened this issue Jan 21, 2016 · 0 comments

Comments

@rhyslbw
Copy link
Member

rhyslbw commented Jan 21, 2016

The Space.jobQueue solution will include a worker for internal background tasks such as this, which should be active by default, but optional to allow these tasks to be moved off the main process. We need to coordinate the rebuild with all application instances since a critical part of the rebuild is holding new events published during the rebuild in memory, then processing them once done. All instances will need to put the nominated projections into this state, then only one instance access the repository and replay the events.

The preparation makes sense to be done either via the distributed commit store or the proposed space:message-queue, and the actual task of replaying the events should be pushed into the job queue so only one worker completes it. If the task times out or fails to complete, it will be failed or will report back as a failure, which then makes it available for another attempt. The complexity here will be knowing when the replay task can proceed, so we may need to consider the discussed option of using meteorhacks:cluster for deployment awareness.
Ensure only one application instance performs the rebuild, but others hold events until complete

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant