-
-
Notifications
You must be signed in to change notification settings - Fork 145
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add document change event broadcast using members list #189
Conversation
ec27bbb
to
a5ed817
Compare
1. Improvements so that each agent can simultaneously send requests and conduct tests. 2. Added handling of `negative WaitGroup counter` errors that appear intermittently in stress tests. 3. Fixed a situation where change reject occurred in the test.
Lines 110 to 136 in a1b8407
Port conflicts occur when multiple packages use this function because it recompiles when the package being tested is replaced. I am looking for a way to use ports stably to run multiple servers.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your contribution.
I left a few simple questions.
yorkie/backend/sync/etcd/pubsub.go
Outdated
docEvent, err := converter.ToDocEvent(event) | ||
if err != nil { | ||
log.Logger.Error(err) | ||
return |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there any reason we broke the error chain here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't think it was necessary to return an error from the API where a particular Document event occurred just because it couldn't broadcast to another agent.
So I left an error log at the time of occurrence. This is because we thought that document sync could be synced through other sync events.
I have been thinking about how to alert when a problem that can occur within the method (such as a convert failure or an abnormal problem in the target agent) occurs continuously, but I was not sure, so I tried to suggest it later.
Can you please let me know if there are more things I should consider?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When we try to convert an event type that is not supported, an error occurs, and there is no need to loop further in the caller. I wish we could handle errors explicitly.
1. Improved 'watch document across agents test'. 2. Remove cluster stress test. 3. Cleanup code. 4. Change service's name Broadcast to Cluster. 5. Remove grpc recovery
Codecov Report
@@ Coverage Diff @@
## main #189 +/- ##
==========================================
- Coverage 61.58% 60.36% -1.22%
==========================================
Files 41 42 +1
Lines 3454 3550 +96
==========================================
+ Hits 2127 2143 +16
- Misses 1131 1211 +80
Partials 196 196
Continue to review full report at Codecov.
|
1. Improve lock usage when accessing cluster server clients. 2. Improved error handling. 3. Remove duplicate lock calls.
The DocEvent and Event types seem to overlap each other. Integrated these two types with each other and improved the logic that previously used DocEvent.
Broadcasting should also stop when the parent context terminates.
- Remove Members from pubsub - Remove code that uses two locks together - Change the key of clusterClientMap to id - Cleanup comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for your work. In particular, changing the topic to multi-key based in PubSub seems to be neat.
Broadcast seems to have many weaknesses compared to Gossip-based algorithms, but first of all, it seems to be able to operate a small-scale cluster.
(I made a few changes during the review.)
Co-authored-by: Hackerwins <[email protected]>
What this PR does / why we need it:
It broadcasts document change events to replicated agents.
Add grpc_recover.Added issue
example)
Which issue(s) this PR fixes:
Address #183
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
Additional documentation:
Checklist: