-
Notifications
You must be signed in to change notification settings - Fork 127
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add multithreaded Pub/Sub message processing #77
Conversation
To implement issue informatikr#49, add a new multithreaded Pub/Sub message processing feature. The main benefit over the existing Pub/Sub code (which is left unchanged) is: - you can make subscription changes at any time - you can safely recover from networking errors such as the redis servier dying - you can detect when Redis has actually processed a subscription request and handlers will now start receiving messages.
@@ -3,21 +3,39 @@ | |||
|
|||
module Database.Redis.PubSub ( | |||
publish, | |||
|
|||
-- ** Subscribing to channels | |||
-- $pubsubexpl |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
some leftover?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, I added an explanation of why there are two different pub/sub implementations. That is a haddock reference to the documentation block at the bottom of the file.
This looks good overall, thank you very much! One last minor nitpick – would be nice to add at least few happy-path tests into the test suite which runs on CI. Otherwise, I'll wait for this and other minor edits and then would be happy to merge! |
Ok, I'll work on adding some tests to the automated test suite plus the small changes tomorrow. Also, the past few days I have been starting to use this in my own project and found one minor place I would like the API to improve. Right now you can subscribe to the same channel multiple times and the controller keeps a list of handlers, but you can't unsubscribe an individual handler, you can just unsubscribe in bulk all handlers. I plan on adding some method to unsubscribe from an individual handler and only one the handler count reaches zero is the channel actually unsubscribed from redis. |
@wuzzeb all right, sounds reasonable |
- Minor fixes suggested by reviewers on github - Added ability to unsubscribe from just the handlers you register, instead of unregistering all handlers for a given channel or pattern channel name
This tests almost everything. The only thing missing from the test is proper reconnection after the redis server network connection dies, but that can't be automated.
|
remPChans = filter (\n -> HM.member n pm) pchans | ||
|
||
-- helper functions to filter out handlers that match | ||
let filterHandle :: Maybe [(UnregisterHandle,a)] -> Maybe [(UnregisterHandle,a)] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what about naming this funciton removeHandle
? It appears to do a portion of removeHandles
and doesn't work completely as filter
so the current name is a bit confusing IMO
All right, thanks a lot! Will make a release soon. |
To implement issue #49, add a new multithreaded Pub/Sub message processing feature.
I added extensive haddocks to describe how to use the new feature.
The main benefit over the existing Pub/Sub code (which is left unchanged) is:
handlers will now start receiving messages.
This likely also solves #39 since you can recover from connection failures, and while not directly implementing it, will allow the timeout in #7 since you can manually implement some timeout.
The design turned out to be relatively simple:
mainly by notifying the sending thread that a subscription change should occur.