-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CircuitBreakerOpenException at high load persisting / restoring #24
Comments
Please take a look at #22 |
Thanks @marcuslinke ! @thomasjaeckle let me know if you have more questions after looking at #22. |
Thanks to both of you, @marcuslinke and @scullxbones - I dindn't think about showing the resolved issues :)
That way I have a constant rate of ~3.000 inserts per second (all on the same dev machine) without loosing events, thanks :) I still have no clue how to investigate which exception caused the CircuitBreakerOpenException (and I'm getting lost in scala code - I use Java for Akka coding). Any tip for that? |
I took a look over the code, both in akka and in the plugin. Based on my reading, if there's no exception preceding the You can test that the timeout is the issue by setting the |
Jep, if I set |
@scullxbones Any chance to adopt the default settings accordingly? |
@marcuslinke i'm not a big fan of these becoming the default settings, mainly because you've disabled the circuit breaker. I've been convinced that fail fast using circuit breakers is the correct default model to use. No timeouts sets you up for cascading failures which can be very ugly things. For more on this, I can't recommend enough the book What I can do is add an item to the README.md about timeouts so that it's easier for people to understand what's happening, and adjust as needed. |
@scullxbones It would be great to have this documented in the README. Nevertheless as I understand these settings it doesn't disable the circuit breaker:
Or am I missing something here? |
hi @marcuslinke ... I double-checked the source code of the To summarize, |
@scullxbones Thanks for clarifying this. So
as the default? |
@marcuslinke i think those defaults are way too long to be honest. That said, this discussion brought a question to mind - i'm wondering if the circuit breaker is being applied correctly in the plugin. Do you know what the plugin was doing when you got the open breaker errors? It may be that it's not being applied at a fine enough grain on replay. It should be fine on append I think. Have you hooked up the metrics as described in the README? Is there anything interesting there? |
@scullxbones As I remember the Also I have to say that i didn't used the metrics library at all. Thats something I have on my list but haven't cared about yet. |
Hi.
I am currently evaluating this nice akka persistence plugin for mongoDB and it suits really well.
But now I ran into some problems regarding the "CircuitBreaker" pattern included.
Under high load of journal writes or journal reads (when restoring) the CircuitBreakerOpenException is thrown.
I have no clue what goes wrong (is there an exception in Casbah driver?) or how to handle this correctly.
I don't say this is wrong - I just don't know what to do with the "CircuitBreakerOpenException" at this point.
Any hints?
The text was updated successfully, but these errors were encountered: