-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The current out-of-memory support is unreliable #96
Comments
This is probably going to be in Fil for Pipelines, a paid product, but still going to need to add some infrastructure here to support hooking that in. |
And, after further thought, can actually probably be done reasonably in open source version, albeit less reliably. |
A better mechanism:
OOM handling actually has two modes we'd like:
|
Currently we do two things:
malloc()
, free up 16 previously-allocated 16MB, which in theory gives us memory for step 2.The problems with this are:
free()
ed. It may never have been reserved meaningfully at all if it's all zeros; this can be fixed, but even then when freed some other program might grab it.malloc()
fails the computer might be locked up to the point of being unusable due to swapping and the like.I expect to be ripping out the 16MB thing in #95, probably, because it's yet another
if
statement to slow things down, and it's not clear it adds anything.Some potential solutions:
mmap()
with lazy disk syncing). If memory runs out, the necessary info will still be on disk and a report can be generated post-crash.The text was updated successfully, but these errors were encountered: