Add a warning before a process is killed

[NicoWeio]

How about a feature that would warn me if the free memory gets close to the minimum? This way I could first try stopping the process myself.

[revast]

+1 for that. It would be cool to have a proper warning that earlyoom will act soon, so that the user can act before earlyoom must act (and data loss could happen)

If it's too late though (e.g. a newly started process is eating up all resources and system would freeze in an instant), it would be best to kill instantly - or is there a way to stop a process with SIGSSTOP which would prevent data loss?

Maybe something like a staggered warning would make sense, If you approach say 20% above the threshold where earlyoom acts, it sends out a warning (providing the warning messages are configured, that is). then next at 15, 10 5%...

[NicoWeio]

@revast Glad you like the idea.
I just thought of "warning" the processes, too. As far as I know, there are flags that would tell a process to stop, but don't kill it instantly. That might help with preventing data loss, too.

[joeytwiddle]

I believe the original idea behind earlyoom was to perform a SIGKILL, much like the standard oomk does, but just do it a bit earlier, before the machine has ground to a halt. So some of these ideas might be out of that scope.

But that's an interesting idea to perform a SIGTERM at an earlier threshold. It might still need to be followed by a SIGKILL later, if the process doesn't close itself as requested.

Also an interesting idea to perform a SIGSTOP to allow time for user interaction.

I suspect rfj would want any user interaction to happen in an entirely different process. But theoretically these proposals could be hidden behind options, and the daemon and UI process could communicate through log messages and signals, or some other kind of API.

If the user does decide to keep a memory-hogging process running, what would happen next? Would the user be offered the next-best process for destruction, or would earlyoom disable itself for a while?

[hakavlad]

@rfjakob
What about low memory warnings? It can be implemented like SIGTERM - the addition of one more threshold.

[cinatic]

i installed earlyoom on arch developing ios in virtualbox (i had sometimes freezes) and it has unexpectatly straight killed my virtual box :| it might be a good idea to restructure the readme.md and make a clear bold warning at the top, anyhow a interesting project i wonder how it will corelate with the oom from facebook

[rfjakob]

@cinatic just FYI oomd from facebook needs kernel patches, earlyoom works on stock kernels

[betcher]

+1, I need low memory warning too.
-m 15,10,5 is a good idea imho.

[hakavlad]

You can offer a process that will be killed when memory is low.

Dialog with zenity - https://help.gnome.org/users/zenity/3.22/list.html.en

zenity --checklist --text="Mem Available: 10% (300 MiB). \nSelect a process to be terminated:" --ok-label "Terminate" --cancel-label "Do nothing" --separator=" "  --width=350 --height=250 --list \
  --title="Low memory!" \
  --column "" --column="Pid" --column="Name" --column="VmRSS" \
    9923 9923 Python3 "800 MiB" \
    2938 2938 tail "400 MiB" \
    3938 3938 chromium "300 MiB"

@rfjakob

[fillest]

Such feature would duplicate any real monitoring alerting system (e.g. kapacitor). Which you should use anyway (if we talk about servers). Meanwhile the extra logic increases the risk of introducing bugs and complicates code maintenance,
Just my 2¢ - probably the logic is trivial enough - though I would like earlyoom to be as reliable as possible :)

[rfjakob]

I don't think I want earlyoom to ask for user input. But having a third threshold for a warning could work without introducing too much complexity.

[revast]

Third threshold sounds good for me. Configurable (on/off, that is) warning via notify-send would be nice.

[hakavlad]

@rfjakob When do you plan to make the third threshold?

[rfjakob]

This will take a month or two - want to go for it?

[hakavlad]

It might look like this:

[rfjakob]

After earlyoom.service no longer runs as root, notifications are pretty difficult now.

I plan to have make earlyoom less aggressive in killing stuff, but I won't a low memory warning.

[hakavlad]

After earlyoom.service no longer runs as root, notifications are pretty difficult now.

@rfjakob No, it is not difficult with earlyoom-root.service, see #169.

@NicoWeio BTW, you can use nohang if you need low memory warnings.