Watchdog

Please note: the project WIKI documentation has been moved to the ProxySQL website

---

ProxySQL is multi-threaded daemon, with one or more threads per module, as already pointed in ProxySQL Threads.
The most critical threads are the the MySQL Threads.
Although the number of critical sections is limited, few bugs in the past caused unexpected locking issues and stalls.
For this reason, ProxySQL 1.4.4 introduces a watchdog that monitors MySQL Threads and MySQL Auxiliary Threads(if enabled). If the watchdog detects that any of these threads are missing heartbeats, it assumes they are blocked and it automatically asserts, resulting in a core dump and an automatic restart from the parent process.

Watchdog is enabled by default.
Because each thread should never block for longer than mysql-poll_timeout milliseconds (2000 by default), watchdog checks heartbeats every mysql-poll_timeout+1000 milliseconds (3 seconds by default, 2000ms + 1000ms): if restart_on_missing_heartbeats heartbeats are missing, watchdog triggers an assert.

restart_on_missing_heartbeats can only be configured in the global section of configuration file, and its default value is 10.
If restart_on_missing_heartbeats is set to 0, watchdogs will still perform checks and generates entries in the error log, but it will not assert and it will not restart proxysql.

See Configuration file for details.

Note on proxysql and gdb

The watchdog module is extremely simple and lock free, therefore it should never block. Nonetheless, the watchdog module has is own heartbeat, therefore is able to detect if proxysql was intentionally blocked, for example with a SIGSTOP signal or SIGINT signal in gdb: if this happens, it will not assert.