wilcard-file-reader: Fix detection of file delete/move when using ivykis poll events

[HofiOne]

This PR has the following changes:

1. Fixes a crash in log pipe queue during file deletion and EOF detection

   The crash was caused by a concurrency issue in the EOF and file deletion detection when using a wildcard-file() source.

   If a file is written after being deleted (e.g. with an application keeping the file open), or if these events happen concurrently, the file state change poller mechanism might schedule another read cycle even though the file has already been marked as fully read and deleted.

   To prevent this re-scheduling between these two checks, the following changes have been made:
   Instead of maintaining an internal EOF state in the WildcardFileReader, when a file deletion notification is received, the poller will be signaled to stop after reaching the next EOF. Only after both conditions are set the reader instance will be deleted.

   Fixes: syslog-ng 3.24.1 sometimes crash in log_pipe_queue #4989

   The fix is inspired by @MrAnno's #160

2. Fixed the file deletion and removal detection when the file-reader uses poll_fd_events to follow file changes, which were mishandled. For example, files that were moved or deleted (such as those rolled by a log-rotator) were read to the end but never read again if they were not touched anymore, therefore switching to the new file never happened.

3. Added a dedicated monitor_freq option to control the poll frequency of the change detection in the directories separately when the poll method is selected via the monitor-method() option.

   The monitor-method() option controls only the change detection method in the directories, not the following of the file changes, and if poll is the selected method the frequency must not necessarily be the same, e.g. if the (earlier) commonly used follow-freq() is set to 0 for switching to the poll_fd_events method for file content change detection, that also might be meant a directory change poll with zero delays (if monitor-method() was set to poll as well), and that could cause a heavy CPU load unnecessarily.

Documented in syslog-ng/syslog-ng.github.io#101

Signed-off-by: Hofi [email protected]

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[kira-syslogng]

Build FAILURE

[HofiOne]

@kira-syslogng test this please;