bgpd and logrotate (Debian 9/10)

[mkoe]

Hi

I'm using the discussion first on an issue we are currently seeing.
Could be, that we have a misconfiguration or a local issue, instead of reporting a bug straight away.
Maybe @donaldsharp, can help me out on this one, or check it, since it was disabled back in june last year,
inside the release of 7.5.

Logrotate FRR 7.3 in bgpd/bgpd_main.c

void sighup(void)
{
	zlog_info("SIGHUP received");

	/* Terminate all thread. */
	bgp_terminate();
	bgp_reset();
	zlog_info("bgpd restarting!");


	/* Reload config file. */
	vty_read_config(NULL, bgpd_di.config_file, config_default);

	/* Try to return to normal operation. */
}

Logrotate FRR 7.5 in bgpd/bgpd_main.c

void sighup(void)
{
	zlog_info("SIGHUP received, ignoring");

	return;

	/*
	 * This is turned off for the moment.  There is all
	 * sorts of config turned off by bgp_terminate
	 * that is not setup properly again in bgp_reset.
	 * I see no easy way to do this nor do I see that
	 * this is a desirable way to reload config
	 * given the yang work.
	 */
	/* Terminate all thread. */
	/*
	 * bgp_terminate();
	 * bgp_reset();
	 * zlog_info("bgpd restarting!");
	 * Reload config file.
	 * vty_read_config(NULL, bgpd_di.config_file, config_default);
	 */
	/* Try to return to normal operation. */
}

Even so that the bgpd is called afterwards again with a USR1 signal, which should result in a zlog_rotate(), does not seem to really help to release the old files, as you can see here.

The current code in 7.5 results in the following situation

lsof -p 20135
...some useless lines.....
bgpd    20135  frr   33w   REG              254,5   529830   88080729 /var/log/frr/bgpd.log.1 (deleted)
bgpd    20135  frr   34w   REG              254,5   529200   88080685 /var/log/frr/bgpd.log.1 (deleted)
.....following 485 lines with same output.....
bgpd    20135  frr  512w   REG              254,5   529200   88347844 /var/log/frr/bgpd.log.1 (deleted)
bgpd    20135  frr  513w   REG              254,5   529200   88347843 /var/log/frr/bgpd.log.1 (deleted)
bgpd    20135  frr  514w   REG              254,5   529200   88347846 /var/log/frr/bgpd.log.1 (deleted)
bgpd    20135  frr  515w   REG              254,5   529200   88347848 /var/log/frr/bgpd.log.1 (deleted)
bgpd    20135  frr  516w   REG              254,5   529200   88347850 /var/log/frr/bgpd.log.1 (deleted)
bgpd    20135  frr  517w   REG              254,5     3780   88347852 /var/log/frr/bgpd.log

So actually the logrotation with logrotate does not work anymore for us, if you are using this in your bgpd.conf log file /var/log/frr/bgpd.log
Files get moved, but SIGHUP gets ignored by the bgpd daemon, as you can see in the code snippet above,
because the files in /var/log/frr/ are not being written/owned by syslog, the SIGHUP is send to the bgpd daemon.
I guess file logging from out the daemon, instead of using syslog, is not really supported anymore with the current code.
Or is it, and it is just a missing configuration?
Or a bug now in the code?

Current configuration in the bgpd.conf log file /var/log/frr/bgpd.log
Logrotate configuration is default from your package.

It was working fine from our perspective until we upgraded to 7.5.
So we just had a incident, because the frr was running out of file handles, since it does not really release the old rotated log file.
With our current logging, we reach about 40-50 deleted Filehandles per day, which do not get released by the daemon, and this results, that the daemon is currently running every ~20 days, into the "too many open files" issue.

I would be good if you have a look at it, maybe we are not be the only ones, using a log file configuration inside the bgpd daemon.

Best regards
Manuel