Unable to start nut-driver after upgrade to Fedora 37

[andrewvillano]

nut-driver.service as well as upsdrvctl is unable to start any of my snmp-ups devices individually or collectively.

These are the errors I'm seeing:

[root@xxxxx run]# upsdrvctl start nutdev7
Network UPS Tools - UPS driver controller 2.8.0
Network UPS Tools - Generic SNMP UPS driver 1.21 (2.8.0)
writepid: fopen /var/run/snmp-ups-nutdev7.pid: Permission denied
Detected Smart-UPS 2200 on host 10.100.2.49 (mib: apcc 1.6)
[nutdev7] Warning: excessive poll failures, limiting error reporting (OID = .1.3.6.1.4.1.318.1.1.1.9.2.3.1.5.1.1.3)
[nutdev7] Warning: excessive poll failures, limiting error reporting (OID = .1.3.6.1.4.1.318.1.1.1.9.3.3.1.6.1.1.1)

Fatal error: unable to create listener socket

bind /var/run/snmp-ups-nutdev7 failed: Permission denied

Exiting.
Driver failed to start (exit status=1)
[root@xxxxx run]#

Thanks in advance.

[jimklimov]

That seems to have been raised just recently on the mailing list and fedora bug tracker, and boils down to wrong packaging (no rights to write into /var/run, there should be a NUT-owned subdirectory):

• https://alioth-lists.debian.net/pipermail/nut-upsuser/2022-November/012997.html
• https://bugzilla.redhat.com/show_bug.cgi?id=2127269

Thanks to @svarshavchik for letting us know.

[jimklimov]

FYI: After a bit more digging in current codebase, I can say that NUT unprivileged daemons (drivers and upsd) default to using the ALTPIDPATH in fact -- which in turn defaults to STATEPATH e.g. /var/state/ups since both daemon types exchange pipe files there so may write there, but some package recipes configure it to e.g. /var/run/nut.

This can be seen by code lines preparing a pidfn (usually) with altpidpath() (see common/common.c) in drivers/main.c, drivers/upsdrvctl.c and server/upsd.c.

And then it gets messy in same common/common.c with:

• writepid() which optionally uses PIDPATH (as defined, no ./nut subdir) if file name is not absolute, and with
• sendsignal() which combines PIDPATH (as defined, no ./nut subdir) and the progname to wrap sendsignalfn() (which reads PID from that absolute filename and signals the process in OS-dependent manner).

These latter usages imply that PIDPATH is expected to be writable by nut processes and should not be the root-secured system location (e.g. /var/run directly) unless NUT tools and daemons run as root; can be /tmp however.

I'll post a clean-up PR to clarify this in configure script comments and docs, and "reference" init-scripts and packaging templates which use $PIDPATH/nut to confuse matters more.

Also linking to #123 which creeps out to other big discussions on PID files :)

---

Checking references to the methods mentioned above wit a focus on PIDPATH usage in current state of master branch:

:; git grep -E '(writepid|sendsignal|altpidpath) *\('

• clients/upslog.c

  • nut/clients/upslog.c

    Lines 615 to 617 in ddbab26

    	writepid(pidfilebase);
    	become_user(new_uid);

    -- writepid(pidfilebase); just before become_user() so expected to be root at that point and may use even a privileged PIDPATH location. The pidfilebase is prog name or set by argument.

• clients/upsmon.c -- the one explicitly documented consumer of configure --with-pidpath

  • nut/clients/upsmon.c

    Lines 2369 to 2392 in ddbab26

    	if (cmd) {
    	#ifndef WIN32
    	if (oldpid < 0) {
    	cmdret = sendsignal(prog, cmd);
    	} else {
    	cmdret = sendsignalpid(oldpid, cmd);
    	}
    	#else
    	cmdret = sendsignal(UPSMON_PIPE_NAME, cmd);
    	#endif
    	/* exit(EXIT_SUCCESS); */
    	exit((cmdret == 0) ? EXIT_SUCCESS : EXIT_FAILURE);
    	}
    	/* otherwise, we are being asked to start.
    	* so check if a previous instance is running by sending signal '0'
    	* (Ie 'kill <pid> 0') */
    	#ifndef WIN32
    	if (oldpid < 0) {
    	cmdret = sendsignal(prog, 0);
    	} else {
    	cmdret = sendsignalpid(oldpid, 0);
    	}
    	#else

    -- signals an older (running) instance of itself, using sendsignal(prog) (under PIDPATH) in some cases

  • nut/clients/upsmon.c

    Lines 2470 to 2488 in ddbab26

    	/* only do the pipe stuff if the user hasn't disabled it */
    	if (use_pipe) {
    	struct passwd *new_uid = get_user_pwent(run_as_user);
    	/* === root parent and unprivileged child split here === */
    	start_pipe();
    	/* write the pid file now, as we will soon lose root */
    	writepid(prog);
    	become_user(new_uid);
    	} else {
    	#ifndef WIN32
    	/* Note: upsmon does not fork in WIN32 */
    	upslogx(LOG_INFO, "Warning: running as one big root process by request (upsmon -p)");
    	#endif
    	writepid(prog);
    	}

    -- writepid(prog) just before become_user() (if running as parent-child pipe and changing privileges at all; saving child PID then), so expected to be root at that point and may use even a privileged PIDPATH location.
  • Note that if upsmon gets split into two processes, the parent part at

    nut/clients/upsmon.c

    Line 2034 in ddbab26

    #ifndef WIN32

    runs (for non-Windows case) until either the child process exits, or until it sends the message to call shutdowncmd (as root which is the purpose of this split). There is no separate PID file for the parent process.

• drivers/main.c

  • nut/drivers/main.c

    Line 903 in ddbab26

    snprintf(buffer, sizeof(buffer), "%s/%s-%s.pid", altpidpath(), progname, upsname);

    prepares buffer with absolute driver PID file name under altpidpath(), used in

    nut/drivers/main.c

    Line 918 in ddbab26

    if (sendsignalfn(buffer, SIGTERM) != 0) {

    and

    nut/drivers/main.c

    Lines 931 to 933 in ddbab26

    	if (sendsignalfn(buffer, SIGKILL) == 0) {
    	sleep(5);
    	if (sendsignalfn(buffer, 0) == 0) {

    to kill off an earlier instance if present, with full-path sendsignalfn().

  • nut/drivers/main.c

    Lines 949 to 953 in ddbab26

    	/* Only write pid if we're not just dumping data, for discovery */
    	if (!dump_data) {
    	pidfn = xstrdup(buffer);
    	writepid(pidfn); /* before backgrounding */
    	}

    calls pidfn=buffer; writepid(pidfn); to save the PID value before possibly forking and backgrounding the driver (if backgrounding is enabled).

  • nut/drivers/main.c

    Lines 1145 to 1148 in ddbab26

    	if (background_flag != 0) {
    	background();
    	writepid(pidfn); /* PID changes when backgrounding */
    	}

    calls writepid(pidfn); to save the PID value after possibly forking and backgrounding the driver (if backgrounding is enabled).
  • Note the PID file is neither saved nor queried in foreground mode (e.g. competing driver instances are not killed off either).

  • nut/drivers/main.c

    Lines 1145 to 1148 in ddbab26

    	if (background_flag != 0) {
    	background();
    	writepid(pidfn); /* PID changes when backgrounding */
    	}

    calls sendsignal() only for WIN32 usecases (so not relying on PIDPATH for POSIX builds)

• drivers/upsdrvctl.c

  • nut/drivers/upsdrvctl.c

    Lines 159 to 174 in ddbab26

    	snprintf(pidfn, sizeof(pidfn), "%s/%s-%s.pid", altpidpath(),
    	ups->driver, ups->upsname);
    	ret = stat(pidfn, &fs);
    	if ((ret != 0) && (ups->port != NULL)) {
    	upslog_with_errno(LOG_ERR, "Can't open %s", pidfn);
    	snprintf(pidfn, sizeof(pidfn), "%s/%s-%s.pid", altpidpath(),
    	ups->driver, xbasename(ups->port));
    	ret = stat(pidfn, &fs);
    	}
    	if (ret != 0) {
    	upslog_with_errno(LOG_ERR, "Can't open %s either", pidfn);
    	exec_error++;
    	return;
    	}

    tries absolute pidfn variants under altpidpath() to stop_driver() and exits if none of those is present

  • nut/drivers/upsdrvctl.c

    Lines 185 to 224 in ddbab26

    	#ifndef WIN32
    	ret = sendsignalfn(pidfn, SIGTERM);
    	#else
    	ret = sendsignal(pidfn, COMMAND_STOP);
    	#endif
    	if (ret < 0) {
    	#ifndef WIN32
    	upsdebugx(2, "SIGTERM to %s failed, retrying with SIGKILL", pidfn);
    	ret = sendsignalfn(pidfn, SIGKILL);
    	#else
    	upsdebugx(2, "Stopping %s failed, retrying again", pidfn);
    	ret = sendsignal(pidfn, COMMAND_STOP);
    	#endif
    	if (ret < 0) {
    	upslog_with_errno(LOG_ERR, "Stopping %s failed", pidfn);
    	exec_error++;
    	return;
    	}
    	}
    	for (i = 0; i < 5 ; i++) {
    	if (sendsignalfn(pidfn, 0) != 0) {
    	upsdebugx(2, "Sending signal to %s failed, driver is finally down or wrongly owned", pidfn);
    	return;
    	}
    	sleep(1);
    	}
    	#ifndef WIN32
    	upslog_with_errno(LOG_ERR, "Stopping %s failed, retrying harder", pidfn);
    	ret = sendsignalfn(pidfn, SIGKILL);
    	#else
    	upslog_with_errno(LOG_ERR, "Stopping %s failed, retrying again", pidfn);
    	ret = sendsignal(pidfn, COMMAND_STOP);
    	#endif
    	if (ret == 0) {
    	for (i = 0; i < 5 ; i++) {
    	if (sendsignalfn(pidfn, 0) != 0) {
    	upsdebugx(2, "Sending signal to %s failed, driver is finally down or wrongly owned", pidfn);

    further in stop_driver() uses the absolutely-pathed sendsignalfn() in POSIX use-cases

• scripts/Windows/wininit.c

  • two WIN32 usecases of sendsignal() (so not relying on PIDPATH for POSIX builds)

• server/upsd.c :

  • nut/server/upsd.c

    Line 1600 in ddbab26

    snprintf(pidfn, sizeof(pidfn), "%s/%s.pid", altpidpath(), progname);

    prepares pidfn under altpidpath()

  • nut/server/upsd.c

    Lines 1686 to 1697 in ddbab26

    	if (cmd) {
    	#ifndef WIN32
    	if (oldpid < 0) {
    	cmdret = sendsignalfn(pidfn, cmd);
    	} else {
    	cmdret = sendsignalpid(oldpid, cmd);
    	}
    	#else
    	cmdret = sendsignal(UPSD_PIPE_NAME, cmd);
    	#endif
    	exit((cmdret == 0) ? EXIT_SUCCESS : EXIT_FAILURE);
    	}

    makes use of sendsignal*() variants, with sendsignal() specifically called only for WIN32 usecases (so not relying on PIDPATH for POSIX builds)

  • nut/server/upsd.c

    Lines 1842 to 1853 in ddbab26

    	if (!foreground) {
    	background();
    	writepid(pidfn);
    	} else {
    	if (foreground == 2) {
    	upslogx(LOG_WARNING, "Running as foreground process, but saving a PID file anyway");
    	writepid(pidfn);
    	} else {
    	upslogx(LOG_WARNING, "Running as foreground process, not saving a PID file");
    	memset(pidfn, 0, sizeof(pidfn));
    	}
    	}

    uses writepid(pidfn) with absolute path (so not relying on PIDPATH)