Support starting rest-server from inetd and similar services

[jsonn]

Output of rest-server --version

rest-server 0.10.0 compiled with go1.14.10 on illumos/amd64

What should rest-server do differently?

At the moment the rest-server must be either run as root to bind to low ports like 443 or use a high port to be able to run unprivileged. Being able to use 443 is desirable, having to run as root is not.

What are you trying to do? What is your use case?

Integration with inetd(1) solves the problem with minimal complexity in the application. Other service managers generally provide compatibility, e.g. SMF on Solaris/Illumos.

[rawtaz]

rest-server must be either run as root to bind to low ports like 443 or use a high port to be able to run unprivileged

You can run rest-server as an unprivileged user but bound to low ports by running setcap 'cap_net_bind_service=+ep' /path/to/rest-server on Linux.

Is there nothing that specific (per binary) in e.g. NetBSD?

[jsonn]

Sure, but that's Linux specific and I'm not using Linux.

[rawtaz]

Yeah, I realized that after realizing where I recognized your name from :-) So there's nothing that detailed in NetBSD? Just curious.

[jsonn]

Not easily accessible and even on Linux, it has its own problems. E.g. such a process could still abuse its privileges when it doesn't know how to drop them. So much better to not have special privileges in first place.

[wojas]

This approach will spawn one rest-server per connection. By default, restic will open 10 connections and locally I even set that to 50 for performance. It will also increase connection latency and break features like quota that assume there is only a single process running.

There are several other solutions to this problem that do not require changes to rest-server and do not have these issues. You probably considered most of these, I am listing them here as a reference for others.

CAP_NET_BIND_SERVICE

On Linux, the CAP_NET_BIND_SERVICE capability allows you to bind any port, as @rawtaz mentioned:

setcap 'cap_net_bind_service=+ep' /path/to/rest-server

HTTP server

Put an HTTP server like Caddy or Nginx in front of rest-server. Caddy also makes it easy to use Let's Encrypt certificates.

I would recommend this approach for a production service that accessible over the internet.

Docker

When you run rest-server in Docker, you can bind a low port externally and have that redirect to your service port.

iptables

You can use iptables for port redirection:

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-port 8443

BSD pf supports similar redirection.

HAproxy

Use a TCP proxy like HAproxy to redirect the port.

systemd socket activation

@fd0 mentioned plans to add systemd socket activation.

inetd with socat

If you prefer inetd, you could use socat to connect to an existing rest-server on a different port.

Untested example command:

socat - TCP4:127.0.0.1:8000

Just accept a high port

Last but not least, you could just accept that the service runs on a high port.

[tim-seoss]

The statement "This approach will spawn one rest-server per connection" is correct when the nowait keyword is used in the inetd configuration, but if the wait option is used instead e.g.

https stream tcp wait restic /usr/local/bin/restic-rest restic-rest --tls --fd 0 [...]

... then this use of inetd is equivalent to (the most basic form) of systemd socket activation (and is where the systemd socket activation option comes from).

i.e. in this case, the rest server would be passed a socket listener as file descriptor zero (as opposed to a socket which has already been accept()ed - which is what happens when nowait is used instead). This process would then be free to accept multiple connections from clients (but only one rest server process would be started, and all inbound connections would be terminated by that one process).

In this design (if desired) the rest server could also be enhanced to exit after a user-configurable period of inactivity (then inetd / systemd would restart it on the receipt of a new connection), conserving system resources.

This would be useful functionality for me personally because I'd like to automatically power-up a set of disks on connection, and power them down after the rest server exits.

n.b. I believe macOS launchd also supports a similar socket activation model (but I think it uses a slightly different API).

[wojas]

I was not aware of these wait socket passing semantics, thanks for explaining! I can see how that would be useful.

[fd0]

Can anybody maybe test the inetd implementation I've pushed to #151?

[rawtaz]

@jsonn Can you test it? :)

[tim-seoss]

I'm away from home until Monday, but I'll give it a spin then.

[fd0]

I've just tried it, it does not work. Unfortunately it's not so easy to support inetd. I've spent half an hour figuring out why the rest-server was instantly killed with a SIGPIPE. Turns out, stderr is closed by inetd, so we cannot write anything to stderr. Since the rest-server does not support logging to a file at all, we cannot tell users when errors occur.

So for now we won't support inetd.

[tim-seoss]

Sorry didn't get around to testing before you did - various things went wrong in my absence which have consumed all my time since getting home!

FWIW, since the patch mentioned above has been merged, it should be possible to use rest-server with inetd by calling the following wrapping shell script from inetd (untested):

#!/bin/sh
# Ensure stderr (file descriptor 2) is closed:
exec 2>&-
# Connect FD2 to a file instead (could instead be a fifo which is read by a logging daemon)
exec 2>> /path/to/rest-server-error.log
exec /path/to/rest-server

[jsonn]

export LISTEN_PID=$$
export LISTEN_FDS=1
exec rest-server ... 3>&0 < /dev/null > /dev/null 2> rest-server-error.log