gpg-agent user services do not work anymore, making it unusable as a ssh-agent

[dwagenk]

Describe the bug

gpg-agent stopped working recently. The systemd user service that is supposed to exist if programs.gnupg.agent.enable = true; fails to start. This does not seem to be relavant for most gpg actions, since it will autostart the agent if needed (albeit possibly missing the pinentry program configuration?), but it breaks usage of gpg-agent as ssh-agent.

Steps To Reproduce

Steps to reproduce the behavior:

1. activate gpg-agent with ssh support, e.g. with this snippet in the system configuration:

  programs = {
    ssh.startAgent = false;
    gnupg.agent = {
      enable = true;
      enableSSHSupport = true;
    };
  };

3. build and switch to new system configuration
4. inspect the systemd user service

systemctl status --user gpg-agent
× gpg-agent.service - GnuPG cryptographic agent and passphrase cache
     Loaded: loaded (/etc/systemd/user/gpg-agent.service; linked-runtime; preset: enabled)
    Drop-In: /nix/store/0ha5ynjimnl1fv83zs7npshcsskjhq8l-user-units/gpg-agent.service.d
             └─overrides.conf
     Active: failed (Result: exit-code) since Tue 2023-01-31 15:12:45 CET; 15min ago
   Duration: 1ms
TriggeredBy: ● gpg-agent.socket
             ● gpg-agent-ssh.socket
       Docs: man:gpg-agent(1)
    Process: 354707 ExecStart=/nix/store/2ixfs8a6jb6i9c89maniy4ilhw9mszcz-gnupg-2.3.7/bin/gpg-agent --supervised --pinentry-program /nix/store/36a1j2mws4di1c766lcxlf3rjpbsk8j5-pinentry-1.2.0-gtk2/bin/pinentry (code=exited, status=2)
   Main PID: 354707 (code=exited, status=2)
        CPU: 1ms

Jan 31 15:12:45 devpool039 systemd[1587]: Started GnuPG cryptographic agent and passphrase cache.
Jan 31 15:12:45 devpool039 systemd[1587]: gpg-agent.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Jan 31 15:12:45 devpool039 gpg-agent[354707]: gpg-agent[354707]: WARNING: "--supervised" is a deprecated option
Jan 31 15:12:45 devpool039 gpg-agent[354707]: gpg-agent (GnuPG) 2.3.7 starting in supervised mode.
Jan 31 15:12:45 devpool039 gpg-agent[354707]: no LISTEN_PID environment variable found in --supervised mode (ignoring)
Jan 31 15:12:45 devpool039 gpg-agent[354707]: no LISTEN_FDS or LISTEN_FDNAMES environment variables found in --supervised mode (assuming 1 active descriptor)
Jan 31 15:12:45 devpool039 gpg-agent[354707]: Fatal: file descriptor 3 must be valid in --supervised mode if LISTEN_FDNAMES is not set
Jan 31 15:12:45 devpool039 systemd[1587]: gpg-agent.service: Failed with result 'exit-code'.

4. try listing ssh keys

ssh-add -l
Error connecting to agent: No such file or directory

Expected behavior

gpg-agent systemd user service starts sucessfully. SSH can use it as an ssh-agent.

Additional context

The missing gpg-agent does not seem to be relevant for performing gpg tasks, since it get's autostarted by gpg when needed. But for using it as an ssh-agent this does not happen.

Upstream has removed the example systemd user units in https://dev.gnupg.org/rGeae28f1bd4a5632e8f8e85b7248d1c4d4a10a5ed , the related ticket https://dev.gnupg.org/T6336 gives some background information:

Actually, the entire systemd based launching is deprecated and thus the logged warning is on purpose.

The problem with the systemd launched gpg-agent is that it creates a race: gpg launches gpg-agent as needed and to avoid concurrent launching by other gpg or gpgsm processes, it takes a file system lock during the launch process. systemd does not know about this and we end up with sometimes end up with two gpg-agent processes. Eventually one of those processes detects that it does not own the socket and terminates itself. No real harm here but you may see smart card lockups or a flushed password cache.

I'm not sure if this applies here at all. My interpretation is that abovementioned race condition is relevant if the systemd user unit is socket activated. If it is started on login gpg should detect the already running agent reliably and not start another one, so that should be fine.

I read https://github.com/NixOS/nixpkgs/blob/1c02dbee4726f9cde21cd103cc1e56dd2b20d72e/nixos/modules/programs/gnupg.nix#L47#L50 as "with ssh support the service is started on login, without it it's socket activated", but I did not find the corresponding implementation. I guess it's actually still socket-activated, but since https://github.com/NixOS/nixpkgs/blob/1c02dbee4726f9cde21cd103cc1e56dd2b20d72e/nixos/modules/programs/gnupg.nix#L141#L145 configures environment.extraInit causing gpg to be used very early in a session and the agent is then setup.

I don't know what the correct solution is, since upstream discourages using the systemd user units and I just want my gpg as ssh-agent back.

Notify maintainers

@fpletz @vrthra @mrcjkb @rnhmjoj @SuperSandro2000 @pennae

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.90, NixOS, 22.11 (Raccoon), 22.11.20230130.f413457`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.1`
 - nixpkgs: `/nix/store/5hf3g9wlhv2y6fbzxfrmrkics88g0pv0-source`

[rnhmjoj]

gpg-agent stopped working recently

Can you be more specific? If you mean this is an issue on 22.11 I can't reproduce it, I don't know about unstable.

albeit possibly missing the pinentry program configuration?

Yes, that's the most annoying issue with gpg-agent. As far as I know, there is no system-wide configuration file, so the user service was necessary to set the pinentry program.

gpg launches gpg-agent as needed and to avoid concurrent launching by other gpg or gpgsm processes, it takes a file system lock during the launch process. systemd does not know about this and we end up with sometimes end up with two gpg-agent processes

Well, I would have gone the other way around: deprecating the automatic starting of the agent that has been a constant source of pain, but go figure.

So, I guess we have implement the services and socket in NixOS, or maybe we could use gpgconf in a user activation script.
The second option is intriguing because it's much less work but I don't know how it works exactly, I'll have to look into it.

[rnhmjoj]

I have set up a branch using gpgconf here and it seems to be working, so far. I should write a NixOS test to be sure, though.

[SuperSandro2000]

You cannot use the gpg agent with ssh support and the normal ssh agent.

Also are you changing the gpg home directory (.gnupg) due to weird expectations in gnupg this is not supported if you want to use the agent.

albeit possibly missing the pinentry program configuration?

Yes, this is expected.

[SuperSandro2000]

I am confused. Is this issue about a problem you have or about the upcoming change?

[dwagenk]

I am confused. Is this issue about a problem you have or about the upcoming change?

This issue is about the NixOS module for gnupg with NixOS version 22.11

You cannot use the gpg agent with ssh support and the normal ssh agent.

correct, the ssh-agent provided by ssh itself is dsiabled (programs.ssh.startAgent = false), this is solely about gpg agent with ssh support.

Also are you changing the gpg home directory (.gnupg)

no, this is just left at the default values

Well, I would have gone the other way around: deprecating the automatic starting of the agent that has been a constant source of pain, but go figure.

I agree, daemons and agents should be mainly configured by the distro, but well, the gnupg project does it this way.

I have set up a branch using gpgconf here and it seems to be working, so far. I should write a NixOS test to be sure, though.

I'll give it a try. I don't like how it will cause files in the $GPGHOME and thus the users home-directory to be written though. My system is working currently, because I moved my gpg config to home-manager, for which this approach works fine, but NixOS itself having to write config files into the users home directory does not seem right.

home-manager specific details

home manager also ships the deprecated user services, but that's not what makes it work. It works by magic of

home.sessionVariablesExtra = optionalString cfg.enableSshSupport ''
        if [[ -z "$SSH_AUTH_SOCK" ]]; then
          export SSH_AUTH_SOCK="$(${gpgPkg}/bin/gpgconf --list-dirs agent-ssh-socket)"
        fi
      '';

and thus causing the gpg-agent to be started during shell initialization with the following output if the agent is not yet running

gpg-connect-agent: no running gpg-agent - starting '/nix/store/2ixfs8a6jb6i9c89maniy4ilhw9mszcz-gnupg-2.3.7/bin/gpg-agent'
gpg-connect-agent: waiting for the agent to come up ... (5s)
gpg-connect-agent: connection to the agent established

[SuperSandro2000]

This issue is about the NixOS module for gnupg with NixOS version 22.11

FYI any updates or fixes will likely not be backported unless they are pretty severe and have a low chance to introduce other breaking changes.

I agree, daemons and agents should be mainly configured by the distro, but well, the gnupg project does it this way.

I have reached out to upstream about the change that they removed the systemd files and we are exploring the options but we don't have to many because upstream is missing some options we would like.

NixOS itself having to write config files into the users home directory does not seem right.

We usually don't do this and I think we still have better options here.

[rnhmjoj]

NixOS itself having to write config files into the users home directory does not seem right.

It's rare but non unheard of, for example the syncthing module does exactly this.

We usually don't do this and I think we still have better options here.

What are you thinking?

Anyway, I wrote a pretty comprehensive test for gnupg and openssh. You should find it useful for experimenting.

[Frontear]

Ancient issue but it could probably be closed, unless OP still has issues.

I havent seen/had any issues in the last many months of using gpg-ssh-agent

[rnhmjoj]

I don't think it's relevant anymore: we now have our own systemd units in nixpkgs, use the system-wide configuration to set up pinentry program (so even if gnupg autostarts the agent it will work) and I've never seen that error about --supervised. Anyway, here is the gnupg test: it passed, so gnupg-agent, including SSH support, is working fine.