nix command broken: massive slowdown, requires flakes-specific experimental flags on non-flakes systems

[nbraud]

Describe the bug

nix search now requires experimental features flakes and nix-command to run, and even then fails to search nixpkgs.

Steps To Reproduce

$ nix search swappy
error: experimental Nix feature 'nix-command' is disabled; use '--extra-experimental-features nix-command' to override
$ nix search --extra-experimental-features nix-command swappy
error: experimental Nix feature 'flakes' is disabled; use '--extra-experimental-features flakes' to override
$ nix search --extra-experimental-features nix-command --extra-experimental-features flakes swappy
error: cannot find flake 'flake:swappy' in the flake registries

Expected behavior

Finding swappy from nixpkgs.

nix-env --version output

$ nix-env --version
nix-env (Nix) 2.4

Additional context

I do not use flakes, nor did I configure anything related to flakes.
The issue occurs both in the system environment, and in my user's environment (ruling out it being an issue with home-manager)

[nbraud]

Apparently, the “working” command is now:

$ nix search --extra-experimental-features nix-command --extra-experimental-features flakes nixpkgs swappy`
* legacyPackages.x86_64-linux.swappy (1.4.0)
  A Wayland native snapshot editing tool, inspired by Snappy on macOS

0. Why are experimental features now required for nix search nixpkgs [...] ?
1. Why is it necessary to explicitly specify nixpkgs here, rather than having it be the default?
2. Why are the contents of nixpkgs listed as legacyPackages? Presumably, nixpkgs isn't deprecated...

[nbraud]

... apparently, this also affects nix run and presumably the rest of the nix CLI

[abathur]

#3567 #3569

[nbraud]

@abathur This doesn't address the issues I brought up?

• Remove "EXPERIMENTAL" note #3567 removes a different note, not the requirement for --extra-experimental-features ;
• Do not recommend experimental CLI in Nix Manual #3569 is about the removal of the nix command from the manual altogether.

The immediate issues at hand here are:

• The nix command (or at least several subcommands, didn't test exhaustively) suddenly started requiring a bunch of --extra-experimental-features goop on every single invocation.
• It became much slower, due to fetching and evaluating flake-registry on every single call:
  the slowdown is in excess of 30s here, which is already a major annoyance, and that's on a very-fast computer with a low-latency, 150Mbps link to the Internet; I can only imagine it is slowed into unusability for users with slower connectivity or computers.
• The output communicates that nixpkgs is somehow “legacy”, which makes no sense (flakes usually use <nixpkgs>)

More generally, I do not think the nix command should do anything flakes-related (incl. querying flake-registry) without explicit user opt-in, given that flakes are both optional and experimental.

[abathur]

@abathur This doesn't address the issues I brought up?

Correct. But the threads explain the situation.

[edolstra]

• The nix command was and is experimental, but in previous versions we didn't have a way to mark experimental features. You now need to enable experimental features explicitly. See the release notes for more info.
• The flake registry is cached and in any case shouldn't take 30s to fetch since it's a small JSON file:

  $ time nix search nixpkgs swappy --refresh
  * legacyPackages.x86_64-linux.swappy (1.3.1)
    A Wayland native snapshot editing tool, inspired by Snappy on macOS
  
  real    0m0.592s
  user    0m0.517s
  sys     0m0.015s
  

• The nixpkgs flake uses the legacyPackages flake output instead of packages because it requires nested package sets. nix search shows the full flake output attribute of the package, which you can use in nix calls like this:

  $ nix build nixpkgs#legacyPackages.x86_64-linux.swappy
  

  But you can also omit legacyPackages.x86_64-linux. since nix searches under that attribute prefix automatically.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-2-4-and-what-s-next/16257/1

[nbraud]

• The nix command was and is experimental, but in previous versions we didn't have a way to mark experimental features. You now need to enable experimental features explicitly. See the release notes for more info.

I'm aware. I'm still asking why flakes is now required for the nix command, even for users who simply do not use flakes.

• The flake registry is cached and in any case shouldn't take 30s to fetch since it's a small JSON file:

Nice to hear it “works on your machine”, but it definitely doesn't on mine: as stated above, on my machine, if nix search deems the cached information too old (or is given --refresh) it takes around 30s to download and evaluate; since this happens every time, here is a recording. Note that even without a “refresh” nix search still takes significantly longer than before.

I can't as easily make a recording of nix search prior to that change, but suffice to say it was near-instant; thus, calling this a “slowdown” (or performance regression) if you prefer. Moreover, the issue seems to affect other subcommands, such as nix run etc.

• The nixpkgs flake uses the legacyPackages flake output instead of packages because it requires nested package sets. nix search shows the full flake output attribute of the package, which you can use in nix calls like this:

  $ nix build nixpkgs#legacyPackages.x86_64-linux.swappy
  

Still. Why are those called “legacy” at all? Users will then assume nixpkgs' contents are deprecated.

[nbraud]

PS: The performance regression is worse than I thought: it takes 9s for nix run -h to run... when it should “do nothing” beyond parsing its CLI flags and printing out an error (-h isn't a valid flag)

Output from time:

real	0m9,068s
user	0m0,010s
sys	0m0,011s

And another asciinema

Testing on other commands, they all seem to have a similar slowdown on startup.

[edolstra]

The slowdown may be due to a DNS lookup at startup, see #5384.

[nbraud]

The slowdown may be due to a DNS lookup at startup, see #5384.

Even assuming a single DNS lookup is done on startup and a 10s timeout is reached, this alone doesn't explain the 30s slowdowns (or more) observed with nix search & friends. Maybe the NSS-related change is causing slowdowns on all DNS lookups?

Moreover, the configured DNS resolver is local and does not take 10s to reply; I will however install a caching stub resolver and monitor the service latencies for queries issues by nix commands, see if this is indeed the issue.

[nbraud]

@edolstra the performance regression is indeed DNS-related, as setting up a stub resolver (Stubby, in this case) “solved” it.
I didn't yet investigate what difference in behaviour (between the resolver on my LAN and Stubby) causes this.

[stale]

I marked this as stale due to inactivity. → More info

[tomberek]

Closing as this was DNS related.

Also note that the nix-command is experimental. Admitedly, there were a few versions released without that concept in existence, but we are not planning on backporting nix search and nix run to 2.3.