NixOS evaluation speed regression

[edolstra]

Describe the bug

NixOS evaluation has gotten ~2x slower since 18.09.

To Reproduce

$ command time nix-instantiate nixos/release-combined.nix -A nixos.tests.misc.x86_64-linux --dry-run

on my system (Ryzen 3700X) takes ~1.7s on 18.09, but ~3.2s on master. And for historical perspective, it takes 0.4s on 14.04.

Also, memory usage went from 113 MiB max resident in 14.04, to 395 MiB in 18.09, and 642 MiB on master. (This is with GC_INITIAL_HEAP_SIZE=1m.)

Edit: original numbers for master were inflated by the use of cleanSource, which adds about 2 seconds to evaluation time (3182fbd). But that didn't affect Hydra.

[edolstra]

Maybe it's time to revisit NixOS/rfcs#22.

[grahamc]

I'm hopeful that Flakes will make 22/(something like 22) an easy, obvious choice for growth for 20.09.

[edolstra]

Flakes help with out-of-tree modules, but they're not necessary for modules that are in Nixpkgs but not in module-list.nix. Those can be imported using imports = [ (pkgs.path + "/nixos/modules/.../bla.nix") ].

[grahamc]

Yeah, think we could create a way for people to contribute modules somewhere organized by the community and fetched via flakes -- instead of inclusion in to NixOS. One subtlety of my previous comment I want to point out is that I don't think we can make such changes for 20.03.

[infinisil]

Related is #57477

[stale]

Hello, I'm a bot and I thank you in the name of the community for opening this issue.

To help our human contributors focus on the most-relevant reports, I check up on old issues to see if they're still relevant. This issue has had no activity for 180 days, and so I marked it as stale, but you can rest assured it will never be closed by a non-human.

The community would appreciate your effort in checking if the issue is still valid. If it isn't, please close it.

If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me". If you'd like it to get more attention, you can ask for help by searching for maintainers and people that previously touched related code and @ mention them in a comment. You can use Git blame or GitHub's web interface on the relevant files to find them.

Lastly, you can always ask for help at our Discourse Forum or at #nixos' IRC channel.

[roberth]

I've written down my thoughts in NixOS always imports all modules.

It's a low tech solution, which makes it viable. Also with flakes we'll need the same transformation to happen anyway.

[stale]

I marked this as stale due to inactivity. → More info

[infinisil]

Still a problem

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/slow-nixos-rebuild/48059/7