Practical pure eval for paths in non-Flake CLIs

[infinisil]

Is your feature request related to a problem? Please describe.

It's currently impractical to evaluate local paths using --pure-eval, a stable feature.

It can be done, but it needs like 20 lines of shell code to tell Nix which local paths can be used: https://gist.github.com/infinisil/6bd89c9fb5e66b635bd30fa9ae5b7bf6#file-nix-instantiate-pure-L5-L26

This gets you more or less the same --pure-eval power as the current Flake CLI.

Describe the solution you'd like

Make --pure-eval mode practically usable for nix- and nix -f CLIs by building the above functionality into the commands directly.

This is backwards compatible, because it would only change behavior when --pure-eval is enabled and paths are passed, which currently always gives an error.

Describe alternatives you've considered

A pure evaluation primop, some brainstorming for which can be seen in https://github.com/tweag/epcb.

Additional context

Slightly related is #5868, which would allow making the shell-based implementation nicer.

Priorities

Add 👍 to issues you find important.

[tomberek]

What could be done to make such a script more ergonomic? Let's take this to the logical next steps:

• user should only need to specify a reference to the directory they want and this wrapping is done for them...
• user should be able to add multiple input directories, not just one...
• we will want eval caching, so it is helpful to keep a record of the calculated hashes in a lockfile somewhere, as well as ensuring others have the same content...
• that eval caching will also have to keyed on the system and other misc arguments...
• default.nix's will need to be more standardized...
• telling everyone to do this is awkward. Would it be nice to have a single command do both? And maybe have a clean way to combine the remote reference and the requested attribute?
  1. clone [email protected]:org/thing && cd thing
  2. nix build . -A myAttr --pure-eval
  • ...or... nix-build [email protected]:org/thing^myAttr (looks familiar)
• this git cloning won't work with nixpkgs, so it needs some special handling to either be a shallow clone or use a tarball....

So we need a lockfile for inputs, a call-thing.nix wrapper in the CLI to implement purePathExpr, and enforce a more consistent standardized format in the entrypoint. I dont see how this exact set of things is not a high-level description of flakes. The steps needed to make pure-eval ergonomic lead you down a very similar set of decisions and choices as flakes. There are a few choices open to us along the way, but not many.

This gets you more or less the same --pure-eval power as the current Flake CLI.

To be blunt, that is because this is re-creating them. And making this approach ergonomic will hit nearly the exact same set of design difficulties. I see a simple text system arg is injected automatically; same outcome as flakes; awkward cross compiling or complex systems.

This wrapper being possible means the underlying Store+Nix layer can be used by various ergonomic layers on top. This is great! Such alternatives can exist! Asking to make one approach blessed means now there will be yet another incomplete and partially usable design while the details are worked on.

This is why I've been repeating the mantra: "as soon as you implement pure-eval and caching, the rest of flakes are almost forced decisions".

conclusion

I'd love to use such an approach to examine the design of a post pure-eval Nix. If there is a specific design choice that needs to be examined, let's look at it - but I don't see this approach leading to any radically different design choices so far.

[infinisil]

@tomberek I get what you're trying to say, but you're blowing this way out of proportion

What could be done to make such a script more ergonomic? Let's take this to the logical next steps:

The script is ergonomic, nix-instantiate-pure -A hello in Nixpkgs works just as you'd expect nix-instantiate --pure-eval -A hello to do, and that's it. This is useful in of itself without anything extra! Let's just accept the basic fact that pure evalutation is useful, even if there's nothing extra.

user should only need to specify a reference to the directory they want and this wrapping is done for them...

Yes you might want some kind of reference for remote references, but I'm just talking about local Nix code (though the stable CLI also supports evaluating code from some basic URLs and references).

user should be able to add multiple input directories, not just one...

No, even with Flakes there's only a single root Flake you evaluate. You're conflating this with input fetching, which are entirely orthogonal.

we will want eval caching, so it is helpful to keep a record of the calculated hashes in a lockfile somewhere, as well as ensuring others have the same content...

Expanding the scope unnecessarily, but also lockfiles have nothing to do with eval caching.

default.nix's will need to be more standardized...

No, why? I just want pure evaluation! The one standard we need is accepting { system }: as an argument, and we do have that already even in stable Nix, and it was heavily re-inforced with Flakes.

This gets you more or less the same --pure-eval power as the current Flake CLI.

To be blunt, that is because this is re-creating them.

It's re-creating one very specific feature of the Flakes CLI, one which is entirely independent of Flakes, which is the point: It shouldn't be Flake-specific.

This wrapper being possible means the underlying Store+Nix layer can be used by various ergonomic layers on top. This is great! Such alternatives can exist!

No, just scroll down to the "argument handling", this script barely works for a super limited set of arguments. This is no alternative, this script pretends to fill the gap of usability in non-Flake CLI's and --pure-eval. Both of these features exist and are stable, but not practically usable together. It shouldn't be the job of third-parties to fill gaps in Nix.

[Ericson2314]

Before we get into ergonomics, we should expose the primitive, which is: when evaluating non-flakes be able to specify an initial store path whitelist.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-build-and-pure-eval-issues/11203/6

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/is-it-wrong-for-me-to-ignore-flakes-and-home-manager-until-they-become-officially-integrated-into-nix/44026/5

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-team-member-suggests-removing-flakes-data-suggest-it-isnt-a-good-idea-please-remove-the-experimental-flag-instead/54959/69