Interactive Idempotent Nix-Build

[CMCDragonkai]

The reason why nix-build by itself doesn't have idempotent builds is because it relies on src = ./.; which has several problems.

First any change to the source directory will create a different input directory.

When nix-instantiate converts the default.nix open expression into a closed expression in drv form, then it will also encode the contents of the ./.. (The transformation from open expression to closed expression is itself interesting). Also this is a consequence of the design of Nix where the hashes of the outputs are not based on the outputs themselves but based on the input expressions.

So every time nix-build builds with ./result symlink pointing to a different place, then you get a different source state!

You can observe idempotency by doing: nix-build --no-out-link or nix-store --realise $(nix-instantiate).

To actually get idempotent builds, we need to have some way of ignoring generated files inside our src. Basically using things like filterSource and cleanSource.

There is an issue addressing this problem: NixOS/nix#885

[CMCDragonkai]

A second problem with using src = ./.; is if the directory contains large files. This can cause an out-of-memory problem.

• Large files within source tree causes build to run out of memory NixOS/nix#358

There is a PR addressing this:

• Fix adding large paths to the store NixOS/nix#619

[CMCDragonkai]

Examples of using filterSource and cleanSource:

https://github.com/coq/coq/blob/42774708bafe48aefe411fab4ca4d75407f9d1d6/default.nix#L69-L74

NixOS/nix#358 (comment)

And someone created a nix module that can simulate the usage of the .gitignore.

https://github.com/NixOS/nixpkgs/blob/4e78aeb441075872c07e6d6dc45f2045a3d87e41/lib/sources.nix#L12-L29

[CMCDragonkai]

Figured this out and I documented this in several major language demos in Gist and in Matrix AI repositories.