Justbuild on Nix

This repository demonstrates a way to build locally with justbuild using dependencies from Nix in a clean and versioned way.

Background: Standard Paths on non-Nix Distributions

On many non-Nix Linux distributions the local tools are installed to a small number of "standard" paths (like /bin, /sbin, /usr/bin, /usr/sbin). Moreover, tools like sh (used by default by the built-in "generic" rule) and env (the default action launcher) are configured to take those standard paths into account if the PATH environment variable is not set on their startup. Hence the tools installed in those standard paths are always available for local builds, without the need of setting PATH in action definitions.

Such an approach obviously has the disadvantage that the cache has to be cleared, whenever the tools installed in those standard paths are updated. For users following a stable release with only security fixes, this can be acceptable.

While not the idiomatic way of building on Nix, such a behavior can be simulated by using as launcher a tool like withExtendedPath extending the value of the environment variable PATH by a given string, e.g., your ~/.nix-profile controlled by the home-manager. To avoid interfering with the clean Nix-idiomatic builds described in the following sections, it is recommended to also set the local build root to a different directory; remember to just gc twice whenever updating your ~/.nix-profile. Your ~/.just-mrrc thus would look something like

{ "local launcher":
  [ "/home/YOURUSERNAME/.nix-profile/bin/withExtendedPath"
  , "/home/YOURUSERNAME/.nix-profile/bin"
  ]
, "local build root": {"root": "home", "path": ".cache/just-nix-home"}
}

Such an approach can be useful when dealing with a large number of project repositories all requiring basically the same tool chain. This discussion also shows how to easily provide a well-defined remote-execution environment on Nix: local launcher and local build root can also be set on the command line when starting just execute.

Getting Nix Paths into Justbuild Actions

On Nix, the usual "standard" paths are pretty empty. Instead, all packages are installed into paths in the nix store containing a recursive hash of the full build description. So a path to the nix store brings a well-defined dependency and we're only left with the problem of getting the right paths into the actions.

Justbuild deliberately ignores any environment variables of the invocation; the environment of an actions has to be provided through the build description. When using rules-cc, the "defaults" targets can be used to set tools and paths for targets defined by a particular rule. We link those targets to the nixpkgs in a maintable way as follows.

• All the "defaults" targets simply take their values from appropriate parts of the "TOOLCHAIN_CONFIG" just configuration variable.
• The just configuration file is generated by a nix derivation using that nix derivations have easy access to the needed paths in the nix store. That derivation also generates an rc-file pointing to that configuration.
• The precise sources of the nixpkgs are pinned using niv.
• A nix shell uses the derivation at the pinned snapshot of the nixpkgs and sets an alias for just-mr to use the derived rc-file. It also adds a script withRc-just-mr that execs just-mr with the derived rc-file.

So to build with the correct dependencies for the checked out version, simply start a nix-shell at the top level of this repository and use just-mr build as usual. It should be noted that the nix-shell does not pull in justbuild itself and instead inherits it from the invoking shell; in that way, when going back to an old snapshot it is still built using the currect justbuild. The reason is that, while newer versions of justbuild can work with a local build root generated by older versions, this is not necessarily the case the other way round (e.g., versions before 1.3.0 are not aware of the large-object CAS introduced in that release, and hence will not find certain artifacts referenced in the action cache). So, by using the current justbuild when checking out older snapshots, we can reconstruct the old actions without the need of cleaning up the local build root.

The just described way of working in a nix-shell using the alias is useful for interactive development. For CI-like usage, the script run-tests can be used that simply runs withRc-just-mr in a nix-shell. Arguments are forwarded to the build command so that you can run, e.g., ./run-tests hello hello or ./run-tests -D '{"RUNS_PER_TEST": 3}'.

Shell commands

The built-in rule "generic" allows to define a target by executing a shell command (via sh -c ...). This is convenient on systems where the shell is configured to have "the standard tools" available by default. On Nix, one would have to set the "env" appropriately on every invocation. As would be quite cumbersome, preference is given to the rule ["shell", "cmds"] that is a replacement for "generic" honoring the shell toolchain (as defined in the appropriate "defaults" target).

Repository Overview

Version pinning and updating

There are two files pinning dependencies

• The nixpkgs are pinned in nix/sources.json. They can be updated using update-nix-dependencies.sh which simply calls niv update in our nix-shell.
• The dependencies on other other justbuild projects are pinned in etc/repos.json. They can be updated using update-just-dependencies.sh which calls etc/generate-repos.sh in our nix-shell, which also brings in tools required for this step.

Specific considerations

• cargo

Logical repositories

This project uses several logical repositories.

• The directories src and test provide standard main and test repositories. The logical structure is, as usual, that the test repository has access to the main repository, but not the other way round. In this way, the main repository can be imported without pulling in the test dependencies. Note that in those directories (where the main project-development work will happen) there is nothing Nix-specific, except maybe for the choice to refrain from using the built-in rule "generic".

• The rules rules/nix and rules/nix-test are derived from the rules-cc by setting the target-file layer to the directory etc/defaults. Here the toolchains are defined in a fine-granular way. The toolchains of rules/nix-test inherit from the ones in rules/nix. In this way, we can bring in additional tools only for tests. Those extra paths are set in TOOLCHAIN_CONFIG["test"]["PATH"].

This fine-granular setting of the toolchains has the effect that actions only have the paths necessary set and in that way are not unnecessarily affected by changes of the depedencies. A good example for this granularity can be seen by building verbosely

[nix-shell]$  just-mr --main test build hello '' --log-limit 5

and looking at the differnt values of the environment: PKG_CONFIG_PATH is only set in the actions calling pkg-config, only the test action has the additional test tools in PATH.

Nix dependencies

As explained, the nix-dependencies are declared in a derivation. For libraries, which probably form the majority of dependencies added, they way they become available is via PKG_CONFIG_PATH which makes them discoverable as ["CC/pkgconfig", "system_library"]. As in the buildPhase of a Nix derivation, the environment variable PKC_CONFIG_PATH is already set appropriately, it is sufficient to simply add new libraries to buildInputs. As an example, note that fmt is available without being explicitly mentioned in buildPhase.