From 13b485749b3d9ba76ada3a6841b7120d864e74b4 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Tue, 9 Jul 2019 21:03:08 +0200 Subject: [PATCH 01/41] Flakes RFC --- rfcs/0000-flakes.md | 660 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 660 insertions(+) create mode 100644 rfcs/0000-flakes.md diff --git a/rfcs/0000-flakes.md b/rfcs/0000-flakes.md new file mode 100644 index 000000000..6b67e15b1 --- /dev/null +++ b/rfcs/0000-flakes.md @@ -0,0 +1,660 @@ +--- +feature: flakes +start-date: 2019-06-25 +author: Eelco Dolstra +co-authors: TBD +shepherd-team: (names, to be nominated and accepted by RFC steering committee) +shepherd-leader: (name to be appointed by RFC steering committee) +related-issues: (will contain links to implementation PRs) +--- + +# Summary +[summary]: #summary + +This RFC proposes a mechanism to package Nix expressions into +composable entities called "flakes". Flakes allow hermetic, +reproducible evaluation of multi-repository Nix projects; impose a +discoverable, standard structure on Nix projects; and replace previous +mechanisms such as Nix channels and the Nix search path. + +# TLDR + +* A prototype implementation is available in Nixpkgs: `nix run + nixpkgs.nixFlakes`. + +* Flakes replace channels. For example, + + # nix run nixpkgs:hello -c hello + + fetches the latest version of [the `nixpkgs` + flake](https://github.com/edolstra/nixpkgs/blob/release-19.03/flake.nix) + from GitHub and builds its `hello` package. Similarly, Nix itself is + a flake, so you can get the latest version as follows: + + # nix run nix + +* Flakes are looked up in a + [registry](https://raw.githubusercontent.com/NixOS/flake-registry/master/flake-registry.json) + that maps identifiers such as `nixpkgs` to actual locations such as + `github:edolstra/nixpkgs/release-19.03`. You can use such locations + ("flake references") directly in the `nix` command: + + ``` + # nix build github:NixOS/patchelf + ``` + +* For a reproducible result, you can also use a specific revision: + + ``` + # nix build nixpkgs/a0e1f50e6f72e5037d71a0b65c67cf0605349a06:hello + ``` + +* To get information about a flake: + + ``` + # nix flake info nixpkgs + Description: A collection of packages for the Nix package manager + Revision: a0e1f50e6f72e5037d71a0b65c67cf0605349a06 + ... + ``` + +* In addition to the global registry, there is a per-user + registry. This can be used to pin flakes to the current version: + ``` + # nix flake pin nixpkgs + ``` + or to a specific version: + ``` + # nix flake add nixpkgs github:edolstra/nixpkgs/a0e1f50e6f72e5037d71a0b65c67cf0605349a06 + ``` + +* Flakes can have dependencies on other flakes. For example, [the + `patchelf` + flake](https://github.com/NixOS/patchelf/blob/master/flake.nix) + depends on the `nixpkgs` flake. To ensure reproducibility, + dependencies are pinned to specific versions using a *lock file* + (e.g. [for + patchelf](https://github.com/NixOS/patchelf/blob/master/flake.lock)). Lock + files are generated automatically. + +* Flakes are evaluated in pure mode, meaning they can't access + anything other than their own source or declared dependencies. This + allows `nix` to cache evaluation results. + +* The `nix` command is flake-based. For example, + ``` + # nix build + ``` + builds the flake in the current directory. + +# Motivation +[motivation]: #motivation + +Flakes are motivated by a number of serious shortcomings in Nix: + +* While Nix pioneered reproducible builds, sadly, Nix expressions are + not nearly as reproducible as Nix builds. Nix expressions can access + arbitrary files (such as `~/.config/nixpkgs/config.nix`), + environment variables, and Git repositories. This means for instance + that it is hard to ensure reproducible evaluation of NixOS or NixOps + configurations. + +* Nix projects lake discoverability and a standard structure. For + example, it's just convention that a repository has a `release.nix` + for Hydra jobs and a `default.nix` for packages. + +* There is no standard way to compose Nix projects. Typical ways are + to rely on the Nix search path (e.g. `import `) or to use + `fetchGit` or `fetchTarball`. The former has poor reproducibility, + while the latter is bad UX because of the need to manually update + Git hashes to update dependencies. + +* `nix-channel` needs a replacement: channels are hard to create, + users cannot easily pin specific versions of channels, channels + interact in *ad hoc* ways with the Nix search path, and so on. + +The flakes mechanism seeks to address all these problems. It can be +seen as a "Cargo/NPM/... for Nix" except that it's built into Nix +rather than a separate tool. + +# Detailed design +[design]: #detailed-design + +## Flakes + +A flake is a Git repository that contains a file named `flake.nix` in +the root directory. (In the future, there might be other types of +flakes, such as Mercurial repositories or tarballs.) `flake.nix` +specifies some metadata about the flake such as dependencies, as well +as its *outputs* (the Nix values such as packages or NixOS modules +provided by the flake). + +As an example, below is the `flake.nix` of +[`dwarffs`](https://github.com/edolstra/dwarffs). It depends on the +Nixpkgs flake and provides a package (i.e. an installable derivation) +and a NixOS module. + +``` +{ + name = "dwarffs"; + + epoch = 201906; + + description = "A filesystem that fetches DWARF debug info from the Internet on demand"; + + inputs = [ "nixpkgs" ]; + + outputs = inputs: rec { + packages.dwarffs = + with import inputs.nixpkgs { system = "x86_64-linux"; }; + + stdenv.mkDerivation { + name = "dwarffs-0.1.${lib.substring 0 8 inputs.self.lastModified}"; + + buildInputs = [ fuse nix nlohmann_json boost ]; + + NIX_CFLAGS_COMPILE = "-I ${nix.dev}/include/nix -include ${nix.dev}/include/nix/config.h -D_FILE_OFFSET_BITS=64"; + + src = inputs.self; + + installPhase = + '' + mkdir -p $out/bin $out/lib/systemd/system + + cp dwarffs $out/bin/ + ln -s dwarffs $out/bin/mount.fuse.dwarffs + + cp ${./run-dwarffs.mount} $out/lib/systemd/system/run-dwarffs.mount + cp ${./run-dwarffs.automount} $out/lib/systemd/system/run-dwarffs.automount + ''; + }; + + nixosModules.dwarffs = import ./module.nix inputs; + + defaultPackage = packages.dwarffs; + + checks.build = packages.dwarffs; + }; +} +``` + +A flake has the following attributes: + +* `epoch`: A number that specifies the version of the flake + syntax/semantics to be used. This allows the interpretation of + flakes to change in the future. It also enables some evolution of + the Nix language; for example, the Nix files in the flake could be + parsed using a syntax determined by the epoch. The only currently + allowed value is `201906`. Nix rejects flakes with an unsupported + epoch. + +* `name`: A identifier for the flake, used to reference it from + `inputs`. (E.g. the `nixpkgs` in `inputs.nixpkgs` refers to the + identifier of the Nixpkgs flake.) + +* `description`: A short description of the flake. + +* `inputs`: The dependencies of the flake, as a list of flake + references (described below). + +* `outputs`: A function that, given an attribute set containing the + outputs of each of the inputs, yields the Nix values provided by + this flake. A number of outputs have a special meaning, as discussed + below. + +## Well-known outputs + +A number of outputs have a specific meaning to Nix or other tools like +Hydra. Currently, these are: + +* `packages`: A set of derivations used as a default by most `nix` + commands. For example, `nix run nixpkgs:hello` uses the + `packages.hello` attribute of the `nixpkgs` flake. It cannot contain + any non-derivation attributes. This also means it cannot be a nested + set! (The rationale is that supporting nested sets requires Nix to + evaluate each attribute in the set, just to discover which packages + are provided.) + +* `defaultPackage`: A derivations used as a default by most `nix` + commands if no attribute is specified. For example, `nix run + dwarffs` uses the `defaultPackage` attribute of the `dwarffs` flake. + +* `checks`: A non-nested set of derivations built by the `nix flake + check` command, or by Hydra if a flake does not have a `hydraJobs` + attribute. + +* `hydraJobs`: A nested set of derivations built by Hydra. + +* `devShell`: A derivation that defines the shell environment used by + `nix dev-shell` if no specific attribute is given. If it does not + exist, then `nix dev-shell` will use `defaultPackage`. + +* `apps`: A set of app definitions used by the `nix app` command. For + example, `nix app blender-bin:blender_2_79` uses the + `apps.blender_2_79` output of the `blender-bin` flake. + +* `defaultApp`: A app definition used by the `nix app` command when no + specific attribute is given. For example, `nix app blender-bin` uses + the `defaultApp` output of the `blender-bin` flake. + +TODO: NixOS-related outputs such as `nixosModules` and `nixosSystems`. + +## Flake references + +Flake references are a vaguely URL-like syntax to specify which flake +to use. This is used on the command line (e.g. in `nix build +nixpkgs:hello`, `nixpkgs` is a flake reference), and in the list of +flake dependencies in `flake.nix` (e.g. in `inputs = [ "nixpkgs" ];`). + +Currently the following types of flake references are supported: + +* Git repositories. These have the form + + (http|https|ssh|git|file):(//)?(\?)? + + with the constraint that must end with `.git` for non-`file` + repositories. are a list of key/value pairs in URI query + parameter syntax. The following parameters are supported: + + * `ref`: The branch or tag to fetch. The default is `master`. + * `rev`: The Git commit hash to fetch. Note that this commit must be + an ancestor of `ref`, since Nix doesn't clone the entire + repository, only the specified `ref` (and the Git protocol doesn't + allow fetching a `rev` without a known `ref`). + * `dir`: The subdirectory of the repository in which `flake.nix` is + located. This parameter enables having multiple flakes in a + repository. The default is the root directory. + + For example, the following are valid Git flake references: + + * `https://example.org/my/repo.git` + * `https://example.org/my/repo.git?dir=flake1` + * `ssh://git@github.com:NixOS/nix.git?ref=v1.2.3` + * `git://github.com/edolstra/dwarffs.git?ref=unstable&rev=e486d8d40e626a20e06d792db8cc5ac5aba9a5b4` + * `file:///home/my-user/some-repo/some-repo.git` + +* Local paths. These have the form + + (\?` must refer to (a subdirectory of) a Git repository. + These differ from `file://` flake references in a few ways: + + * They refer to the working tree (unless an explicit `rev` or `ref` + is specified), so evaluation can access dirty files. (Dirty files + are files that are tracked by git but have uncommitted changes.) + + * The `dir` parameter is automatically derived. For example, if + `/foo/bar` is a Git repository, then the flake reference + `/foo/bar/flake` is equivalent to `/foo/bar?dir=flake`. + +* GitHub repositories. These are downloaded as tarball archives, + rather than through Git. This is often much faster and uses less + disk space since it doesn't require fetching the entire history of + the repository. On the other hand, it doesn't allow incremental + fetching (but full downloads are often faster than incremental + fetches!). The syntax is: + + github:/(/)?(\?)? + + `` specifies the name of a branch or tag (`ref`), or a + commit hash (`rev`). Note that unlike GitHub, Git allows fetching by + commit hash without specifying a branch or tag. + + The only supported parameter is `dir` (see above). + + Some examples: + + * `github:edolstra/dwarffs` + * `github:edolstra/dwarffs/unstable` + * `github:edolstra/dwarffs/d3f2baba8f425779026c6ec04021b2e927f61e31` + +* Indirections through the flake registry. These have the form + + (/(/rev)?)? + + These perform a lookup of `` in the flake registry. The + specified `rev` and/or `ref` are then merged with the entry in the + registry. (See below.) For example, `nixpkgs` and + `nixpkgs/release-19.03` are indirect flake references. + +In the future, we should also add tarball flake references +(e.g. `https://example.org/nixpkgs.tar.xz`). It would also be +straight-forward to add flake references to Mercurial repositories +since Nix already has Mercurial support. + +## Flake registries + +Flake registries map symbolic flake identifiers (e.g. `nixpkgs`) to +"direct" flake references (i.e. any type of flake reference that's not +an indirection). This is a convenience to users, allowing them to do + + nix run nixpkgs:hello + +rather than + + nix run github:NixOS/nixpkgs:hello + +There are multiple registries: + +* The global registry + https://raw.githubusercontent.com/NixOS/flake-registry/master/flake-registry.json. (This + location can be overriden via the `flake-registry` option.) Nix + automatically fetches this registry periodically. The check interval + is determined by the `tarball-ttl` option. + +* The local registry `~/.config/nix/registry.json`. + +* Registry entries specified on the command line. + +A registry is a JSON file that looks like this: + +``` +{ + "version": 1, + "flakes": { + "nixpkgs": { + "uri": "github:NixOS/nixpkgs" + }, + ... + } +} +``` + +With this registry, flake references resolve as follows: + +* `nixpkgs` -> `github:NixOS/nixpkgs` +* `nixpkgs/release-19.03` -> `github:NixOS/nixpkgs/release-19.03` +* `nixpkgs/f1c995e694685d6dfb877f6428d3e050d30e253c` -> `github:NixOS/nixpkgs/f1c995e694685d6dfb877f6428d3e050d30e253c` + +The registries are searched in reverse order. Thus the local registry +overrides the global registry, and the command line takes precedence +over the local and global registries. + +## `nix` command line interface + +I propose to make flakes the primary way to specify packages in the +`nix` command line interface. Note that the `nix` UI is still marked +as experimental, so we have some freedom to make incompatible +changes. The legacy commands (`nix-build`, `nix-shell`, `nix-env` and +`nix-instantiate`) should not be changed to avoid breakage. + +Most `nix` subcommands work on a list of arguments called +"installables" by lack of a better term. For example, + + # nix run nixpkgs:hello dwarffs:dwarffs + +takes two flake-based installables. The general form is: + + : + +Examples of installables: + +* `nixpkgs:packages.hello` +* `nixpkgs:hello` - short for `nixpkgs:packages.hello` +* `nixpkgs/release-19.03:hello` - overrides the Git branch to use +* `github:NixOS/nixpkgs/4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082:hello` - + specifies the exact Git revision to use +* `dwarffs` - short for `dwarffs:defaultPackage` +* `nix:hydraJobs.build.x86_64-darwin` +* `.:hydraJobs.build.x86_64-darwin` - refers to the flake in the + current directory (which can be a dirty Git tree) +* `.` - short for `.:defaultPackage` + +If no argument is given, the default is `.`; thus, + + # nix build + +is equivalent to + + # nix build .:defaultPackage + +For backwards compatibility, it's possible to use non-flake Nix +expressions using `-f`, e.g. `nix build -f foo.nix foo.bar`. + +## Lock files and reproducible evaluation + +Dependencies specified in `flake.nix` are typically "unlocked": they +are specified as flake references that don't specify an exact revision +(e.g. `nixpkgs` rather than +`github:NixOS/nixpkgs/4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082`). To +ensure reproducibility, Nix will automatically generate and use a +*lock file* called `flake.lock` in the flake's directory. The lock +file contains a tree of mappings from the flake references specified +in `flake.nix` to direct flake references that contain revisions. + +For example, the inputs +``` +inputs = + [ "nixpkgs" + github:edolstra/import-cargo + ]; +``` +might result in the following lock file: +``` +{ + "version": 2, + "inputs": { + "github:edolstra/import-cargo": { + "id": "import-cargo", + "inputs": {}, + "narHash": "sha256-mxwKMDFOrhjrBQhIWwwm8mmEugyx/oVlvBH1CKxchlw=", + "uri": "github:edolstra/import-cargo/c33e13881386931038d46a7aca4c9561144d582e" + }, + "nixpkgs": { + "id": "nixpkgs", + "inputs": {}, + "narHash": "sha256-p7UqhvhwS5MZfqUbLbFm+nfG/SMJrgpNXxWpRMFif8c=", + "uri": "github:NixOS/nixpkgs/4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082" + } + } +} +``` + +Thus, when we build this flake, the input `nixpkgs` is mapped to +`github:edolstra/import-cargo/c33e13881386931038d46a7aca4c9561144d582e`. Nix +will also check that the content hash of the input is equal to the one +recorded in the lock file. This check is superfluous for Git +repositories (since the commit hash serves a similar purpose), but for +GitHub archives, we cannot directly check that the contents match the +commit hash. + +Note that lock files are only used at top-level: the `flake.lock` files +in dependencies (if they exist) are ignored. + +Nix automatically creates a `flake.lock` file when you build a local +repository (e.g. `nix build /path/to/repo`). It will also update the +lock file if inputs are added or removed. You can pass +`--recreate-lock-file` to force Nix to recreate the lock file from +scratch (and thus check for the latest version of each input). + +Lock files are not sufficient by themselves to ensure reproducible +evaluation. It is also necessary to prevent certain impurities. In +particular, the `nix` command now defaults to evaluating in "pure" +mode, which means that the following are disallowed: + +* Access to files outside of the top-level flake, its inputs, or paths + fetched using `fetchTarball`, `fetchGit` and so on with a commit + hash or content hash. In particular this means that Nixpkgs will not + be able to use `~/.config/nixpkgs` anymore. + +* Access to the environment. This means that `builtins.getEnv ""` + always returns an empty string. + +* Access to the system type (`builtins.currentSystem`). + +* Access to the current time (`builtins.currentTime`). + +* Use of the Nix search path (`<...>`); composition must be done + through flake inputs or `fetchX` builtins. + +Pure mode can be disabled by passing `--impure` on the command line. + +## Evaluation caching + +The fact that evaluation is now hermetic allows `nix` to cache flake +attributes. For example (doing `nix build` on an already present +package): + + $ time nix build nixpkgs:firefox + real 0m1.497s + + $ time nix build nixpkgs:firefox + real 0m0.052s + +The evaluation cache is kept in `~/.cache/nix/eval-cache-v1.sqlite`, +which has entries like + + INSERT INTO Attributes VALUES( + X'92a907d4efe933af2a46959b082cdff176aa5bfeb47a98fabd234809a67ab195', + 'packages.firefox', + 1, + '/nix/store/pbalzf8x19hckr8cwdv62rd6g0lqgc38-firefox-67.0.drv /nix/store/g6q0gx0v6xvdnizp8lrcw7c4gdkzana0-firefox-67.0 out'); + +where the hash `92a9...` is a fingerprint over the flake store path +and the contents of its lockfile. Because flakes are evaluated in pure +mode, this uniquely identifies the evaluation result. + +Currently caching is only done for top-level attributes (e.g. for +`packages.firefox` in the command above). In the future, we could also +add other evaluated values to the cache (e.g. `packages.stdenv`) to +speed up subsequent evaluations of other top-level attributes. + +## Flake-related commands + +The command `nix flake` has various subcommands for managing +flakes. These are: + +* `nix flake list`: Show all flakes in the global and local registry. + +* `nix flake add `: Add an entry to the + local registry. (Recall that this overrides the global registry.) + For examples, + + # nix flake add nixpkgs github:my-repo/my-nixpkgs + + redirects `nixpkgs` to a different repository. Similarly, + + # nix flake add nixpkgs github:NixOS/nixpkgs/444f22ca892a873f76acd88d5d55bdc24ed08757 + + pins `nixpkgs` to a specific revision. + + Note that registries only have an effect on flake references used on + the command line or when lock files are generated or updated. + +* `nix flake remove `: Remove an entry from the local + registry. + +* `nix flake pin `: Compute a locked flake reference and + add it to the local registry. For example: + + # nix flake pin nixpkgs + + will add an mapping like `nixpkgs` -> + `github:NixOS/nixpkgs/444f22ca892a873f76acd88d5d55bdc24ed08757` to + the local registry. + +* `nix flake init`: Create a skeleton `flake.nix` in the current + directory. + +* `nix flake update`: Recreate the lock file from scratch. + +* `nix flake check`: Do some checks on the flake (e.g. check that all + `packages` are really packages), then build the `checks` + derivations. For example, `nix flake check patchelf` fetches the + `patchelf` flake, evaluates it and builds it. + +* `nix flake clone`: Do a `git clone` to obtain the source of the + specified flake, e.g. `nix flake clone dwarffs` will yield a clone + of the `dwarffs` repository. + +## Offline use + +Since flakes replace channels, we have to make sure that they support +offline use at least as well. Channels only require network access +when you do `nix-channel --update`. By contrast, the `nix` command +will periodically fetch the latest version of the global registry and +of the top-level flake (e.g. `nix build nixpkgs:hello` may cause it to +fetch a new version of Nixpkgs). To make sure things work offline, we +do the following: + +* It's not a failure if we can't fetch the registry. + +* It's not a failure if we can't fetch the latest version of a + flake. (For example, `fetchGit` no longer fails if we already have + the specified repository and `ref`.) + +* Fetched flakes are registered as garbage collector roots, so running + the garbage collector while on an airplane will not ruin your day. + +* There is an option `--no-net` to explicitly prevent updating the + registry or doing other things that need the network (such as + substituting). This is the default if there are no configured + non-loopback network interfaces. + +# Drawbacks +[drawbacks]: #drawbacks + +Pure evaluation breaks certain workflows. In particular, it breaks the +use of the Nixpkgs configuration file. Similarly, there are people who +rely on `$NIX_PATH` to pass configuration data to NixOps +configurations. + +# Alternatives +[alternatives]: #alternatives + +For composition of multi-repository projects, the main alternative is +to continue on with explicit `fetchGit` / `fetchTarball` calls to pull +in other repositories. However, since there is no explicit listing of +dependencies, this does not provide automatic updating (i.e. there is +no equivalent of `nix flake update`). + +Instead of a `flake.nix`, flakes could store their metadata in a +simpler format such as JSON or TOML. This avoids the Turing tarpit +where getting flake metadata requires the execution of an arbitrarily +complex, possibly non-terminating program. + +# Unresolved questions +[unresolved]: #unresolved-questions + +* How to handle the system type? Currently `x86_64-linux` is + hard-coded everywhere. + +* How to do Nixpkgs overlays? In principle, overlays would just be a + kind of flake output. + +* More in general, how to handle flake arguments? This must be done in + a way that maintains hermetic evaluation and evaluation caching. + +* What are the criteria for inclusion in the global flake registry? + +* Hammer out the details of NixOS/NixOps support for flakes. + +* Currently, if flake dependencies (repositories or branches) get + deleted, rebuilding the flake may fail. (This is similar to + `fetchurl` referencing a stale URL.) We need a command to gather all + flake dependencies and copy them somewhere else (possibly vendor + them into the repository of the calling flake). + +# Future work +[future]: #future-work + +* The "epoch" feature enables future Nix changes, including language + changes. For example, changing the parsing of multiline strings + (https://github.com/NixOS/nix/pull/2490) could be conditional on the + flake's epoch. + +* Currently flake outputs are untyped; we only have some conventions + about what they should be (e.g. `packages` should be an attribute + set of derivations). For discoverability, it would be nice if + outputs were typed. Maybe this could be done via the Nix + configurations concept + (https://gist.github.com/edolstra/29ce9d8ea399b703a7023073b0dbc00d). + +* Automatically generate documentation from flakes. This partially + depends on the previous item. + +# Acknowledgements + +Funding for the development of the flakes prototype was provided by +[Target Corporation](https://www.target.com/). From 734d92071a2583e1b9e79543c8f33d5674138e42 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Tue, 9 Jul 2019 21:46:20 +0200 Subject: [PATCH 02/41] Updates --- rfcs/0000-flakes.md | 109 ++++++++++++++++++++++++++++++++------------ 1 file changed, 79 insertions(+), 30 deletions(-) diff --git a/rfcs/0000-flakes.md b/rfcs/0000-flakes.md index 6b67e15b1..e48890c4f 100644 --- a/rfcs/0000-flakes.md +++ b/rfcs/0000-flakes.md @@ -125,14 +125,15 @@ rather than a separate tool. A flake is a Git repository that contains a file named `flake.nix` in the root directory. (In the future, there might be other types of flakes, such as Mercurial repositories or tarballs.) `flake.nix` -specifies some metadata about the flake such as dependencies, as well -as its *outputs* (the Nix values such as packages or NixOS modules -provided by the flake). +specifies some metadata about the flake such as dependencies (called +*inputs*), as well as its *outputs* (the Nix values such as packages +or NixOS modules provided by the flake). As an example, below is the `flake.nix` of -[`dwarffs`](https://github.com/edolstra/dwarffs). It depends on the -Nixpkgs flake and provides a package (i.e. an installable derivation) -and a NixOS module. +[`dwarffs`](https://github.com/edolstra/dwarffs) (a FUSE filesystem +for automatically fetching DWARF debug symbols by ELF build ID). It +depends on the Nixpkgs flake and provides a package (i.e. an +installable derivation) and a NixOS module. ``` { @@ -146,10 +147,12 @@ and a NixOS module. outputs = inputs: rec { packages.dwarffs = - with import inputs.nixpkgs { system = "x86_64-linux"; }; + with inputs.nixpkgs.packages; + with inputs.nixpkgs.builders; + with inputs.nixpkgs.lib; stdenv.mkDerivation { - name = "dwarffs-0.1.${lib.substring 0 8 inputs.self.lastModified}"; + name = "dwarffs-0.1.${substring 0 8 inputs.self.lastModified}"; buildInputs = [ fuse nix nlohmann_json boost ]; @@ -198,9 +201,42 @@ A flake has the following attributes: references (described below). * `outputs`: A function that, given an attribute set containing the - outputs of each of the inputs, yields the Nix values provided by - this flake. A number of outputs have a special meaning, as discussed - below. + outputs of each of the input flakes keyed by their identifier, + yields the Nix values provided by this flake. Thus, in the example + above, `inputs.nixpkgs` contains the result of the call to the + `outputs` function of the `nixpkgs` flake, and + `inputs.nixpkgs.packages.fuse` refers to the `packages.fuse` output + attribute of `nixpkgs`. + + In addition to the outputs of each input, each input in `inputs` + also contains some metadata about the inputs. These are: + + * `outPath`: The path in the Nix store of the flake's source + tree. This means that you could import Nixpkgs in a more + legacy-ish way by writing + + with import inputs.nixpkgs { system = "x86_64-linux"; }; + + since `nixpkgs` still contains a `/default.nix`. In this case we + bypass its outputs entirely and only use the flake mechanism to + get its source tree. + + * `rev`: The commit hash of the flake's Git repository. + + * `revCount`: The number of ancestors of the revision `rev`. This is + not available for `github:...` repositories (see below), since + they're fetched as tarballs rather than as Git repositories. + + * `lastModified`: The commit time of the revision `rev`, in the + format `%Y%m%d%H%M%S` (e.g. `20181231100934`). Unlike `revCount`, + this is available for both Git and GitHub repositories, so it's + useful for generating (hopefully) monotonically increasing version + strings. + + The special input named `self` refers to the outputs and source tree + of *this* flake. + + A number of outputs have a special meaning, as discussed below. ## Well-known outputs @@ -215,7 +251,7 @@ Hydra. Currently, these are: evaluate each attribute in the set, just to discover which packages are provided.) -* `defaultPackage`: A derivations used as a default by most `nix` +* `defaultPackage`: A derivation used as a default by most `nix` commands if no attribute is specified. For example, `nix run dwarffs` uses the `defaultPackage` attribute of the `dwarffs` flake. @@ -233,7 +269,7 @@ Hydra. Currently, these are: example, `nix app blender-bin:blender_2_79` uses the `apps.blender_2_79` output of the `blender-bin` flake. -* `defaultApp`: A app definition used by the `nix app` command when no +* `defaultApp`: An app definition used by the `nix app` command when no specific attribute is given. For example, `nix app blender-bin` uses the `defaultApp` output of the `blender-bin` flake. @@ -252,9 +288,9 @@ Currently the following types of flake references are supported: (http|https|ssh|git|file):(//)?(\?)? - with the constraint that must end with `.git` for non-`file` - repositories. are a list of key/value pairs in URI query - parameter syntax. The following parameters are supported: + with the constraint that `` must end with `.git` for + non-`file` repositories. `` are a list of key/value pairs in + URI query parameter syntax. The following parameters are supported: * `ref`: The branch or tag to fetch. The default is `master`. * `rev`: The Git commit hash to fetch. Note that this commit must be @@ -380,13 +416,13 @@ changes. The legacy commands (`nix-build`, `nix-shell`, `nix-env` and `nix-instantiate`) should not be changed to avoid breakage. Most `nix` subcommands work on a list of arguments called -"installables" by lack of a better term. For example, +"installables" for lack of a better word. For example, # nix run nixpkgs:hello dwarffs:dwarffs takes two flake-based installables. The general form is: - : + (:)? Examples of installables: @@ -414,23 +450,23 @@ expressions using `-f`, e.g. `nix build -f foo.nix foo.bar`. ## Lock files and reproducible evaluation -Dependencies specified in `flake.nix` are typically "unlocked": they -are specified as flake references that don't specify an exact revision -(e.g. `nixpkgs` rather than +Inputs specified in `flake.nix` are typically "unlocked": they are +flake references that don't specify an exact revision (e.g. `nixpkgs` +rather than `github:NixOS/nixpkgs/4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082`). To ensure reproducibility, Nix will automatically generate and use a *lock file* called `flake.lock` in the flake's directory. The lock file contains a tree of mappings from the flake references specified in `flake.nix` to direct flake references that contain revisions. -For example, the inputs +For example, if `flake.nix` contains ``` inputs = [ "nixpkgs" github:edolstra/import-cargo ]; ``` -might result in the following lock file: +then the resulting lock file might be: ``` { "version": 2, @@ -473,10 +509,10 @@ evaluation. It is also necessary to prevent certain impurities. In particular, the `nix` command now defaults to evaluating in "pure" mode, which means that the following are disallowed: -* Access to files outside of the top-level flake, its inputs, or paths - fetched using `fetchTarball`, `fetchGit` and so on with a commit - hash or content hash. In particular this means that Nixpkgs will not - be able to use `~/.config/nixpkgs` anymore. +* Access to files outside of the top-level flake or its inputs, as + well as paths fetched using `fetchTarball`, `fetchGit` and so on + with a commit hash or content hash. In particular this means that + Nixpkgs will not be able to use `~/.config/nixpkgs` anymore. * Access to the environment. This means that `builtins.getEnv ""` always returns an empty string. @@ -529,7 +565,7 @@ flakes. These are: * `nix flake add `: Add an entry to the local registry. (Recall that this overrides the global registry.) - For examples, + For example, # nix flake add nixpkgs github:my-repo/my-nixpkgs @@ -575,8 +611,7 @@ offline use at least as well. Channels only require network access when you do `nix-channel --update`. By contrast, the `nix` command will periodically fetch the latest version of the global registry and of the top-level flake (e.g. `nix build nixpkgs:hello` may cause it to -fetch a new version of Nixpkgs). To make sure things work offline, we -do the following: +fetch a new version of Nixpkgs). To make sure things work offline: * It's not a failure if we can't fetch the registry. @@ -592,6 +627,16 @@ do the following: substituting). This is the default if there are no configured non-loopback network interfaces. +## CI integration + +Flakes are useful for CI systems since `flake.nix` and the lock file +exactly specify the input repositories. For Hydra this means that you +no longer need to configure any jobset inputs; just a top-level flake +reference is enough. For example, the [configuration of the `patchelf` +jobset](https://hydra.nixos.org/jobset/patchelf/master#tabs-configuration) +just points to `github:NixOS/patchelf` and doesn't need to specify +`nixpkgs`. + # Drawbacks [drawbacks]: #drawbacks @@ -654,6 +699,10 @@ complex, possibly non-terminating program. * Automatically generate documentation from flakes. This partially depends on the previous item. +* Evaluation caches could be shared in a binary-cache-like + mechanism. This might make the use of import-from-derivation in + Nixpkgs acceptable. + # Acknowledgements Funding for the development of the flakes prototype was provided by From 3b16530cea252580b9f0281d773745f7bb2d5b47 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Tue, 9 Jul 2019 21:48:06 +0200 Subject: [PATCH 03/41] Update date --- rfcs/0000-flakes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/0000-flakes.md b/rfcs/0000-flakes.md index e48890c4f..a221d93b8 100644 --- a/rfcs/0000-flakes.md +++ b/rfcs/0000-flakes.md @@ -1,6 +1,6 @@ --- feature: flakes -start-date: 2019-06-25 +start-date: 2019-07-09 author: Eelco Dolstra co-authors: TBD shepherd-team: (names, to be nominated and accepted by RFC steering committee) From c83185c04d53ddd2f19c5573f424b3a005f40d85 Mon Sep 17 00:00:00 2001 From: zimbatm Date: Thu, 11 Jul 2019 10:11:50 +0000 Subject: [PATCH 04/41] fix typo --- rfcs/0000-flakes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/0000-flakes.md b/rfcs/0000-flakes.md index a221d93b8..3d6ea5b4a 100644 --- a/rfcs/0000-flakes.md +++ b/rfcs/0000-flakes.md @@ -99,7 +99,7 @@ Flakes are motivated by a number of serious shortcomings in Nix: that it is hard to ensure reproducible evaluation of NixOS or NixOps configurations. -* Nix projects lake discoverability and a standard structure. For +* Nix projects lack discoverability and a standard structure. For example, it's just convention that a repository has a `release.nix` for Hydra jobs and a `default.nix` for packages. From cc5d0a24970a49d7da294544aff2fb3c26139635 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Thu, 11 Jul 2019 12:51:34 +0200 Subject: [PATCH 05/41] Rename 'epoch' -> 'edition' This follows the Rust naming. --- rfcs/0000-flakes.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/rfcs/0000-flakes.md b/rfcs/0000-flakes.md index 3d6ea5b4a..356a08c39 100644 --- a/rfcs/0000-flakes.md +++ b/rfcs/0000-flakes.md @@ -139,7 +139,7 @@ installable derivation) and a NixOS module. { name = "dwarffs"; - epoch = 201906; + edition = 201906; description = "A filesystem that fetches DWARF debug info from the Internet on demand"; @@ -183,13 +183,13 @@ installable derivation) and a NixOS module. A flake has the following attributes: -* `epoch`: A number that specifies the version of the flake +* `edition`: A number that specifies the version of the flake syntax/semantics to be used. This allows the interpretation of flakes to change in the future. It also enables some evolution of the Nix language; for example, the Nix files in the flake could be - parsed using a syntax determined by the epoch. The only currently + parsed using a syntax determined by the edition. The only currently allowed value is `201906`. Nix rejects flakes with an unsupported - epoch. + edition. * `name`: A identifier for the flake, used to reference it from `inputs`. (E.g. the `nixpkgs` in `inputs.nixpkgs` refers to the @@ -684,10 +684,10 @@ complex, possibly non-terminating program. # Future work [future]: #future-work -* The "epoch" feature enables future Nix changes, including language +* The "edition" feature enables future Nix changes, including language changes. For example, changing the parsing of multiline strings (https://github.com/NixOS/nix/pull/2490) could be conditional on the - flake's epoch. + flake's edition. * Currently flake outputs are untyped; we only have some conventions about what they should be (e.g. `packages` should be an attribute From f2cf8a85551afbf87387336f0930be05c598685d Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Tue, 16 Jul 2019 00:15:17 +0200 Subject: [PATCH 06/41] Rename to RFC #49 --- rfcs/{0000-flakes.md => 0049-flakes.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename rfcs/{0000-flakes.md => 0049-flakes.md} (100%) diff --git a/rfcs/0000-flakes.md b/rfcs/0049-flakes.md similarity index 100% rename from rfcs/0000-flakes.md rename to rfcs/0049-flakes.md From 1441693f56590589fe4de2346dafc7418e14135f Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Tue, 16 Jul 2019 00:19:44 +0200 Subject: [PATCH 07/41] Typo --- rfcs/0049-flakes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 356a08c39..e128d284a 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -703,7 +703,7 @@ complex, possibly non-terminating program. mechanism. This might make the use of import-from-derivation in Nixpkgs acceptable. -# Acknowledgements +# Acknowledgments Funding for the development of the flakes prototype was provided by [Target Corporation](https://www.target.com/). From 9334af11eee6fb6e30349c8b57dd2ffa13d27c1d Mon Sep 17 00:00:00 2001 From: zimbatm Date: Tue, 16 Jul 2019 09:24:55 +0000 Subject: [PATCH 08/41] Update rfcs/0049-flakes.md Co-Authored-By: Frederik Rietdijk --- rfcs/0049-flakes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index e128d284a..b05196dcb 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -191,7 +191,7 @@ A flake has the following attributes: allowed value is `201906`. Nix rejects flakes with an unsupported edition. -* `name`: A identifier for the flake, used to reference it from +* `name`: An identifier for the flake, used to reference it from `inputs`. (E.g. the `nixpkgs` in `inputs.nixpkgs` refers to the identifier of the Nixpkgs flake.) From 57f139b076266cf7a3f18ffcc675506739ab8b86 Mon Sep 17 00:00:00 2001 From: zimbatm Date: Tue, 16 Jul 2019 09:25:42 +0000 Subject: [PATCH 09/41] Update rfcs/0049-flakes.md Co-Authored-By: Frederik Rietdijk --- rfcs/0049-flakes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index b05196dcb..702d140fd 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -586,7 +586,7 @@ flakes. These are: # nix flake pin nixpkgs - will add an mapping like `nixpkgs` -> + will add a mapping like `nixpkgs` -> `github:NixOS/nixpkgs/444f22ca892a873f76acd88d5d55bdc24ed08757` to the local registry. From 368d9325a3c76457496cbe27c227bcd7fb1ffea7 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Fri, 19 Jul 2019 18:16:00 +0200 Subject: [PATCH 10/41] Remove mention of apps and other minor fixes --- rfcs/0049-flakes.md | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 702d140fd..9dadeae67 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -256,7 +256,7 @@ Hydra. Currently, these are: dwarffs` uses the `defaultPackage` attribute of the `dwarffs` flake. * `checks`: A non-nested set of derivations built by the `nix flake - check` command, or by Hydra if a flake does not have a `hydraJobs` + check` command, and by Hydra if a flake does not have a `hydraJobs` attribute. * `hydraJobs`: A nested set of derivations built by Hydra. @@ -265,14 +265,6 @@ Hydra. Currently, these are: `nix dev-shell` if no specific attribute is given. If it does not exist, then `nix dev-shell` will use `defaultPackage`. -* `apps`: A set of app definitions used by the `nix app` command. For - example, `nix app blender-bin:blender_2_79` uses the - `apps.blender_2_79` output of the `blender-bin` flake. - -* `defaultApp`: An app definition used by the `nix app` command when no - specific attribute is given. For example, `nix app blender-bin` uses - the `defaultApp` output of the `blender-bin` flake. - TODO: NixOS-related outputs such as `nixosModules` and `nixosSystems`. ## Flake references @@ -296,7 +288,8 @@ Currently the following types of flake references are supported: * `rev`: The Git commit hash to fetch. Note that this commit must be an ancestor of `ref`, since Nix doesn't clone the entire repository, only the specified `ref` (and the Git protocol doesn't - allow fetching a `rev` without a known `ref`). + allow fetching a `rev` without a known `ref`). The default is the + commit currently pointed to by `ref`. * `dir`: The subdirectory of the repository in which `flake.nix` is located. This parameter enables having multiple flakes in a repository. The default is the root directory. From f2c6f44dbeff9de89a1a897923021dd5c7520b00 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Fri, 19 Jul 2019 19:04:21 +0200 Subject: [PATCH 11/41] Typo --- rfcs/0049-flakes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 9dadeae67..ddb8eee1d 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -504,7 +504,7 @@ mode, which means that the following are disallowed: * Access to files outside of the top-level flake or its inputs, as well as paths fetched using `fetchTarball`, `fetchGit` and so on - with a commit hash or content hash. In particular this means that + without a commit hash or content hash. In particular this means that Nixpkgs will not be able to use `~/.config/nixpkgs` anymore. * Access to the environment. This means that `builtins.getEnv ""` From 4bcf19618bc500f107b75d5ad046acf12f6466fe Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Fri, 19 Jul 2019 19:34:16 +0200 Subject: [PATCH 12/41] Note about caching --- rfcs/0049-flakes.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index ddb8eee1d..d976c5168 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -521,9 +521,9 @@ Pure mode can be disabled by passing `--impure` on the command line. ## Evaluation caching -The fact that evaluation is now hermetic allows `nix` to cache flake -attributes. For example (doing `nix build` on an already present -package): +The fact that by default (i.e. in pure mode) evaluation is hermetic +allows `nix` to cache flake attributes. For example (doing `nix build` +on an already present package): $ time nix build nixpkgs:firefox real 0m1.497s From 15f1c51c5bff173f67bf57c22cc9b85ebb5c65ff Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Fri, 19 Jul 2019 19:35:40 +0200 Subject: [PATCH 13/41] Lock file is only created if it doesn't exist --- rfcs/0049-flakes.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index d976c5168..16c5bc690 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -491,11 +491,12 @@ commit hash. Note that lock files are only used at top-level: the `flake.lock` files in dependencies (if they exist) are ignored. -Nix automatically creates a `flake.lock` file when you build a local -repository (e.g. `nix build /path/to/repo`). It will also update the -lock file if inputs are added or removed. You can pass -`--recreate-lock-file` to force Nix to recreate the lock file from -scratch (and thus check for the latest version of each input). +When you build a local repository (e.g. `nix build /path/to/repo`), +Nix automatically creates a `flake.lock` file if it doesn't already +exists. It will also update the lock file if inputs are added or +removed. You can pass `--recreate-lock-file` to force Nix to recreate +the lock file from scratch (and thus check for the latest version of +each input). Lock files are not sufficient by themselves to ensure reproducible evaluation. It is also necessary to prevent certain impurities. In From b2d6b9c548ac7fcadf4d5b2f3828788782626bf7 Mon Sep 17 00:00:00 2001 From: Toon Nolten Date: Tue, 23 Jul 2019 21:54:37 +0200 Subject: [PATCH 14/41] Transposition typo GitHub <-> Git --- rfcs/0049-flakes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 16c5bc690..82a488611 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -327,7 +327,7 @@ Currently the following types of flake references are supported: github:/(/)?(\?)? `` specifies the name of a branch or tag (`ref`), or a - commit hash (`rev`). Note that unlike GitHub, Git allows fetching by + commit hash (`rev`). Note that unlike Git, GitHub allows fetching by commit hash without specifying a branch or tag. The only supported parameter is `dir` (see above). From 859d44aeb4ec47111ba3da83adc0e4477b441a4c Mon Sep 17 00:00:00 2001 From: Shea Levy Date: Thu, 1 Aug 2019 08:29:32 -0400 Subject: [PATCH 15/41] Add shepherd metadata --- rfcs/0049-flakes.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 82a488611..cae25b3e5 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -3,8 +3,8 @@ feature: flakes start-date: 2019-07-09 author: Eelco Dolstra co-authors: TBD -shepherd-team: (names, to be nominated and accepted by RFC steering committee) -shepherd-leader: (name to be appointed by RFC steering committee) +shepherd-team: Domen Kožar, Alyssa Ross, Shea Levy, John Ericson +shepherd-leader: Domen Kožar related-issues: (will contain links to implementation PRs) --- From efa667fdb4561c5abef3d896ad2b2a4f8f4a018c Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Fri, 16 Aug 2019 17:21:16 +0200 Subject: [PATCH 16/41] Remove mention of evaluation caching This is an implementation detail, not really relevant here. --- rfcs/0049-flakes.md | 34 ---------------------------------- 1 file changed, 34 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index cae25b3e5..4ed4d09aa 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -520,36 +520,6 @@ mode, which means that the following are disallowed: Pure mode can be disabled by passing `--impure` on the command line. -## Evaluation caching - -The fact that by default (i.e. in pure mode) evaluation is hermetic -allows `nix` to cache flake attributes. For example (doing `nix build` -on an already present package): - - $ time nix build nixpkgs:firefox - real 0m1.497s - - $ time nix build nixpkgs:firefox - real 0m0.052s - -The evaluation cache is kept in `~/.cache/nix/eval-cache-v1.sqlite`, -which has entries like - - INSERT INTO Attributes VALUES( - X'92a907d4efe933af2a46959b082cdff176aa5bfeb47a98fabd234809a67ab195', - 'packages.firefox', - 1, - '/nix/store/pbalzf8x19hckr8cwdv62rd6g0lqgc38-firefox-67.0.drv /nix/store/g6q0gx0v6xvdnizp8lrcw7c4gdkzana0-firefox-67.0 out'); - -where the hash `92a9...` is a fingerprint over the flake store path -and the contents of its lockfile. Because flakes are evaluated in pure -mode, this uniquely identifies the evaluation result. - -Currently caching is only done for top-level attributes (e.g. for -`packages.firefox` in the command above). In the future, we could also -add other evaluated values to the cache (e.g. `packages.stdenv`) to -speed up subsequent evaluations of other top-level attributes. - ## Flake-related commands The command `nix flake` has various subcommands for managing @@ -693,10 +663,6 @@ complex, possibly non-terminating program. * Automatically generate documentation from flakes. This partially depends on the previous item. -* Evaluation caches could be shared in a binary-cache-like - mechanism. This might make the use of import-from-derivation in - Nixpkgs acceptable. - # Acknowledgments Funding for the development of the flakes prototype was provided by From 79e33d81fe256c6c1a5cdb63c56c75c9fcaaf9c5 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Fri, 16 Aug 2019 17:22:00 +0200 Subject: [PATCH 17/41] Mention XDG_CONFIG_HOME --- rfcs/0049-flakes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 4ed4d09aa..d280522d3 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -372,7 +372,7 @@ There are multiple registries: automatically fetches this registry periodically. The check interval is determined by the `tarball-ttl` option. -* The local registry `~/.config/nix/registry.json`. +* The local registry `$XDG_CONFIG_HOME/nix/registry.json`. * Registry entries specified on the command line. From 970c31c0fdad2b083458a98789f00a969f73e0b6 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Fri, 16 Aug 2019 17:25:23 +0200 Subject: [PATCH 18/41] Use git+ prefix in URIs https://github.com/NixOS/nix/issues/3045 --- rfcs/0049-flakes.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index d280522d3..3724552f6 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -278,11 +278,10 @@ Currently the following types of flake references are supported: * Git repositories. These have the form - (http|https|ssh|git|file):(//)?(\?)? + git(+http|+https|+ssh|+git|+file|):(//)?(\?)? - with the constraint that `` must end with `.git` for - non-`file` repositories. `` are a list of key/value pairs in - URI query parameter syntax. The following parameters are supported: + `` are a list of key/value pairs in URI query parameter + syntax. The following parameters are supported: * `ref`: The branch or tag to fetch. The default is `master`. * `rev`: The Git commit hash to fetch. Note that this commit must be @@ -296,11 +295,11 @@ Currently the following types of flake references are supported: For example, the following are valid Git flake references: - * `https://example.org/my/repo.git` - * `https://example.org/my/repo.git?dir=flake1` - * `ssh://git@github.com:NixOS/nix.git?ref=v1.2.3` - * `git://github.com/edolstra/dwarffs.git?ref=unstable&rev=e486d8d40e626a20e06d792db8cc5ac5aba9a5b4` - * `file:///home/my-user/some-repo/some-repo.git` + * `git+https://example.org/my/repo` + * `git+https://example.org/my/repo?dir=flake1` + * `git+ssh://git@github.com:NixOS/nix?ref=v1.2.3` + * `git://github.com/edolstra/dwarffs?ref=unstable&rev=e486d8d40e626a20e06d792db8cc5ac5aba9a5b4` + * `git+file:///home/my-user/some-repo/some-repo` * Local paths. These have the form From d63b4cc237523eca263ff5dd410c6cb3d456a2a0 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Fri, 16 Aug 2019 17:30:21 +0200 Subject: [PATCH 19/41] Typo --- rfcs/0049-flakes.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 3724552f6..691357f5c 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -628,8 +628,8 @@ complex, possibly non-terminating program. * How to handle the system type? Currently `x86_64-linux` is hard-coded everywhere. -* How to do Nixpkgs overlays? In principle, overlays would just be a - kind of flake output. +* What to do with Nixpkgs overlays? In principle, overlays would just + be a kind of flake output. * More in general, how to handle flake arguments? This must be done in a way that maintains hermetic evaluation and evaluation caching. From d053724fe95c67fd22eb8d71d1f211ac18a6ff09 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Fri, 16 Aug 2019 17:36:07 +0200 Subject: [PATCH 20/41] Remove superfluous name attribute --- rfcs/0049-flakes.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 691357f5c..e243c3ee4 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -137,8 +137,6 @@ installable derivation) and a NixOS module. ``` { - name = "dwarffs"; - edition = 201906; description = "A filesystem that fetches DWARF debug info from the Internet on demand"; @@ -191,10 +189,6 @@ A flake has the following attributes: allowed value is `201906`. Nix rejects flakes with an unsupported edition. -* `name`: An identifier for the flake, used to reference it from - `inputs`. (E.g. the `nixpkgs` in `inputs.nixpkgs` refers to the - identifier of the Nixpkgs flake.) - * `description`: A short description of the flake. * `inputs`: The dependencies of the flake, as a list of flake From e04ad161eb489a59c491291b17ceb1647cd230b5 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Fri, 30 Aug 2019 22:51:14 +0200 Subject: [PATCH 21/41] Document the new inputs syntax --- rfcs/0049-flakes.md | 92 +++++++++++++++++++++++++++++++++------------ 1 file changed, 68 insertions(+), 24 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index e243c3ee4..7a33adb3e 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -137,26 +137,24 @@ installable derivation) and a NixOS module. ``` { - edition = 201906; + edition = 201909; description = "A filesystem that fetches DWARF debug info from the Internet on demand"; - inputs = [ "nixpkgs" ]; - - outputs = inputs: rec { + outputs = { self, nixpkgs }: rec { packages.dwarffs = - with inputs.nixpkgs.packages; - with inputs.nixpkgs.builders; - with inputs.nixpkgs.lib; + with nixpkgs.packages; + with nixpkgs.builders; + with nixpkgs.lib; stdenv.mkDerivation { - name = "dwarffs-0.1.${substring 0 8 inputs.self.lastModified}"; + name = "dwarffs-0.1.${substring 0 8 self.lastModified}"; buildInputs = [ fuse nix nlohmann_json boost ]; NIX_CFLAGS_COMPILE = "-I ${nix.dev}/include/nix -include ${nix.dev}/include/nix/config.h -D_FILE_OFFSET_BITS=64"; - src = inputs.self; + src = self; installPhase = '' @@ -170,7 +168,7 @@ installable derivation) and a NixOS module. ''; }; - nixosModules.dwarffs = import ./module.nix inputs; + nixosModules.dwarffs = ...; defaultPackage = packages.dwarffs; @@ -186,13 +184,13 @@ A flake has the following attributes: flakes to change in the future. It also enables some evolution of the Nix language; for example, the Nix files in the flake could be parsed using a syntax determined by the edition. The only currently - allowed value is `201906`. Nix rejects flakes with an unsupported + allowed value is `201909`. Nix rejects flakes with an unsupported edition. * `description`: A short description of the flake. -* `inputs`: The dependencies of the flake, as a list of flake - references (described below). +* `inputs`: An attrset specifying the dependencies of the flake + (described below). * `outputs`: A function that, given an attribute set containing the outputs of each of the input flakes keyed by their identifier, @@ -227,9 +225,6 @@ A flake has the following attributes: useful for generating (hopefully) monotonically increasing version strings. - The special input named `self` refers to the outputs and source tree - of *this* flake. - A number of outputs have a special meaning, as discussed below. ## Well-known outputs @@ -261,12 +256,65 @@ Hydra. Currently, these are: TODO: NixOS-related outputs such as `nixosModules` and `nixosSystems`. +## Flake inputs + +The attribute `inputs` specifies the dependencies of a flake. These +specify the location of the dependency using a flake reference (see +below). For example, the following specifies a dependency on the Nixpkgs +and Hydra repositories: + + # An indirection through the flake registry. + inputs.nixpkgs.uri = "nixpkgs"; + + # A GitHub repository. + inputs.import-cargo.uri = "github:edolstra/import-cargo"; + +If the `uri` attribute is omitted, it defaults to the attribute +name. Thus, + + inputs.nixpkgs = {}; + +is equivalent to `inputs.nixpkgs.uri = "nixpkgs";`. + +Each input is fetched, evaluated and passed to the `outputs` function +as a set of attributes with the same name as the corresponding +input. The special input named `self` refers to the outputs and source +tree of *this* flake. Thus, a typical `outputs` function looks like +this: + + outputs = { self, nixpkgs, import-cargo }: { + ... outputs ... + }; + +It is also possible to omit inputs entirely and *only* list them as +expected function arguments in `outputs`. Thus, + + outputs = { self, nixpkgs }: ...; + +without an `inputs.nixpkgs` attribute will simply look up `nixpkgs` in +the flake registry. + +Repositories that don't contain a `flake.nix` can also be used as +inputs, by setting the input's `flake` attribute to `false`: + + inputs.grcov = { + uri = github:mozilla/grcov; + flake = false; + }; + + outputs = { self, nixpkgs, grcov }: { + packages.grcov = stdenv.mkDerivation { + src = grcov; + ... + }; + }; + ## Flake references Flake references are a vaguely URL-like syntax to specify which flake to use. This is used on the command line (e.g. in `nix build nixpkgs:hello`, `nixpkgs` is a flake reference), and in the list of -flake dependencies in `flake.nix` (e.g. in `inputs = [ "nixpkgs" ];`). +flake dependencies in `flake.nix` (e.g. in `inputs.nixpkgs = {};`). Currently the following types of flake references are supported: @@ -447,24 +495,20 @@ in `flake.nix` to direct flake references that contain revisions. For example, if `flake.nix` contains ``` -inputs = - [ "nixpkgs" - github:edolstra/import-cargo - ]; +inputs.nixpkgs = {}; +inputs.import-cargo.uri = "github:edolstra/import-cargo"; ``` then the resulting lock file might be: ``` { "version": 2, "inputs": { - "github:edolstra/import-cargo": { - "id": "import-cargo", + "import-cargo": { "inputs": {}, "narHash": "sha256-mxwKMDFOrhjrBQhIWwwm8mmEugyx/oVlvBH1CKxchlw=", "uri": "github:edolstra/import-cargo/c33e13881386931038d46a7aca4c9561144d582e" }, "nixpkgs": { - "id": "nixpkgs", "inputs": {}, "narHash": "sha256-p7UqhvhwS5MZfqUbLbFm+nfG/SMJrgpNXxWpRMFif8c=", "uri": "github:NixOS/nixpkgs/4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082" From 2bdbe7008f669f4460c2f3d0147656ef7816ddc4 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Fri, 20 Sep 2019 18:21:42 +0200 Subject: [PATCH 22/41] Change flakeref/attrpath seperator from ':' to '#' to reduce ambiguity --- rfcs/0049-flakes.md | 38 ++++++++++++++++++++------------------ 1 file changed, 20 insertions(+), 18 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 7a33adb3e..abba8ec11 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -24,7 +24,7 @@ mechanisms such as Nix channels and the Nix search path. * Flakes replace channels. For example, - # nix run nixpkgs:hello -c hello + # nix run nixpkgs#hello -c hello fetches the latest version of [the `nixpkgs` flake](https://github.com/edolstra/nixpkgs/blob/release-19.03/flake.nix) @@ -46,7 +46,7 @@ mechanisms such as Nix channels and the Nix search path. * For a reproducible result, you can also use a specific revision: ``` - # nix build nixpkgs/a0e1f50e6f72e5037d71a0b65c67cf0605349a06:hello + # nix build nixpkgs/a0e1f50e6f72e5037d71a0b65c67cf0605349a06#hello ``` * To get information about a flake: @@ -399,11 +399,11 @@ Flake registries map symbolic flake identifiers (e.g. `nixpkgs`) to "direct" flake references (i.e. any type of flake reference that's not an indirection). This is a convenience to users, allowing them to do - nix run nixpkgs:hello + nix run nixpkgs#hello rather than - nix run github:NixOS/nixpkgs:hello + nix run github:NixOS/nixpkgs#hello There are multiple registries: @@ -452,24 +452,24 @@ changes. The legacy commands (`nix-build`, `nix-shell`, `nix-env` and Most `nix` subcommands work on a list of arguments called "installables" for lack of a better word. For example, - # nix run nixpkgs:hello dwarffs:dwarffs + # nix run nixpkgs#hello dwarffs#dwarffs takes two flake-based installables. The general form is: - (:)? + (#)? Examples of installables: -* `nixpkgs:packages.hello` -* `nixpkgs:hello` - short for `nixpkgs:packages.hello` -* `nixpkgs/release-19.03:hello` - overrides the Git branch to use -* `github:NixOS/nixpkgs/4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082:hello` - +* `nixpkgs#packages.hello` +* `nixpkgs#hello` - short for `nixpkgs:packages.hello` +* `nixpkgs/release-19.03#hello` - overrides the Git branch to use +* `github:NixOS/nixpkgs/4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082#hello` - specifies the exact Git revision to use -* `dwarffs` - short for `dwarffs:defaultPackage` -* `nix:hydraJobs.build.x86_64-darwin` -* `.:hydraJobs.build.x86_64-darwin` - refers to the flake in the +* `dwarffs` - short for `dwarffs#defaultPackage` +* `nix#hydraJobs.build.x86_64-darwin` +* `.#hydraJobs.build.x86_64-darwin` - refers to the flake in the current directory (which can be a dirty Git tree) -* `.` - short for `.:defaultPackage` +* `.` - short for `.#defaultPackage` If no argument is given, the default is `.`; thus, @@ -477,7 +477,7 @@ If no argument is given, the default is `.`; thus, is equivalent to - # nix build .:defaultPackage + # nix build .#defaultPackage For backwards compatibility, it's possible to use non-flake Nix expressions using `-f`, e.g. `nix build -f foo.nix foo.bar`. @@ -501,17 +501,19 @@ inputs.import-cargo.uri = "github:edolstra/import-cargo"; then the resulting lock file might be: ``` { - "version": 2, + "version": 3, "inputs": { "import-cargo": { "inputs": {}, "narHash": "sha256-mxwKMDFOrhjrBQhIWwwm8mmEugyx/oVlvBH1CKxchlw=", - "uri": "github:edolstra/import-cargo/c33e13881386931038d46a7aca4c9561144d582e" + "uri": "github:edolstra/import-cargo/c33e13881386931038d46a7aca4c9561144d582e", + "originalUri": "github:edolstra/import-cargo" }, "nixpkgs": { "inputs": {}, "narHash": "sha256-p7UqhvhwS5MZfqUbLbFm+nfG/SMJrgpNXxWpRMFif8c=", - "uri": "github:NixOS/nixpkgs/4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082" + "uri": "github:NixOS/nixpkgs/4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082", + "originalUri": "nixpkgs" } } } From e09a75e3b82d67cd35e796c4c9dc1b7a97806626 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Tue, 1 Oct 2019 11:55:15 +0200 Subject: [PATCH 23/41] Drop Cargo/NPM comparison --- rfcs/0049-flakes.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index abba8ec11..82c776e7d 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -113,9 +113,7 @@ Flakes are motivated by a number of serious shortcomings in Nix: users cannot easily pin specific versions of channels, channels interact in *ad hoc* ways with the Nix search path, and so on. -The flakes mechanism seeks to address all these problems. It can be -seen as a "Cargo/NPM/... for Nix" except that it's built into Nix -rather than a separate tool. +The flakes mechanism seeks to address all these problems. # Detailed design [design]: #detailed-design From fe0386df16efa1b203b043fb6ef7eba60a230945 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Tue, 1 Oct 2019 11:55:44 +0200 Subject: [PATCH 24/41] Address why flakes shouldn't be done in a separate tool --- rfcs/0049-flakes.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 82c776e7d..c869a544a 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -660,6 +660,17 @@ simpler format such as JSON or TOML. This avoids the Turing tarpit where getting flake metadata requires the execution of an arbitrarily complex, possibly non-terminating program. +Flakes could be implemented as an external tool on top of Nix. Indeed, +there is nothing that flakes allow you to do that couldn't previously +be done using `fetchGit`, the `--pure-eval` flag and some shell +scripting. However, implementing flake-like functionality in an +external tool would defeat the goals of this RFC. First, it probably +wouldn't lead to a standard way to structure and compose Nix projects, +since we might well end up with numerous competing +"standards". Second, it would degrade rather than improve the Nix UX, +since users would now have to deal with Nix *and* the flake-like tool +on top of it. + # Unresolved questions [unresolved]: #unresolved-questions From 7e9497c8100a517773e3204d3238946285adae49 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Tue, 8 Oct 2019 16:32:09 +0200 Subject: [PATCH 25/41] uri -> url --- rfcs/0049-flakes.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index c869a544a..26f4b9134 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -262,17 +262,17 @@ below). For example, the following specifies a dependency on the Nixpkgs and Hydra repositories: # An indirection through the flake registry. - inputs.nixpkgs.uri = "nixpkgs"; + inputs.nixpkgs.url = "nixpkgs"; # A GitHub repository. - inputs.import-cargo.uri = "github:edolstra/import-cargo"; + inputs.import-cargo.url = "github:edolstra/import-cargo"; -If the `uri` attribute is omitted, it defaults to the attribute +If the `url` attribute is omitted, it defaults to the attribute name. Thus, inputs.nixpkgs = {}; -is equivalent to `inputs.nixpkgs.uri = "nixpkgs";`. +is equivalent to `inputs.nixpkgs.url = "nixpkgs";`. Each input is fetched, evaluated and passed to the `outputs` function as a set of attributes with the same name as the corresponding @@ -296,7 +296,7 @@ Repositories that don't contain a `flake.nix` can also be used as inputs, by setting the input's `flake` attribute to `false`: inputs.grcov = { - uri = github:mozilla/grcov; + url = github:mozilla/grcov; flake = false; }; @@ -422,7 +422,7 @@ A registry is a JSON file that looks like this: "version": 1, "flakes": { "nixpkgs": { - "uri": "github:NixOS/nixpkgs" + "url": "github:NixOS/nixpkgs" }, ... } @@ -494,7 +494,7 @@ in `flake.nix` to direct flake references that contain revisions. For example, if `flake.nix` contains ``` inputs.nixpkgs = {}; -inputs.import-cargo.uri = "github:edolstra/import-cargo"; +inputs.import-cargo.url = "github:edolstra/import-cargo"; ``` then the resulting lock file might be: ``` @@ -504,14 +504,14 @@ then the resulting lock file might be: "import-cargo": { "inputs": {}, "narHash": "sha256-mxwKMDFOrhjrBQhIWwwm8mmEugyx/oVlvBH1CKxchlw=", - "uri": "github:edolstra/import-cargo/c33e13881386931038d46a7aca4c9561144d582e", - "originalUri": "github:edolstra/import-cargo" + "url": "github:edolstra/import-cargo/c33e13881386931038d46a7aca4c9561144d582e", + "originalUrl": "github:edolstra/import-cargo" }, "nixpkgs": { "inputs": {}, "narHash": "sha256-p7UqhvhwS5MZfqUbLbFm+nfG/SMJrgpNXxWpRMFif8c=", - "uri": "github:NixOS/nixpkgs/4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082", - "originalUri": "nixpkgs" + "url": "github:NixOS/nixpkgs/4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082", + "originalUrl": "nixpkgs" } } } From c24dca83ae3b2c211edb26d1519a20788d3e3bb2 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Mon, 4 Nov 2019 17:20:50 +0100 Subject: [PATCH 26/41] Big simplification This removes: * Discussion of UI (`nix`) changes. The `nix` CLI is experimental anyway so no need to discuss it in an RFC. * Flake references. These are mainly of use in the CLI. In flake.nix, we can use attrsets instead (e.g. `{ type = "github"; owner = "NixOS"; repo = "nixpkgs"; }` instead of `github:NixOS/nixpkgs`). * The well-known outputs. The flake mechanism doesn't care about what the outputs are, only tools (like `nix`) care about particular outputs. * Most of the discussion about registries. There are still indirect flake inputs (e.g. `nixpkgs`) but how these are resolved is an implementation detail of tools that update the lock file. --- rfcs/0049-flakes.md | 507 ++++++++------------------------------------ 1 file changed, 87 insertions(+), 420 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 26f4b9134..9cd00c049 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -17,76 +17,6 @@ reproducible evaluation of multi-repository Nix projects; impose a discoverable, standard structure on Nix projects; and replace previous mechanisms such as Nix channels and the Nix search path. -# TLDR - -* A prototype implementation is available in Nixpkgs: `nix run - nixpkgs.nixFlakes`. - -* Flakes replace channels. For example, - - # nix run nixpkgs#hello -c hello - - fetches the latest version of [the `nixpkgs` - flake](https://github.com/edolstra/nixpkgs/blob/release-19.03/flake.nix) - from GitHub and builds its `hello` package. Similarly, Nix itself is - a flake, so you can get the latest version as follows: - - # nix run nix - -* Flakes are looked up in a - [registry](https://raw.githubusercontent.com/NixOS/flake-registry/master/flake-registry.json) - that maps identifiers such as `nixpkgs` to actual locations such as - `github:edolstra/nixpkgs/release-19.03`. You can use such locations - ("flake references") directly in the `nix` command: - - ``` - # nix build github:NixOS/patchelf - ``` - -* For a reproducible result, you can also use a specific revision: - - ``` - # nix build nixpkgs/a0e1f50e6f72e5037d71a0b65c67cf0605349a06#hello - ``` - -* To get information about a flake: - - ``` - # nix flake info nixpkgs - Description: A collection of packages for the Nix package manager - Revision: a0e1f50e6f72e5037d71a0b65c67cf0605349a06 - ... - ``` - -* In addition to the global registry, there is a per-user - registry. This can be used to pin flakes to the current version: - ``` - # nix flake pin nixpkgs - ``` - or to a specific version: - ``` - # nix flake add nixpkgs github:edolstra/nixpkgs/a0e1f50e6f72e5037d71a0b65c67cf0605349a06 - ``` - -* Flakes can have dependencies on other flakes. For example, [the - `patchelf` - flake](https://github.com/NixOS/patchelf/blob/master/flake.nix) - depends on the `nixpkgs` flake. To ensure reproducibility, - dependencies are pinned to specific versions using a *lock file* - (e.g. [for - patchelf](https://github.com/NixOS/patchelf/blob/master/flake.lock)). Lock - files are generated automatically. - -* Flakes are evaluated in pure mode, meaning they can't access - anything other than their own source or declared dependencies. This - allows `nix` to cache evaluation results. - -* The `nix` command is flake-based. For example, - ``` - # nix build - ``` - builds the flake in the current directory. - # Motivation [motivation]: #motivation @@ -113,7 +43,9 @@ Flakes are motivated by a number of serious shortcomings in Nix: users cannot easily pin specific versions of channels, channels interact in *ad hoc* ways with the Nix search path, and so on. -The flakes mechanism seeks to address all these problems. +The flakes mechanism seeks to address all these problems. This RFC, +however, only describes the format and semantics of flakes; it doesn't +describe changes to the `nix` command to support flakes. # Detailed design [design]: #detailed-design @@ -223,56 +155,29 @@ A flake has the following attributes: useful for generating (hopefully) monotonically increasing version strings. - A number of outputs have a special meaning, as discussed below. - -## Well-known outputs - -A number of outputs have a specific meaning to Nix or other tools like -Hydra. Currently, these are: - -* `packages`: A set of derivations used as a default by most `nix` - commands. For example, `nix run nixpkgs:hello` uses the - `packages.hello` attribute of the `nixpkgs` flake. It cannot contain - any non-derivation attributes. This also means it cannot be a nested - set! (The rationale is that supporting nested sets requires Nix to - evaluate each attribute in the set, just to discover which packages - are provided.) - -* `defaultPackage`: A derivation used as a default by most `nix` - commands if no attribute is specified. For example, `nix run - dwarffs` uses the `defaultPackage` attribute of the `dwarffs` flake. - -* `checks`: A non-nested set of derivations built by the `nix flake - check` command, and by Hydra if a flake does not have a `hydraJobs` - attribute. - -* `hydraJobs`: A nested set of derivations built by Hydra. - -* `devShell`: A derivation that defines the shell environment used by - `nix dev-shell` if no specific attribute is given. If it does not - exist, then `nix dev-shell` will use `defaultPackage`. - -TODO: NixOS-related outputs such as `nixosModules` and `nixosSystems`. + The value returned by the `outputs` function must be an attribute + set. The attributes can have arbitrary values; however, some tools + may require specific attributes to have a specific value (e.g. the + `nix` command may expect the value of `packages` to be an attribute + set of derivations). ## Flake inputs The attribute `inputs` specifies the dependencies of a flake. These -specify the location of the dependency using a flake reference (see -below). For example, the following specifies a dependency on the Nixpkgs -and Hydra repositories: - - # An indirection through the flake registry. - inputs.nixpkgs.url = "nixpkgs"; +specify the location of the dependency, or a symbolic flake identifier +that is looked up in a registry or in a command-line flag. For +example, the following specifies a dependency on the Nixpkgs and Hydra +repositories: # A GitHub repository. - inputs.import-cargo.url = "github:edolstra/import-cargo"; - -If the `url` attribute is omitted, it defaults to the attribute -name. Thus, - - inputs.nixpkgs = {}; + inputs.import-cargo = { + type = "github"; + owner = "edolstra"; + repo = "import-cargo"; + }; -is equivalent to `inputs.nixpkgs.url = "nixpkgs";`. + # An indirection through the flake registry. + inputs.nixpkgs.id = "nixpkgs"; Each input is fetched, evaluated and passed to the `outputs` function as a set of attributes with the same name as the corresponding @@ -296,7 +201,9 @@ Repositories that don't contain a `flake.nix` can also be used as inputs, by setting the input's `flake` attribute to `false`: inputs.grcov = { - url = github:mozilla/grcov; + type = "github"; + owner = "mozilla"; + repo = "grcov"; flake = false; }; @@ -307,238 +214,90 @@ inputs, by setting the input's `flake` attribute to `false`: }; }; -## Flake references - -Flake references are a vaguely URL-like syntax to specify which flake -to use. This is used on the command line (e.g. in `nix build -nixpkgs:hello`, `nixpkgs` is a flake reference), and in the list of -flake dependencies in `flake.nix` (e.g. in `inputs.nixpkgs = {};`). - -Currently the following types of flake references are supported: - -* Git repositories. These have the form - - git(+http|+https|+ssh|+git|+file|):(//)?(\?)? - - `` are a list of key/value pairs in URI query parameter - syntax. The following parameters are supported: - - * `ref`: The branch or tag to fetch. The default is `master`. - * `rev`: The Git commit hash to fetch. Note that this commit must be - an ancestor of `ref`, since Nix doesn't clone the entire - repository, only the specified `ref` (and the Git protocol doesn't - allow fetching a `rev` without a known `ref`). The default is the - commit currently pointed to by `ref`. - * `dir`: The subdirectory of the repository in which `flake.nix` is - located. This parameter enables having multiple flakes in a - repository. The default is the root directory. - - For example, the following are valid Git flake references: - - * `git+https://example.org/my/repo` - * `git+https://example.org/my/repo?dir=flake1` - * `git+ssh://git@github.com:NixOS/nix?ref=v1.2.3` - * `git://github.com/edolstra/dwarffs?ref=unstable&rev=e486d8d40e626a20e06d792db8cc5ac5aba9a5b4` - * `git+file:///home/my-user/some-repo/some-repo` - -* Local paths. These have the form - - (\?` must refer to (a subdirectory of) a Git repository. - These differ from `file://` flake references in a few ways: - - * They refer to the working tree (unless an explicit `rev` or `ref` - is specified), so evaluation can access dirty files. (Dirty files - are files that are tracked by git but have uncommitted changes.) - - * The `dir` parameter is automatically derived. For example, if - `/foo/bar` is a Git repository, then the flake reference - `/foo/bar/flake` is equivalent to `/foo/bar?dir=flake`. - -* GitHub repositories. These are downloaded as tarball archives, - rather than through Git. This is often much faster and uses less - disk space since it doesn't require fetching the entire history of - the repository. On the other hand, it doesn't allow incremental - fetching (but full downloads are often faster than incremental - fetches!). The syntax is: - - github:/(/)?(\?)? - - `` specifies the name of a branch or tag (`ref`), or a - commit hash (`rev`). Note that unlike Git, GitHub allows fetching by - commit hash without specifying a branch or tag. - - The only supported parameter is `dir` (see above). - - Some examples: - - * `github:edolstra/dwarffs` - * `github:edolstra/dwarffs/unstable` - * `github:edolstra/dwarffs/d3f2baba8f425779026c6ec04021b2e927f61e31` - -* Indirections through the flake registry. These have the form - - (/(/rev)?)? - - These perform a lookup of `` in the flake registry. The - specified `rev` and/or `ref` are then merged with the entry in the - registry. (See below.) For example, `nixpkgs` and - `nixpkgs/release-19.03` are indirect flake references. - -In the future, we should also add tarball flake references -(e.g. `https://example.org/nixpkgs.tar.xz`). It would also be -straight-forward to add flake references to Mercurial repositories -since Nix already has Mercurial support. - -## Flake registries - -Flake registries map symbolic flake identifiers (e.g. `nixpkgs`) to -"direct" flake references (i.e. any type of flake reference that's not -an indirection). This is a convenience to users, allowing them to do - - nix run nixpkgs#hello - -rather than - - nix run github:NixOS/nixpkgs#hello - -There are multiple registries: - -* The global registry - https://raw.githubusercontent.com/NixOS/flake-registry/master/flake-registry.json. (This - location can be overriden via the `flake-registry` option.) Nix - automatically fetches this registry periodically. The check interval - is determined by the `tarball-ttl` option. - -* The local registry `$XDG_CONFIG_HOME/nix/registry.json`. - -* Registry entries specified on the command line. - -A registry is a JSON file that looks like this: - -``` -{ - "version": 1, - "flakes": { - "nixpkgs": { - "url": "github:NixOS/nixpkgs" - }, - ... - } -} -``` - -With this registry, flake references resolve as follows: - -* `nixpkgs` -> `github:NixOS/nixpkgs` -* `nixpkgs/release-19.03` -> `github:NixOS/nixpkgs/release-19.03` -* `nixpkgs/f1c995e694685d6dfb877f6428d3e050d30e253c` -> `github:NixOS/nixpkgs/f1c995e694685d6dfb877f6428d3e050d30e253c` - -The registries are searched in reverse order. Thus the local registry -overrides the global registry, and the command line takes precedence -over the local and global registries. - -## `nix` command line interface - -I propose to make flakes the primary way to specify packages in the -`nix` command line interface. Note that the `nix` UI is still marked -as experimental, so we have some freedom to make incompatible -changes. The legacy commands (`nix-build`, `nix-shell`, `nix-env` and -`nix-instantiate`) should not be changed to avoid breakage. - -Most `nix` subcommands work on a list of arguments called -"installables" for lack of a better word. For example, - - # nix run nixpkgs#hello dwarffs#dwarffs - -takes two flake-based installables. The general form is: +The following input types are specified at present: - (#)? +* `git`: A Git repository or dirty local working tree. -Examples of installables: +* `github`: A more efficient scheme to fetch repositories from GitHub + as tarballs. These have slightly different semantics from `git` + (in particular, the `revCount` attribute is not available). -* `nixpkgs#packages.hello` -* `nixpkgs#hello` - short for `nixpkgs:packages.hello` -* `nixpkgs/release-19.03#hello` - overrides the Git branch to use -* `github:NixOS/nixpkgs/4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082#hello` - - specifies the exact Git revision to use -* `dwarffs` - short for `dwarffs#defaultPackage` -* `nix#hydraJobs.build.x86_64-darwin` -* `.#hydraJobs.build.x86_64-darwin` - refers to the flake in the - current directory (which can be a dirty Git tree) -* `.` - short for `.#defaultPackage` +* `tarball`: A `.tar.{gz,xz,bz2}` file. -If no argument is given, the default is `.`; thus, +* `path`: A directory in the file system. This generally should be + avoided in favor of `git` inputs, since `path` inputs have no + concept of revisions (only a content hash) or tracked files + (anything in the source directory is copied). - # nix build +* `hg`: A Mercurial repository. -is equivalent to +## Lock files - # nix build .#defaultPackage +Inputs specified in `flake.nix` are typically "unlocked" in that they +don't specify an exact revision. To ensure reproducibility, Nix will +automatically generate and use a *lock file* called `flake.lock` in +the flake's directory. The lock file contains a tree of mappings from +the inputs specified in `flake.nix` to inputs specifications that +contain revisions. -For backwards compatibility, it's possible to use non-flake Nix -expressions using `-f`, e.g. `nix build -f foo.nix foo.bar`. - -## Lock files and reproducible evaluation - -Inputs specified in `flake.nix` are typically "unlocked": they are -flake references that don't specify an exact revision (e.g. `nixpkgs` -rather than -`github:NixOS/nixpkgs/4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082`). To -ensure reproducibility, Nix will automatically generate and use a -*lock file* called `flake.lock` in the flake's directory. The lock -file contains a tree of mappings from the flake references specified -in `flake.nix` to direct flake references that contain revisions. - -For example, if `flake.nix` contains -``` -inputs.nixpkgs = {}; -inputs.import-cargo.url = "github:edolstra/import-cargo"; -``` -then the resulting lock file might be: +For example, if `flake.nix` has the inputs in the example above, then +the resulting lock file might be: ``` { - "version": 3, + "version": 4, "inputs": { "import-cargo": { "inputs": {}, "narHash": "sha256-mxwKMDFOrhjrBQhIWwwm8mmEugyx/oVlvBH1CKxchlw=", - "url": "github:edolstra/import-cargo/c33e13881386931038d46a7aca4c9561144d582e", - "originalUrl": "github:edolstra/import-cargo" + "original": { + "type": "github", + "owner": "edolstra", + "repo": import-cargo" + }, + "resolved": { + "type": "github", + "owner": "edolstra", + "repo": "import-cargo", + "rev": "c33e13881386931038d46a7aca4c9561144d582e" + } }, "nixpkgs": { "inputs": {}, "narHash": "sha256-p7UqhvhwS5MZfqUbLbFm+nfG/SMJrgpNXxWpRMFif8c=", - "url": "github:NixOS/nixpkgs/4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082", - "originalUrl": "nixpkgs" + "original": { + "type": "github", + "owner": "NixOS", + "repo": nixpkgs" + }, + "resolved": { + "type": "github", + "owner": "NixOS", + "repo": "nixpkgs", + "rev": "4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082" + } } } } ``` Thus, when we build this flake, the input `nixpkgs` is mapped to -`github:edolstra/import-cargo/c33e13881386931038d46a7aca4c9561144d582e`. Nix -will also check that the content hash of the input is equal to the one -recorded in the lock file. This check is superfluous for Git -repositories (since the commit hash serves a similar purpose), but for -GitHub archives, we cannot directly check that the contents match the -commit hash. - -Note that lock files are only used at top-level: the `flake.lock` files -in dependencies (if they exist) are ignored. - -When you build a local repository (e.g. `nix build /path/to/repo`), -Nix automatically creates a `flake.lock` file if it doesn't already -exists. It will also update the lock file if inputs are added or -removed. You can pass `--recreate-lock-file` to force Nix to recreate -the lock file from scratch (and thus check for the latest version of -each input). +revision `c33e13881386931038d46a7aca4c9561144d582e` of the +`edolstra/import-cargo` repository on GitHub. Nix will also check that +the content hash of the input is equal to the one recorded in the lock +file. This check is superfluous for Git repositories (since the commit +hash serves a similar purpose), but for GitHub archives, we cannot +directly check that the contents match the commit hash. + +Note that lock files are only used at top-level: the `flake.lock` +files in dependencies (if they exist) are ignored. The lock file +transitively locks direct as well as indirect dependencies. + +## Reproducible evaluation Lock files are not sufficient by themselves to ensure reproducible -evaluation. It is also necessary to prevent certain impurities. In -particular, the `nix` command now defaults to evaluating in "pure" -mode, which means that the following are disallowed: +evaluation. We also need to disallow certain impurities that the Nix +language previously allowed. In particular, the following are +disallowed in a flake: * Access to files outside of the top-level flake or its inputs, as well as paths fetched using `fetchTarball`, `fetchGit` and so on @@ -555,89 +314,6 @@ mode, which means that the following are disallowed: * Use of the Nix search path (`<...>`); composition must be done through flake inputs or `fetchX` builtins. -Pure mode can be disabled by passing `--impure` on the command line. - -## Flake-related commands - -The command `nix flake` has various subcommands for managing -flakes. These are: - -* `nix flake list`: Show all flakes in the global and local registry. - -* `nix flake add `: Add an entry to the - local registry. (Recall that this overrides the global registry.) - For example, - - # nix flake add nixpkgs github:my-repo/my-nixpkgs - - redirects `nixpkgs` to a different repository. Similarly, - - # nix flake add nixpkgs github:NixOS/nixpkgs/444f22ca892a873f76acd88d5d55bdc24ed08757 - - pins `nixpkgs` to a specific revision. - - Note that registries only have an effect on flake references used on - the command line or when lock files are generated or updated. - -* `nix flake remove `: Remove an entry from the local - registry. - -* `nix flake pin `: Compute a locked flake reference and - add it to the local registry. For example: - - # nix flake pin nixpkgs - - will add a mapping like `nixpkgs` -> - `github:NixOS/nixpkgs/444f22ca892a873f76acd88d5d55bdc24ed08757` to - the local registry. - -* `nix flake init`: Create a skeleton `flake.nix` in the current - directory. - -* `nix flake update`: Recreate the lock file from scratch. - -* `nix flake check`: Do some checks on the flake (e.g. check that all - `packages` are really packages), then build the `checks` - derivations. For example, `nix flake check patchelf` fetches the - `patchelf` flake, evaluates it and builds it. - -* `nix flake clone`: Do a `git clone` to obtain the source of the - specified flake, e.g. `nix flake clone dwarffs` will yield a clone - of the `dwarffs` repository. - -## Offline use - -Since flakes replace channels, we have to make sure that they support -offline use at least as well. Channels only require network access -when you do `nix-channel --update`. By contrast, the `nix` command -will periodically fetch the latest version of the global registry and -of the top-level flake (e.g. `nix build nixpkgs:hello` may cause it to -fetch a new version of Nixpkgs). To make sure things work offline: - -* It's not a failure if we can't fetch the registry. - -* It's not a failure if we can't fetch the latest version of a - flake. (For example, `fetchGit` no longer fails if we already have - the specified repository and `ref`.) - -* Fetched flakes are registered as garbage collector roots, so running - the garbage collector while on an airplane will not ruin your day. - -* There is an option `--no-net` to explicitly prevent updating the - registry or doing other things that need the network (such as - substituting). This is the default if there are no configured - non-loopback network interfaces. - -## CI integration - -Flakes are useful for CI systems since `flake.nix` and the lock file -exactly specify the input repositories. For Hydra this means that you -no longer need to configure any jobset inputs; just a top-level flake -reference is enough. For example, the [configuration of the `patchelf` -jobset](https://hydra.nixos.org/jobset/patchelf/master#tabs-configuration) -just points to `github:NixOS/patchelf` and doesn't need to specify -`nixpkgs`. - # Drawbacks [drawbacks]: #drawbacks @@ -674,25 +350,19 @@ on top of it. # Unresolved questions [unresolved]: #unresolved-questions -* How to handle the system type? Currently `x86_64-linux` is - hard-coded everywhere. - -* What to do with Nixpkgs overlays? In principle, overlays would just - be a kind of flake output. - -* More in general, how to handle flake arguments? This must be done in - a way that maintains hermetic evaluation and evaluation caching. - -* What are the criteria for inclusion in the global flake registry? - -* Hammer out the details of NixOS/NixOps support for flakes. +* Should flakes have arguments (like "system type")? This must be done + in a way that maintains hermetic evaluation and evaluation caching. * Currently, if flake dependencies (repositories or branches) get - deleted, rebuilding the flake may fail. (This is similar to + deleted upstream, rebuilding the flake may fail. (This is similar to `fetchurl` referencing a stale URL.) We need a command to gather all flake dependencies and copy them somewhere else (possibly vendor them into the repository of the calling flake). +* Maybe flake metadata should be stored in a `flake.json` or + `flake.toml` file. This would prevent ambiguities when the Nix + language changes in a future edition. + # Future work [future]: #future-work @@ -708,9 +378,6 @@ on top of it. configurations concept (https://gist.github.com/edolstra/29ce9d8ea399b703a7023073b0dbc00d). -* Automatically generate documentation from flakes. This partially - depends on the previous item. - # Acknowledgments Funding for the development of the flakes prototype was provided by From 3058b36fae77a2f394e885f05b9e3f391d8885bd Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Mon, 4 Nov 2019 17:28:10 +0100 Subject: [PATCH 27/41] Bump edition --- rfcs/0049-flakes.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 9cd00c049..c0af9b321 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -67,7 +67,7 @@ installable derivation) and a NixOS module. ``` { - edition = 201909; + edition = 201911; description = "A filesystem that fetches DWARF debug info from the Internet on demand"; @@ -114,7 +114,7 @@ A flake has the following attributes: flakes to change in the future. It also enables some evolution of the Nix language; for example, the Nix files in the flake could be parsed using a syntax determined by the edition. The only currently - allowed value is `201909`. Nix rejects flakes with an unsupported + allowed value is `201911`. Nix rejects flakes with an unsupported edition. * `description`: A short description of the flake. From dd6c657e5c8c684445ad52cd1836b2665e85e779 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Wed, 6 Nov 2019 11:22:33 +0100 Subject: [PATCH 28/41] Tweaks --- rfcs/0049-flakes.md | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index c0af9b321..149766a4a 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -52,12 +52,10 @@ describe changes to the `nix` command to support flakes. ## Flakes -A flake is a Git repository that contains a file named `flake.nix` in -the root directory. (In the future, there might be other types of -flakes, such as Mercurial repositories or tarballs.) `flake.nix` -specifies some metadata about the flake such as dependencies (called -*inputs*), as well as its *outputs* (the Nix values such as packages -or NixOS modules provided by the flake). +A flake is a directory that contains a file named `flake.nix` in the +root directory. `flake.nix` specifies some metadata about the flake +such as dependencies (called *inputs*), as well as its *outputs* (the +Nix values such as packages or NixOS modules provided by the flake). As an example, below is the `flake.nix` of [`dwarffs`](https://github.com/edolstra/dwarffs) (a FUSE filesystem @@ -146,8 +144,8 @@ A flake has the following attributes: * `rev`: The commit hash of the flake's Git repository. * `revCount`: The number of ancestors of the revision `rev`. This is - not available for `github:...` repositories (see below), since - they're fetched as tarballs rather than as Git repositories. + not available for `github` repositories (see below), since they're + fetched as tarballs rather than as Git repositories. * `lastModified`: The commit time of the revision `rev`, in the format `%Y%m%d%H%M%S` (e.g. `20181231100934`). Unlike `revCount`, @@ -328,8 +326,7 @@ configurations. For composition of multi-repository projects, the main alternative is to continue on with explicit `fetchGit` / `fetchTarball` calls to pull in other repositories. However, since there is no explicit listing of -dependencies, this does not provide automatic updating (i.e. there is -no equivalent of `nix flake update`). +dependencies, this does not provide automatic updating. Instead of a `flake.nix`, flakes could store their metadata in a simpler format such as JSON or TOML. This avoids the Turing tarpit From abb27991fda3ba63333b42413242cb988310057c Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Tue, 4 Feb 2020 14:05:29 +0100 Subject: [PATCH 29/41] Update example lock file --- rfcs/0049-flakes.md | 99 ++++++++++++++++++++++++++++----------------- 1 file changed, 61 insertions(+), 38 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 149766a4a..788144e94 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -242,53 +242,76 @@ For example, if `flake.nix` has the inputs in the example above, then the resulting lock file might be: ``` { - "version": 4, - "inputs": { - "import-cargo": { - "inputs": {}, - "narHash": "sha256-mxwKMDFOrhjrBQhIWwwm8mmEugyx/oVlvBH1CKxchlw=", - "original": { - "type": "github", - "owner": "edolstra", - "repo": import-cargo" - }, - "resolved": { - "type": "github", - "owner": "edolstra", - "repo": "import-cargo", - "rev": "c33e13881386931038d46a7aca4c9561144d582e" - } - }, - "nixpkgs": { - "inputs": {}, - "narHash": "sha256-p7UqhvhwS5MZfqUbLbFm+nfG/SMJrgpNXxWpRMFif8c=", - "original": { - "type": "github", - "owner": "NixOS", - "repo": nixpkgs" - }, - "resolved": { - "type": "github", - "owner": "NixOS", - "repo": "nixpkgs", - "rev": "4a7047c6e93e8480eb4ca7fd1fd5a2aa457d9082" - } - } + "version": 4, + "inputs": { + "grcov": { + "info": { + "lastModified": 1580729070, + "narHash": "sha256-235uMxYlHxJ5y92EXZWAYEsEb6mm+b069GAd+BOIOxI=" + }, + "inputs": {}, + "locked": { + "owner": "mozilla", + "repo": "grcov", + "rev": "989a84bb29e95e392589c4e73c29189fd69a1d4e", + "type": "github" + }, + "original": { + "owner": "mozilla", + "repo": "grcov", + "type": "github" + } + }, + "import-cargo": { + "info": { + "lastModified": 1567183309, + "narHash": "sha256-wIXWOpX9rRjK5NDsL6WzuuBJl2R0kUCnlpZUrASykSc=" + }, + "inputs": {}, + "locked": { + "owner": "edolstra", + "repo": "import-cargo", + "rev": "8abf7b3a8cbe1c8a885391f826357a74d382a422", + "type": "github" + }, + "original": { + "owner": "edolstra", + "repo": "import-cargo", + "type": "github" + } + }, + "nixpkgs": { + "info": { + "lastModified": 1580555482, + "narHash": "sha256-OnpEWzNxF/AU4KlqBXM2s5PWvfI5/BS6xQrPvkF5tO8=" + }, + "inputs": {}, + "locked": { + "owner": "edolstra", + "repo": "nixpkgs", + "rev": "7f8d4b088e2df7fdb6b513bc2d6941f1d422a013", + "type": "github" + }, + "original": { + "id": "nixpkgs", + "type": "indirect" + } } + } } ``` Thus, when we build this flake, the input `nixpkgs` is mapped to -revision `c33e13881386931038d46a7aca4c9561144d582e` of the -`edolstra/import-cargo` repository on GitHub. Nix will also check that -the content hash of the input is equal to the one recorded in the lock +revision `7f8d4b088e2df7fdb6b513bc2d6941f1d422a013` of the +`edolstra/nixpkgs` repository on GitHub. Nix will also check that the +content hash of the input is equal to the one recorded in the lock file. This check is superfluous for Git repositories (since the commit hash serves a similar purpose), but for GitHub archives, we cannot directly check that the contents match the commit hash. -Note that lock files are only used at top-level: the `flake.lock` -files in dependencies (if they exist) are ignored. The lock file -transitively locks direct as well as indirect dependencies. +The lock file transitively locks direct as well as indirect +dependencies. However, lock file generation itself *does* use the lock +files of dependencies. ## Reproducible evaluation From b60f889f3ef1d600b78eafcb3242563f816ed4f4 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Tue, 4 Feb 2020 14:07:58 +0100 Subject: [PATCH 30/41] Add system type to examples --- rfcs/0049-flakes.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 788144e94..354c2b8dc 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -70,8 +70,8 @@ installable derivation) and a NixOS module. description = "A filesystem that fetches DWARF debug info from the Internet on demand"; outputs = { self, nixpkgs }: rec { - packages.dwarffs = - with nixpkgs.packages; + packages.x86_64-linux.dwarffs = + with nixpkgs.packages.x86_64-linux; with nixpkgs.builders; with nixpkgs.lib; @@ -98,9 +98,9 @@ installable derivation) and a NixOS module. nixosModules.dwarffs = ...; - defaultPackage = packages.dwarffs; + defaultPackage.x86_64-linux = packages.x86_64-linux.dwarffs; - checks.build = packages.dwarffs; + checks.build = packages.x86_64-linux.dwarffs; }; } ``` @@ -156,8 +156,9 @@ A flake has the following attributes: The value returned by the `outputs` function must be an attribute set. The attributes can have arbitrary values; however, some tools may require specific attributes to have a specific value (e.g. the - `nix` command may expect the value of `packages` to be an attribute - set of derivations). + `nix` command may expect the value of `packages.x86_64-linux` to be + an attribute set of derivations built for the `x86_64-linux` + platform). ## Flake inputs @@ -206,7 +207,7 @@ inputs, by setting the input's `flake` attribute to `false`: }; outputs = { self, nixpkgs, grcov }: { - packages.grcov = stdenv.mkDerivation { + packages.x86_64-linux.grcov = stdenv.mkDerivation { src = grcov; ... }; From 95a9ff71dbafab2e38aad05fed00bfcb52439597 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Tue, 4 Feb 2020 14:10:24 +0100 Subject: [PATCH 31/41] Document narHash attribute --- rfcs/0049-flakes.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 354c2b8dc..370c91014 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -141,7 +141,7 @@ A flake has the following attributes: bypass its outputs entirely and only use the flake mechanism to get its source tree. - * `rev`: The commit hash of the flake's Git repository. + * `rev`: The commit hash of the flake's repository, if applicable. * `revCount`: The number of ancestors of the revision `rev`. This is not available for `github` repositories (see below), since they're @@ -153,6 +153,9 @@ A flake has the following attributes: useful for generating (hopefully) monotonically increasing version strings. + * `narHash`: The SHA-256 (in SRI format) of the NAR serialization of + the flake's source tree. + The value returned by the `outputs` function must be an attribute set. The attributes can have arbitrary values; however, some tools may require specific attributes to have a specific value (e.g. the From 969b0790c6deff2e8ba45be42d2d25a552023d25 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Tue, 4 Feb 2020 14:13:35 +0100 Subject: [PATCH 32/41] Link to require.nix --- rfcs/0049-flakes.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 370c91014..d2b233e8c 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -405,4 +405,7 @@ on top of it. # Acknowledgments Funding for the development of the flakes prototype was provided by -[Target Corporation](https://www.target.com/). +[Target Corporation](https://www.target.com/). The flakes project was +inspired/motivated by [Shea Levy's work on +`require.nix`](https://www.youtube.com/watch?v=DHOLjsyXPtM) and +ensuing discussions at NixCon 2018. From 3c734b8a5575a3e0839a4ed14e606be66e2ecf87 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Tue, 4 Feb 2020 14:35:44 +0100 Subject: [PATCH 33/41] Document input overrides / 'follows' attribute --- rfcs/0049-flakes.md | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index d2b233e8c..b2b967bbb 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -233,6 +233,38 @@ The following input types are specified at present: * `hg`: A Mercurial repository. +Transivitive inputs can be overriden from a `flake.nix` file. For +example, the following overrides the `nixpkgs` input of the `nixops` +input: + + inputs.nixops.inputs.nixpkgs = { + type = "github"; + owner = "my-org"; + repo = "nixpkgs"; + }; + +It is also possible to "inherit" an input from another input. This is +useful to minimize flake dependencies. For example, the following sets +the `nixpkgs` input of the top-level flake to be equal to the +`nixpkgs` input of the `dwarffs` input of the top-level flake: + + inputs.nixops.follows = "dwarffs/nixpkgs"; + +The value of the `follows` attribute is a `/`-separated sequence of +input names denoting the path of inputs to be followed from the root +flake. + +Overrides and `follows` can be combined, e.g. + + inputs.nixops.inputs.nixpkgs.follows = "dwarffs/nixpkgs"; + +sets the `nixpkgs` input of `nixops` to be the same as the `nixpkgs` +input of `dwarffs`. It is worth noting, however, that it is generally +not useful to eliminate transitive `nixpkgs` flake inputs in this +way. Most flakes provide their functionality through Nixpkgs overlays +or NixOS modules, which are composed into the top-level flake's +`nixpkgs` input; so their own `nixpkgs` input is usually irrelevant. + ## Lock files Inputs specified in `flake.nix` are typically "unlocked" in that they From 07767e6700ea336277bc6d2d03f38bd7c0b86462 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Fri, 6 Mar 2020 14:43:59 +0100 Subject: [PATCH 34/41] Change the lock file to a directed graph This allows circular dependencies between flakes and makes lock files more compact (e.g. you can deduplicate all the 'nixpkgs' inputs). --- rfcs/0049-flakes.md | 138 +++++++++++++++++++++++++++++++++----------- 1 file changed, 103 insertions(+), 35 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index b2b967bbb..ac854fe6f 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -270,35 +270,45 @@ or NixOS modules, which are composed into the top-level flake's Inputs specified in `flake.nix` are typically "unlocked" in that they don't specify an exact revision. To ensure reproducibility, Nix will automatically generate and use a *lock file* called `flake.lock` in -the flake's directory. The lock file contains a tree of mappings from -the inputs specified in `flake.nix` to inputs specifications that -contain revisions. +the flake's directory. The lock file contains a graph structure +isomorphic to the graph of dependencies of the root flake. Each node +in the graph (except the root node) maps the (usually) unlocked input +specifications in `flake.nix` to locked input specifications. Each +node also contains some metadata, such as the dependencies (outgoing +edges) of the node. For example, if `flake.nix` has the inputs in the example above, then the resulting lock file might be: ``` { - "version": 4, - "inputs": { - "grcov": { + "version": 5, + "root": "n1", + "nodes": { + "n1": { + "inputs": { + "nixpkgs": "n2", + "import-cargo": "n3", + "grcov": "n4" + } + }, + "n2": { "info": { - "lastModified": 1580729070, - "narHash": "sha256-235uMxYlHxJ5y92EXZWAYEsEb6mm+b069GAd+BOIOxI=" + "lastModified": 1580555482, + "narHash": "sha256-OnpEWzNxF/AU4KlqBXM2s5PWvfI5/BS6xQrPvkF5tO8=" }, "inputs": {}, "locked": { - "owner": "mozilla", - "repo": "grcov", - "rev": "989a84bb29e95e392589c4e73c29189fd69a1d4e", + "owner": "edolstra", + "repo": "nixpkgs", + "rev": "7f8d4b088e2df7fdb6b513bc2d6941f1d422a013", "type": "github" }, "original": { - "owner": "mozilla", - "repo": "grcov", - "type": "github" + "id": "nixpkgs", + "type": "indirect" } }, - "import-cargo": { + "n3": { "info": { "lastModified": 1567183309, "narHash": "sha256-wIXWOpX9rRjK5NDsL6WzuuBJl2R0kUCnlpZUrASykSc=" @@ -316,38 +326,96 @@ the resulting lock file might be: "type": "github" } }, - "nixpkgs": { + "n4": { "info": { - "lastModified": 1580555482, - "narHash": "sha256-OnpEWzNxF/AU4KlqBXM2s5PWvfI5/BS6xQrPvkF5tO8=" + "lastModified": 1580729070, + "narHash": "sha256-235uMxYlHxJ5y92EXZWAYEsEb6mm+b069GAd+BOIOxI=" }, "inputs": {}, "locked": { - "owner": "edolstra", - "repo": "nixpkgs", - "rev": "7f8d4b088e2df7fdb6b513bc2d6941f1d422a013", + "owner": "mozilla", + "repo": "grcov", + "rev": "989a84bb29e95e392589c4e73c29189fd69a1d4e", "type": "github" }, "original": { - "id": "nixpkgs", - "type": "indirect" - } + "owner": "mozilla", + "repo": "grcov", + "type": "github" + }, + "flake": false } } } ``` -Thus, when we build this flake, the input `nixpkgs` is mapped to -revision `7f8d4b088e2df7fdb6b513bc2d6941f1d422a013` of the -`edolstra/nixpkgs` repository on GitHub. Nix will also check that the -content hash of the input is equal to the one recorded in the lock -file. This check is superfluous for Git repositories (since the commit -hash serves a similar purpose), but for GitHub archives, we cannot -directly check that the contents match the commit hash. - -The lock file transitively locks direct as well as indirect -dependencies. However, lock file generation itself *does* use the lock -files of dependencies. +This graph has 4 nodes: the root flake, and its 3 dependencies. The +nodes have arbitrary labels (e.g. `n1`). The label of the root node of +the graph is specified by the `root` attribute. Nodes contain the +following fields: + +* `info`: Metadata about the source tree. This always includes + `narHash`. It also includes input type-specific attributes such as + the `lastModified` or `revCount`. The main reason for these + attributes is to allow flake inputs to be substituted from a binary + cache: `narHash` allows the store path to be computed, while the + other attributes are necessary because they provide information not + stored in the store path. + +* `inputs`: The dependencies of this node, as a mapping from input + names (e.g. `nixpkgs`) to node labels (e.g. `n2`). + +* `original`: The original input specification from `flake.lock`, as a + set of `builtins.fetchTree` arguments. + +* `locked`: The locked input specification, as a set of + `builtins.fetchTree` arguments. Thus, in the example above, when we + build this flake, the input `nixpkgs` is mapped to revision + `7f8d4b088e2df7fdb6b513bc2d6941f1d422a013` of the `edolstra/nixpkgs` + repository on GitHub. + +* `flake`: A Boolean denoting whether this is a flake or non-flake + dependency. Corresponds to the `flake` attribute in the `inputs` + attribute in `flake.nix`. + +The `info`, `original` and `locked` attributes are omitted for the +root node. This is because we cannot record the commit hash or content +hash of the root flake, since modifying `flake.lock` will invalidate +these. + +The graph representation of lock files allows circular dependencies +between flakes. For example, here are two flakes that reference each +other: +``` +{ + edition = 201909; + inputs.b = ... location of flake B ...; + # Tell the 'b' flake not to fetch 'a' again, to ensure its 'a' is + # *this* 'a'. + inputs.b.inputs.a.follows = ""; + outputs = { self, b }: { + foo = 123 + b.bar; + xyzzy = 1000; + }; +} +``` +and +``` +{ + edition = 201909; + inputs.a = ... location of flake A ...; + inputs.a.inputs.b.follows = ""; + outputs = { self, a }: { + bar = 456 + a.xyzzy; + }; +} +``` + +Lock files transitively lock direct as well as indirect +dependencies. That is, if a lock file exists and is up to date, Nix +will not look at the lock files of dependencies. However, lock file +generation itself *does* use the lock files of dependencies by +default. ## Reproducible evaluation From 421a9cd635a1a7f2b65f20aae9d36e70448782f3 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Fri, 10 Apr 2020 10:26:59 +0200 Subject: [PATCH 35/41] Remove the 'edition' field Future editions of flakes or the Nix language can be supported by renaming flake.nix (e.g. flake-v2.nix). This avoids a bootstrap problem where we don't know which grammar to use to parse flake*.nix. It also allows a project to support multiple flake editions, in theory. --- rfcs/0049-flakes.md | 17 ----------------- 1 file changed, 17 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index ac854fe6f..194b92a3d 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -65,8 +65,6 @@ installable derivation) and a NixOS module. ``` { - edition = 201911; - description = "A filesystem that fetches DWARF debug info from the Internet on demand"; outputs = { self, nixpkgs }: rec { @@ -107,14 +105,6 @@ installable derivation) and a NixOS module. A flake has the following attributes: -* `edition`: A number that specifies the version of the flake - syntax/semantics to be used. This allows the interpretation of - flakes to change in the future. It also enables some evolution of - the Nix language; for example, the Nix files in the flake could be - parsed using a syntax determined by the edition. The only currently - allowed value is `201911`. Nix rejects flakes with an unsupported - edition. - * `description`: A short description of the flake. * `inputs`: An attrset specifying the dependencies of the flake @@ -388,7 +378,6 @@ between flakes. For example, here are two flakes that reference each other: ``` { - edition = 201909; inputs.b = ... location of flake B ...; # Tell the 'b' flake not to fetch 'a' again, to ensure its 'a' is # *this* 'a'. @@ -402,7 +391,6 @@ other: and ``` { - edition = 201909; inputs.a = ... location of flake A ...; inputs.a.inputs.b.follows = ""; outputs = { self, a }: { @@ -490,11 +478,6 @@ on top of it. # Future work [future]: #future-work -* The "edition" feature enables future Nix changes, including language - changes. For example, changing the parsing of multiline strings - (https://github.com/NixOS/nix/pull/2490) could be conditional on the - flake's edition. - * Currently flake outputs are untyped; we only have some conventions about what they should be (e.g. `packages` should be an attribute set of derivations). For discoverability, it would be nice if From 6fa3b7b49b0e09cc3108de7de6d2dbf29383079c Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Sun, 12 Apr 2020 16:30:00 +0200 Subject: [PATCH 36/41] Fix example --- rfcs/0049-flakes.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 194b92a3d..815318389 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -169,7 +169,10 @@ repositories: }; # An indirection through the flake registry. - inputs.nixpkgs.id = "nixpkgs"; + inputs.nixpkgs = { + type = "indirect"; + id = "nixpkgs"; + }; Each input is fetched, evaluated and passed to the `outputs` function as a set of attributes with the same name as the corresponding From bfa627e2466867a4c206ca9f33e8fa1d2984eebf Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Thu, 16 Apr 2020 00:37:26 +0200 Subject: [PATCH 37/41] lastModified -> lastModifiedDate --- rfcs/0049-flakes.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 815318389..33316e89e 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -74,7 +74,7 @@ installable derivation) and a NixOS module. with nixpkgs.lib; stdenv.mkDerivation { - name = "dwarffs-0.1.${substring 0 8 self.lastModified}"; + name = "dwarffs-0.1.${substring 0 8 self.lastModifiedDate}"; buildInputs = [ fuse nix nlohmann_json boost ]; @@ -137,12 +137,15 @@ A flake has the following attributes: not available for `github` repositories (see below), since they're fetched as tarballs rather than as Git repositories. - * `lastModified`: The commit time of the revision `rev`, in the + * `lastModifiedDate`: The commit time of the revision `rev`, in the format `%Y%m%d%H%M%S` (e.g. `20181231100934`). Unlike `revCount`, this is available for both Git and GitHub repositories, so it's useful for generating (hopefully) monotonically increasing version strings. + * `lastModified`: The commit time of the revision `rev` as an integer + denoting the number of seconds since 1970. + * `narHash`: The SHA-256 (in SRI format) of the NAR serialization of the flake's source tree. From 8c36f764fbae6a09b06fe2a35b5828e79e24d950 Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Thu, 14 May 2020 10:56:59 +0200 Subject: [PATCH 38/41] Typo --- rfcs/0049-flakes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 33316e89e..4d07c45d0 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -229,7 +229,7 @@ The following input types are specified at present: * `hg`: A Mercurial repository. -Transivitive inputs can be overriden from a `flake.nix` file. For +Transitive inputs can be overriden from a `flake.nix` file. For example, the following overrides the `nixpkgs` input of the `nixops` input: From f053ecf7b9299b734e5b809a73f88ec077f99b68 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Domen=20Ko=C5=BEar?= Date: Fri, 29 May 2020 14:23:50 +0200 Subject: [PATCH 39/41] Update rfcs/0049-flakes.md Co-authored-by: Jan Tojnar --- rfcs/0049-flakes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 4d07c45d0..88cce5a03 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -161,7 +161,7 @@ A flake has the following attributes: The attribute `inputs` specifies the dependencies of a flake. These specify the location of the dependency, or a symbolic flake identifier that is looked up in a registry or in a command-line flag. For -example, the following specifies a dependency on the Nixpkgs and Hydra +example, the following specifies a dependency on the Nixpkgs and import-cargo repositories: # A GitHub repository. From 0dbf231a0c8898e18d1a4b7ed0d39cadb08e82eb Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Sat, 30 May 2020 01:07:12 +0200 Subject: [PATCH 40/41] Update to lock file version 6 https://github.com/NixOS/nix/commit/950b46821f644eb3f92725460584a3102f356179 --- rfcs/0049-flakes.md | 50 ++++++++++++++++++++------------------------- 1 file changed, 22 insertions(+), 28 deletions(-) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 88cce5a03..944a82b89 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -277,7 +277,7 @@ For example, if `flake.nix` has the inputs in the example above, then the resulting lock file might be: ``` { - "version": 5, + "version": 6, "root": "n1", "nodes": { "n1": { @@ -288,16 +288,14 @@ the resulting lock file might be: } }, "n2": { - "info": { - "lastModified": 1580555482, - "narHash": "sha256-OnpEWzNxF/AU4KlqBXM2s5PWvfI5/BS6xQrPvkF5tO8=" - }, "inputs": {}, "locked": { "owner": "edolstra", "repo": "nixpkgs", "rev": "7f8d4b088e2df7fdb6b513bc2d6941f1d422a013", - "type": "github" + "type": "github", + "lastModified": 1580555482, + "narHash": "sha256-OnpEWzNxF/AU4KlqBXM2s5PWvfI5/BS6xQrPvkF5tO8=" }, "original": { "id": "nixpkgs", @@ -305,16 +303,14 @@ the resulting lock file might be: } }, "n3": { - "info": { - "lastModified": 1567183309, - "narHash": "sha256-wIXWOpX9rRjK5NDsL6WzuuBJl2R0kUCnlpZUrASykSc=" - }, "inputs": {}, "locked": { "owner": "edolstra", "repo": "import-cargo", "rev": "8abf7b3a8cbe1c8a885391f826357a74d382a422", - "type": "github" + "type": "github", + "lastModified": 1567183309, + "narHash": "sha256-wIXWOpX9rRjK5NDsL6WzuuBJl2R0kUCnlpZUrASykSc=" }, "original": { "owner": "edolstra", @@ -323,16 +319,14 @@ the resulting lock file might be: } }, "n4": { - "info": { - "lastModified": 1580729070, - "narHash": "sha256-235uMxYlHxJ5y92EXZWAYEsEb6mm+b069GAd+BOIOxI=" - }, "inputs": {}, "locked": { "owner": "mozilla", "repo": "grcov", "rev": "989a84bb29e95e392589c4e73c29189fd69a1d4e", - "type": "github" + "type": "github", + "lastModified": 1580729070, + "narHash": "sha256-235uMxYlHxJ5y92EXZWAYEsEb6mm+b069GAd+BOIOxI=" }, "original": { "owner": "mozilla", @@ -350,14 +344,6 @@ nodes have arbitrary labels (e.g. `n1`). The label of the root node of the graph is specified by the `root` attribute. Nodes contain the following fields: -* `info`: Metadata about the source tree. This always includes - `narHash`. It also includes input type-specific attributes such as - the `lastModified` or `revCount`. The main reason for these - attributes is to allow flake inputs to be substituted from a binary - cache: `narHash` allows the store path to be computed, while the - other attributes are necessary because they provide information not - stored in the store path. - * `inputs`: The dependencies of this node, as a mapping from input names (e.g. `nixpkgs`) to node labels (e.g. `n2`). @@ -370,14 +356,22 @@ following fields: `7f8d4b088e2df7fdb6b513bc2d6941f1d422a013` of the `edolstra/nixpkgs` repository on GitHub. + It also includes the attribute `narHash`, specifying the expected + contents of the tree in the Nix store (as computed by `nix + hash-path`), and may include input-type-specific attributes such as + the `lastModified` or `revCount`. The main reason for these + attributes is to allow flake inputs to be substituted from a binary + cache: `narHash` allows the store path to be computed, while the + other attributes are necessary because they provide information not + stored in the store path. + * `flake`: A Boolean denoting whether this is a flake or non-flake dependency. Corresponds to the `flake` attribute in the `inputs` attribute in `flake.nix`. -The `info`, `original` and `locked` attributes are omitted for the -root node. This is because we cannot record the commit hash or content -hash of the root flake, since modifying `flake.lock` will invalidate -these. +The `original` and `locked` attributes are omitted for the root +node. This is because we cannot record the commit hash or content hash +of the root flake, since modifying `flake.lock` will invalidate these. The graph representation of lock files allows circular dependencies between flakes. For example, here are two flakes that reference each From ca2845b81814c345945de4be0b36cd5fb4eb080b Mon Sep 17 00:00:00 2001 From: Eelco Dolstra Date: Mon, 1 Jun 2020 15:55:55 +0200 Subject: [PATCH 41/41] Mention GitLab --- rfcs/0049-flakes.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/rfcs/0049-flakes.md b/rfcs/0049-flakes.md index 944a82b89..40d9d184c 100644 --- a/rfcs/0049-flakes.md +++ b/rfcs/0049-flakes.md @@ -220,6 +220,8 @@ The following input types are specified at present: as tarballs. These have slightly different semantics from `git` (in particular, the `revCount` attribute is not available). +* `gitlab`: Like `github`, but for Git repositories hosted on GitLab. + * `tarball`: A `.tar.{gz,xz,bz2}` file. * `path`: A directory in the file system. This generally should be