Prepare inclusion in nixos-search #2971

ncfavier · 2022-05-20T12:35:58Z

Description

Lays the groundwork for NixOS/nixos-search#323 by making

nix run nixos-search#flake-info -- flake github:nix-community/home-manager

succeed. Note that this PR does NOT solve the problem of how to expose Home Manager modules/options in the flake, it only ensures that we can index packages and NixOS options.

Best reviewed one commit at a time.

This PR partly supersedes #1934, although it does not add an auto-update workflow.

Tested the NixOS module and the manual. nix-darwin testers welcome.

Checklist

Change is backwards compatible.
Code formatted with ./format.
Code tested through nix-shell --pure tests -A run.all.
Test cases updated/added. See example.
Commit messages are formatted like
```
{component}: {description}

{long description}
```
See CONTRIBUTING for more information and recent commit messages for examples.

teto · 2022-06-05T10:06:13Z

I am happy to see this initiative but why mixing it with format changes ? can these go in another MR ?

Is the following expected ?

$ nix run nixos-search#flake-info -- flake github:ncfavier/home-manager/nixos-searchable
warning: Using saved setting for 'extra-substituters = https://nixos-search.cachix.org' from ~/.local/share/nix/trusted-settings.json.
warning: Using saved setting for 'extra-trusted-public-keys = nixos-search.cachix.org-1:1HV3YF8az4fywnH+pAd+CXFEdpTXtv9WpoivPi+H70o=' from ~/.local/share/nix/trusted-settings.json.
[2022-06-05T10:03:11Z ERROR command_run] command 'nix eval --json --no-write-lock-file -f /tmp/.tmpRoip4S/extract.nix --override-flake input-flake github:ncfavier/home-manager/nixos-searchable --argstr flake github:ncfavier/home-manager/nixos-searchable all' failed: exit status: 1
    stdout:
    
    stderr:
    warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels/nixos' does not exist, ignoring
    error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)
    
           at /tmp/.tmpRoip4S/extract.nix:5:21:
    
                4|
                5|   nixpkgs = (import <nixpkgs> {});
                 |                     ^
                6|   lib = nixpkgs.lib;
    (use '--show-trace' to show detailed location information)
    
Error: Getting flake info caused an error: Failed to gather information about github:ncfavier/home-manager/nixos-searchable

Caused by:
    command 'nix eval --json --no-write-lock-file -f /tmp/.tmpRoip4S/extract.nix --override-flake input-flake github:ncfavier/home-manager/nixos-searchable --argstr flake github:ncfavier/home-manager/nixos-searchable all' failed: exit status: 1

ncfavier · 2022-06-05T11:02:38Z

I am happy to see this initiative but why mixing it with format changes ? can these go in another MR ?

~~I suppose they can, but does it matter? This is just a big clean-up PR.~~ Removed the formatting changes.

Is the following expected ?

Nope, that's a bug in nixos-search. ~~In the meantime you can do export NIX_PATH=nixpkgs=channel:nixpkgs-unstable.~~ Fixed now.

teto · 2022-06-05T11:21:49Z

arf nmd issue:

    error: a 'aarch64-darwin' with features {} is required to build '/nix/store/rsiqrib1rv2shc6azv12xcbkvxs9hj32-nmd.drv', but I am a 'x86_64-linux' with features {benchmark, big-parallel, kvm, nixos-test}
    (use '--show-trace' to show detailed location information)

ncfavier · 2022-06-05T11:34:33Z

Ah, I guess I didn't run into that because that path was substituted by the nix-community cache.

So we should probably add nmd and nmt to the flake inputs instead of having an IFD (which I did initially but then I thought it wasn't needed).

ncfavier · 2022-06-05T12:32:54Z

Should be fixed now

flake.nix

Add a lockfile to work around NixOS/nix#6541 (and because it's a good idea anyway). Also use flake-utils, and restrict ourselves to the five platforms supported by nixpkgs. Otherwise, the IFD for nmd fails on weird platforms. This fixes `nix flake check`. Remove the redundant `apps` output, see nix-community#2442 (comment)

Make sure the option is included in the NixOS/nix-darwin manual (but the HM submodule options aren't). Also add a static description to the HM submodule type so that we don't need to evaluate the submodules just to build the option manual. This makes nixos-search able to index the home-manager flake. Also clean up some TODOs.

This avoids having to use `pkgs.fetchFromGitLab` in an IFD, which causes issues when indexing packages with nixos-search because `pkgs` is instantiated with every platform.

ncfavier · 2022-06-07T18:54:35Z

Thanks for the quick merge! (EDIT: wait, it's been 18 days? where does time go...)

@rycee quick heads-up since I don't know how familiar you are with flakes: inputs like nmd and nmt can now be updated with nix flake lock --update-input nmd (you'll need to enable the flakes and nix-command experimental features). But maybe we want to automate this as part of #1934?

rycee

I don't know much about Flakes so I'll defer to @teto's judgement about merging. I just added a few minor comments that I didn't see discussed, probably because they are too blindingly obvious for people who aren't ignorant about Flakes 🙂

rycee · 2022-06-15T23:17:34Z

default.nix

+  flake = (import
+    (let lock = builtins.fromJSON (builtins.readFile ./flake.lock);
+    in fetchTarball {
+      url =
+        "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
+      sha256 = lock.nodes.flake-compat.locked.narHash;
+    }) { src = ./.; }).defaultNix;


This feels pretty ugly to me. Why not simply change the pkgs.fetchFromGitLab to fetchTarball when fetching nmd and nmt?

This is only a compatibility layer that should progressively become useless as we transition to flakes and the new nix CLI.

Why not simply change the pkgs.fetchFromGitLab to fetchTarball when fetching nmd and nmt?

We could, but if we're going to use flakes, we might as well use them fully, i.e. for all inputs. I think they're a nicer way to manage dependencies, and they make updating and overriding easier.

I agree that eventually it will be worth switching over to flakes but I'm skeptical about delving too deep while the feature is marked as experimental. At least unless there is a really compelling reason. But in this case there is a straight forward way to avoid the pkgs reference without touching flakes and needing compatibility shims.

I also would like to move the nmt and nmd projects over to sourcehut at some point and it doesn't seem like the flake-compat project supports the sourcehut URLs. I'll probably maintain a mirror at the gitlab location so this is not a blocker, though.

It's also not a blocker because we can just use URLs like git+https://git.sr.ht/~rycee/nmt.

I'm skeptical about delving too deep while the feature is marked as experimental.

It is only marked as experimental to not have to create backwards compatibility while it still might change.

rycee · 2022-06-15T23:31:58Z

flake.lock

+    "nixpkgs": {
+      "locked": {
+        "lastModified": 1654230545,
+        "narHash": "sha256-8Vlwf0x8ow6pPOK2a04bT+pxIeRnM1+O0Xv9/CuDzRs=",
+        "owner": "nixos",
+        "repo": "nixpkgs",
+        "rev": "236cc2971ac72acd90f0ae3a797f9f83098b17ec",
+        "type": "github"
+      },
+      "original": {
+        "owner": "nixos",
+        "ref": "nixos-unstable",
+        "repo": "nixpkgs",
+        "type": "github"
+      }
+    },


I want it to be explicit to the user which Nixpkgs they are using in their configuration. I.e., HM should never be responsible for pinning Nixpkgs on behalf of users. So if we are pinning Nixpkgs here wouldn't we also need to change

pkgs ? builtins.getAttr system nixpkgs.outputs.legacyPackages

to simply

pkgs

in the homeManagerConfiguration function?

Ideally, perhaps disallowing

home-manager.url = "github:nix-community/home-manager";

without an accompanying

home-manager.inputs.nixpkgs.follows = "nixpkgs";

would be most reassuring but I don't know if that is possible.

So if we are pinning Nixpkgs here wouldn't we also need to change

Agreed, I'll do that in a follow-up PR.

Ideally, perhaps disallowing

That's not possible, and I think it would be overkill: we only need to ensure that the nixpkgs input is not passed down to the HM modules, but we may (and should) use the locked nixpkgs for flake packages.

integration-common.nix

Prevents Home Manager from pinning nixpkgs on behalf of users, see nix-community#2971 (comment)

ncfavier · 2022-06-16T10:40:36Z

#3028

Prevents Home Manager from pinning nixpkgs on behalf of users, see nix-community#2971 (comment)

Prevents Home Manager from pinning Nixpkgs on behalf of users, see nix-community#2971 (comment)

wmertens · 2022-06-20T06:23:40Z

so what is now the way to pass nixpkgs? It is not clear to me at all. I tried passing pkgs = inputs.nixpkgs.outputs.legacyPackages but then lib is missing.

The docs for flakes haven't been updated yet.

ncfavier · 2022-06-20T08:53:39Z

nixpkgs.legacyPackages.${system}

See #3037 for the docs update

* Add flake.lock and clean up flake.nix Add a lockfile to work around NixOS/nix#6541 (and because it's a good idea anyway). Also use flake-utils, and restrict ourselves to the five platforms supported by nixpkgs. Otherwise, the IFD for nmd fails on weird platforms. This fixes `nix flake check`. Remove the redundant `apps` output, see nix-community#2442 (comment) * nixos,nix-darwin: factor out into a common module * nixos,nix-darwin: make `home-managers.users` shallowly visible Make sure the option is included in the NixOS/nix-darwin manual (but the HM submodule options aren't). Also add a static description to the HM submodule type so that we don't need to evaluate the submodules just to build the option manual. This makes nixos-search able to index the home-manager flake. Also clean up some TODOs. * flake: add nmd and nmt This avoids having to use `pkgs.fetchFromGitLab` in an IFD, which causes issues when indexing packages with nixos-search because `pkgs` is instantiated with every platform.

Prevents Home Manager from pinning Nixpkgs on behalf of users, see nix-community#2971 (comment)

nixos-discourse · 2022-08-15T12:38:44Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/the-nix-module-in-home-manager-and-nixos/20999/11

* Add flake.lock and clean up flake.nix Add a lockfile to work around NixOS/nix#6541 (and because it's a good idea anyway). Also use flake-utils, and restrict ourselves to the five platforms supported by nixpkgs. Otherwise, the IFD for nmd fails on weird platforms. This fixes `nix flake check`. Remove the redundant `apps` output, see nix-community#2442 (comment) * nixos,nix-darwin: factor out into a common module * nixos,nix-darwin: make `home-managers.users` shallowly visible Make sure the option is included in the NixOS/nix-darwin manual (but the HM submodule options aren't). Also add a static description to the HM submodule type so that we don't need to evaluate the submodules just to build the option manual. This makes nixos-search able to index the home-manager flake. Also clean up some TODOs. * flake: add nmd and nmt This avoids having to use `pkgs.fetchFromGitLab` in an IFD, which causes issues when indexing packages with nixos-search because `pkgs` is instantiated with every platform.

Prevents Home Manager from pinning Nixpkgs on behalf of users, see nix-community#2971 (comment)

ncfavier requested review from sumnerevans and rycee as code owners May 20, 2022 12:35

ncfavier mentioned this pull request May 20, 2022

Add home-manager search feature NixOS/nixos-search#323

Open

ncfavier marked this pull request as draft May 21, 2022 12:40

ncfavier force-pushed the nixos-searchable branch from f420bf3 to 8d75e43 Compare May 21, 2022 18:14

ncfavier marked this pull request as ready for review May 21, 2022 18:18

This was referenced May 21, 2022

add flake attribute apps to make it easier to run #2442

Merged

Add flake.lock & workflow to update it #1934

Merged

ncfavier force-pushed the nixos-searchable branch from 176833a to bfb27a7 Compare June 5, 2022 12:31

teto reviewed Jun 7, 2022

View reviewed changes

flake.nix Outdated Show resolved Hide resolved

ncfavier added 4 commits June 7, 2022 20:07

nixos,nix-darwin: factor out into a common module

d71bb80

flake: add nmd and nmt

5b98520

This avoids having to use `pkgs.fetchFromGitLab` in an IFD, which causes issues when indexing packages with nixos-search because `pkgs` is instantiated with every platform.

ncfavier force-pushed the nixos-searchable branch from bfb27a7 to 5b98520 Compare June 7, 2022 18:09

teto merged commit 64ab7d6 into nix-community:master Jun 7, 2022

ncfavier deleted the nixos-searchable branch June 7, 2022 18:49

ncfavier mentioned this pull request Jun 7, 2022

nixos: use user service for activation #2548

Open

7 tasks

thiagokokada mentioned this pull request Jun 9, 2022

bug: man page duplicates option names into beginning of option description text #2820

Closed

1 task

ShamrockLee mentioned this pull request Jun 15, 2022

Questions about the update and release model of the flake.lock-included pattern #3026

Closed

rycee reviewed Jun 15, 2022

View reviewed changes

ncfavier added a commit to ncfavier/home-manager that referenced this pull request Jun 16, 2022

flake: make homeManagerConfiguration require a pkgs argument

243b64c

Prevents Home Manager from pinning nixpkgs on behalf of users, see nix-community#2971 (comment)

ncfavier added a commit to ncfavier/home-manager that referenced this pull request Jun 16, 2022

flake: make homeManagerConfiguration require a pkgs argument

6c89c00

Prevents Home Manager from pinning nixpkgs on behalf of users, see nix-community#2971 (comment)

ncfavier mentioned this pull request Jun 16, 2022

Clean up after #2971 #3028

Merged

rycee pushed a commit to ncfavier/home-manager that referenced this pull request Jun 18, 2022

flake: make homeManagerConfiguration require a pkgs argument

4c785d2

Prevents Home Manager from pinning nixpkgs on behalf of users, see nix-community#2971 (comment)

rycee pushed a commit to ncfavier/home-manager that referenced this pull request Jun 18, 2022

flake: make pkgs required in homeManagerConfiguration

2ff38c6

Prevents Home Manager from pinning Nixpkgs on behalf of users, see nix-community#2971 (comment)

jevy pushed a commit to jevy/home-manager that referenced this pull request Jul 1, 2022

flake: make pkgs required in homeManagerConfiguration

b1a8dab

Prevents Home Manager from pinning Nixpkgs on behalf of users, see nix-community#2971 (comment)

spacekookie pushed a commit to spacekookie/home-manager that referenced this pull request Feb 10, 2023

flake: make pkgs required in homeManagerConfiguration

b1cec0a

Prevents Home Manager from pinning Nixpkgs on behalf of users, see nix-community#2971 (comment)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Prepare inclusion in nixos-search #2971

Prepare inclusion in nixos-search #2971

ncfavier commented May 20, 2022 •

edited

Loading

teto commented Jun 5, 2022

ncfavier commented Jun 5, 2022 •

edited

Loading

teto commented Jun 5, 2022

ncfavier commented Jun 5, 2022 •

edited

Loading

ncfavier commented Jun 5, 2022

ncfavier commented Jun 7, 2022 •

edited

Loading

rycee left a comment

rycee Jun 15, 2022

ncfavier Jun 16, 2022

rycee Jun 18, 2022

ncfavier Jun 18, 2022

SuperSandro2000 Jun 20, 2022

rycee Jun 15, 2022

ncfavier Jun 16, 2022

ncfavier commented Jun 16, 2022

wmertens commented Jun 20, 2022

ncfavier commented Jun 20, 2022 •

edited

Loading

nixos-discourse commented Aug 15, 2022

Prepare inclusion in nixos-search #2971

Prepare inclusion in nixos-search #2971

Conversation

ncfavier commented May 20, 2022 • edited Loading

Description

Checklist

teto commented Jun 5, 2022

ncfavier commented Jun 5, 2022 • edited Loading

teto commented Jun 5, 2022

ncfavier commented Jun 5, 2022 • edited Loading

ncfavier commented Jun 5, 2022

ncfavier commented Jun 7, 2022 • edited Loading

rycee left a comment

Choose a reason for hiding this comment

rycee Jun 15, 2022

Choose a reason for hiding this comment

ncfavier Jun 16, 2022

Choose a reason for hiding this comment

rycee Jun 18, 2022

Choose a reason for hiding this comment

ncfavier Jun 18, 2022

Choose a reason for hiding this comment

SuperSandro2000 Jun 20, 2022

Choose a reason for hiding this comment

rycee Jun 15, 2022

Choose a reason for hiding this comment

ncfavier Jun 16, 2022

Choose a reason for hiding this comment

ncfavier commented Jun 16, 2022

wmertens commented Jun 20, 2022

ncfavier commented Jun 20, 2022 • edited Loading

nixos-discourse commented Aug 15, 2022

ncfavier commented May 20, 2022 •

edited

Loading

ncfavier commented Jun 5, 2022 •

edited

Loading

ncfavier commented Jun 5, 2022 •

edited

Loading

ncfavier commented Jun 7, 2022 •

edited

Loading

ncfavier commented Jun 20, 2022 •

edited

Loading