Installation suggestion of service packages is completely wrong

[roberth]

1. Search for the docker package
2. Find it
3. Read the instructions for NixOS: nix-env -iA nixos.docker
4. Docker doesn't work

Many packages should be installed as system services. Docker is a classic example (that might actually start to support rootless operation), but many other packages should be installed as system services as well: databases, web servers, you name it.

Ideas:

• Add a meta field such as meta.nixosOptionPrefix = "services.postgresql";. This can be used to show a search link that shows all relevant options.

• Add a meta field to indicate whether the package typically needs system privileges to function, necessitating the use of NixOS options. meta.requiresSystemInstallation = true;. This can be used to hide the non-system installation method from the view when appropriate.

• Add a meta field with an example NixOS config for the package. Too much duplication and maintenance overhead. Such examples can be provided in the *.enable option if the module author wishes to do so.

• Somewhat related, show how to install packages in a shell.

• Perhaps also add an example see comment

[SuperSandro2000]

Read the instructions for NixOS: nix-env -iA nixos.docker

We should just remove that text entirely. Even if the service does not require extra configuration that is bad advice.

• Add a meta field to indicate whether the package typically needs system privileges to function

There is no such thing in linux. You probably mean that package must be run under root.
Which would be misleading because eg mkfs.ext4 needs root permissions but no extra configuration.
Most configuration options are to configure things unrelated to privileges like starting systemd services or adding packages to dbus.

Add a meta field with an example NixOS config for the package.

That creates code duplication between modules and packages and adds an extra maintenance and review burden to keep both of them up to date. Also probably only a very small number of packages would leverage this function at all.

If we are already linking to the nixos option prefix why not take an example from there?

Also this opens a giant pandoras box which are good recommendable defaults. Setting services.postgresql.enable to true is enough to get a postgres running but you also want to configure a database, users, exact postgres version and backup.

[roberth]

• Add a meta field to indicate whether the package typically needs system privileges to function

There is no such thing in linux.

I was somewhat intentionally vague here. Generally the answer to "why can I install only via NixOS" is that the package needs some kind of privilege, whether that's UID 0, access to a file descriptor that refers to a privileged TCP port, or just the presence of a config file in /etc.
We could flip this term around and call it policy-free I guess, referring to the original Nix paper, but that doesn't seem to be an accurate use of the term.
How about meta.requiresSystemInstallation?

Add a meta field with an example NixOS config for the package.

duplication

Fair enough. Striking this one out.

[ncfavier]

• Add a meta field such as meta.nixosOptionPrefix = "services.postgresql";. This can be used to show a search link that shows all relevant options.

That's a great idea, kind of a more useful inverse to relatedPackages. For the format, I suggest meta.relatedNixosOptions = [ "services.foobar" "programs.foobar" ];.

Like Sandro, I don't really like the installation instructions on nixos-search. Something about teaching a man to fish.

I think requiresSystemInstallation would only be needed as a caveat emptor if we do keep the instructions, but if we remove them and point at the relevant NixOS options then it's a bit superfluous.

[roberth]

Like Sandro, I don't really like the installation instructions on nixos-search.

In their current form, the instructions are awful. One way is to reduce them to an attribute name; another is to embrace the concept of context aware documentation, which is kind of nice. Maven central has the same feature, where it offers a couple of syntaxes that are ready to copy and paste.

Something about teaching a man to fish.

It is a form of documentation and I don't think pasting in some attribute names inhibits understanding. Playing with it interactively to see how the example changes is a good way of learning, and it can help out with less obvious aspects without getting in the way much. Examples of this are when to use nativeBuildInputs or buildPackages (hidden behind some choice widget), or dealing with packages in nested attrsets in callPackage (ie. if the attr path has a .).

Being able to link to such instructions will also be helpful when answering beginner questions. We should have good docs for "how to install a package" and this could be one location to have such docs. The benefit of doing it here is the available context and interactivity. A single "how to install" document would be quite lengthy and it'd contain a lot of irrelevant text given any specific use case.

EDIT: I've separated this out into a simpler issue with a different scope #508.

[ysndr]

What might be helpful, but but arguably a more heuristic solution would be to search for related options using the search and showing the results as suggested related options in the package field.
This might give wrong options (and we should also be much more specific/restrictive with the search we issue) but can lead people to related options more intuitively.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/should-service-module-instructions-for-nixos-and-standalone-users-appear-on-the-nixpkgs-search-page/21025/3

[rapenne-s]

There is still a value to have the docker binary but not running as a service; you can use it to operate a remote docker daemon. The same apply to many commands like databases or webservers, they provide binaries that could be useful for an end user.

But I agree the nix-env command should go away, maybe replaced by a nix-shell at least.

[gilice]

Hey guys, I'd like to suggest that we split this into two separate issues;

• one regarding the installation instructions
• and one for displaying packages.x system options (completely unrelated I had the same idea and someone pointed me here).

It could work something like this (pseudo code):

let output = "";
for package in packages {
    if keyNotEmpty(package.meta[programName]){
        output += program_warning(package.meta[programName]);
    } else if keyNotEmpty(options.programs[package.name]) {
        output += maybe_program_warning(package.name);
    }

    output += package;

}
render(output);

where

program_warning(x) { "You shouldn't install this package globally. Set 'programs.${x}.enable = true' in your system config." }

and

maybe_program_warning(x) { "Warning: There's a global program setting with the same name. If this looks like what you want, add 'programs.${x}.enable = true' to your system config instead of installing this package." }

are roughly these.

[roberth]

I think any solution will have to start by adding metadata to the packages, because heuristics will inevitably lead to missing cases and more problems.
We'll want to open a tracking issue in nixpkgs for this purpose, to assign and split up the work, following the nixos directory structure. Maybe not before doing a proof of concept though.

[gilice]

I think any solution will have to start by adding metadata to the packages, because heuristics will inevitably lead to missing cases and more problems.

While this is true, I don't really get what you mean with this. The error message would explicitly state that this might not be the exact package you're looking for, but can point one to the right direction and make life easier overall.

Adding an explicit programNames = [] to all packages that don't have a programs.x.enabled options would be too much, in my opinion.

[nrdxp]

fwiw, I've wondering if we should just stick nixos modules for packages directly in their meta, something like meta.nixos.modules and then directly refererence them from their packages in module-list to make the connection explicit.

[roberth]

@riceicetea Maybe you're right, but I think we have the capability to do it right, meaning that heuristics are a distraction. Of course this is highly subjective, but that's how I see it.

fwiw, I've wondering if we should just stick nixos modules for packages directly in their meta, something like meta.nixos.modules and then directly refererence them from their packages in module-list to make the connection explicit.

This is close to my suggestion of doing meta.nixosOptionPrefix = "services.postgresql";, but we'd have to do a pass on all the options to gather in which files they were defined, and use that to determine the prefix for use in the link that points to the options search.

A link to a section in the manual would be even nicer actually, but many modules don't have such a section yet.
Risking scope creep, I'll add that such sections would actually also benefit from a search link, or even duplicating the relevant options' docs, kind of like we do for Nixpkgs config now, ending the service section/chapter with an options reference: https://nixos.org/manual/nixpkgs/unstable/#sec-modify-via-packageOverrides

[gilice]

Another idea I had -if Nixos decides not to implement this idea as it'd be plenty of work to implement and could lead to name clashes over time - is to make an alternative search engine for nixos.
This would be similar to what lib.rs is for crates.io;

maybe an unified search, that looks in all of the categories (packages, options, flakes) at once? One could instantly see that both a package and a programs.x.enabled option is provided for a search query.

[MMesch]

I posted this before on discourse but maybe this is a better place. We had one more idea about this this morning when writing nixos tests during Summer of Nix:

If setup correctly, a NixOS test can startup a full, preconfigured instance of a service with a single command in a VM. It just needs port forwarding through the qemu VM network to be accessible from the host machine, the browser etc. Tests are also known by packages through the passthru argument. So why not expose tests with a particular name, e.g. exampleInstance on the search page with an associated command to run them? The command could be either existing ones, or a new one that directly includes the one-liner that forwards the port. The test snippet itself would serve as template documentation to setup your own configuration. Maybe it could even really be connected to flake templates so that you can just do flake init -t nixpkgs#elasticsearch or something similar.

Thoughts?

[roberth]

Combining tests and examples has not led to a good collection of examples for dockerTools. (Something I've wanted to fix but never get around to, NixOS/nixpkgs#139895)

A more flexible approach is to have passthru.nixosModuleExample = ../../nixos/examples/foo.nix; and then import that module in a test, if feasible. This should lead to cleaner examples as well.

Maybe it could even really be connected to flake templates

I think a template should offer more value than to only wrap an existing example in a trivial way.
Well though-out examples would drown in the sea of cheap examples.

Maybe we could implement a good "ontology" of templates by building them with derivations, but we'll need a better UI than a flat attribute set. I'd want to do it with modules that together construct a derivation that outputs the template files.

[MMesch]

Agree that it would make more sense to separate examples from tests and put them in a different place as long as they are tested. Then examples would act as a doc test and it would probably be a good thing to have more such integration tests on nixpkgs? (no idea)

I think a flake should offer more value than to only wrap an existing example in a trivial way. Well though-out examples would drown in the sea of cheap examples.

I don't really follow here. First I think examples on nixpkgs could be well thought out and second I think, if the templates are in a specific folder they would be neatly separated from potentially better curated ones (although I wonder why someone would not submit this better example directly to nixpkgs/examples then). Saving the user a few commands/clicks can be quite essential to good UX.

Maybe we could implement a good "ontology" of templates by building them with derivations, but we'll need a better UI than a flat attribute set. I'd want to do it with modules that together construct a derivation that outputs the template files.

This sounds nice although too complicated for myself to do :)

[roberth]

flake

I don't really follow

I meant to say template. Edited.

ontology

too complicated

I might be phrasing it in a complicated way. How about: generate templates and sharing some logic between them.

You're right that it would be a different project, not something we're going to solve in this thread. :)

I'll add reference your suggestion in the issue description.
Let's switch this discussion back to the search UI's presentation logic.

[FRidh]

Many packages should be installed as system services. Docker is a classic example (that might actually start to support rootless operation), but many other packages should be installed as system services as well: databases, web servers, you name it.

There are also many libraries which should also not be installed in a library. I think we should have something like

passthru.provides.executables = ...
passthru.provides.libraries = ...

which can be a list of strings or a boolean. I recently introduced

passthru.provides.setupHook = true;

in 33d12e5f0bcc61d8e12d07fd73ad981f0d42ab59 for Python packaging.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nvidia-drivers-update/33872/5

[TLATER]

Just want to share the user story from that discourse thread:

User wants to install more up-to-date nvidia drivers, so on recommendation of someone online goes to search.nixos.org to see what version is actually the current one offered.

Spots the following:

How to install linuxKernel.packages.linux_5_15.nvidia_x11?

(selects option for NixOS configuration)

Add the following Nix code to your NixOS Configuration, usually located in /etc/nixos/configuration.nix

environment.systemPackages = [
  pkgs.linuxKernel.packages.linux_zen.nvidia_x11
];

It's hard to be more misleading. Yes, there are cases where using packages independently of the NixOS configuration is useful, but explicitly telling people to install the nvidia drivers via environment.systemPackages on NixOS is just... not great.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/i-must-be-doing-something-wrong/35819/2

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/george-hotz-checking-out-nix-for-the-first-time/36838/21

[Atemu]

I think that any kind of "installation instructions" should be removed from the package search. We cannot automatically determine how a package can be used nor how the user intends to use it.

The package's attribute name and package metadata should be the only important things to search; its function should be a map of loose metadata search term -> concrete attribute name + metadata.

Now, the installation instructions exist for a reason I believe and that is to tell the user what they can do with the attribute name. This effect must be retained but I think its implementation should change.
I propose to add a little link behind the attribute name instead. A little something like this:

Attribute name: hello ^{How do I use this?}

This link would lead to a manual section of sorts which describes in a generic way what an attribute name even is (that's not immediately obvious) and what you can use it for. The guide would provide a brief overview of the many ways of using a package: Temporary shells, permanent declarative config, imperatively managed profile etc.

This guide should of course also mention that, if the package is a service or other "system component", the user probably doesn't want to install it in some global context such as the nix profile or systemPackages.
This is important because most non-NixOS Linux users will be used to doing precisely that because it's the one and only way to interact with package names in typical FHS distros. Since NixOS breaks common expectations here, we must explicitly tell the user not do that.

I think such a setup would be a lot more helpful than giving the user one or two example commands with the attribute name with no further context. It'd provide a much more granular guidance on a problem the user is consulting the package search for and go a long way towards teaching them "how to fish".

---

On package-related NixOS options: Eventhough we shouldn't infer intended use from this metadata field, links to the related options search should absolutely still be added to the package metadata and shown on the search page.
They could serve as a jump-off point to the options search and can be lead users to using better means of managing the package in question. For example: Did you know that there is a programs.steam option that that avoids many pitfalls compared to just adding pkgs.steam to systemPackages; enabling drivers etc.? You would likely never know if you didn't explicitly search for this option. If the package search prominently displayed the option alongside the package, I think that a user would be much more likely to use it.

There should probably also be a little explainer link on these what the difference between a package and a NixOS option is (again, extremely obvious to us but very likely not obvious at all to a beginner).

[tobiasBora]

I actually just discovered this issue that is related to the issue I raised a few months ago here NixOS/nixpkgs#267284

On the one hand, I'm in favor of keeping instructions directly in the search.nixos.org website: adding a single link to a generic resource would, in my opinion, not be very helpful since it would anyway not help people to identify cases where a simple nix-env/… is not enough. But nothing prevents us from adding a link to a more generic documentation and a warning saying that this documentation is automatically generated and might not be 100% accurate. Another reason that pushes me not to remove the instructions is that some people use the instructions since they can easily forget the nix-env syntax.

In my opinion, there are so many edge cases that we should start with a simple field like:

meta: {
  instructions: "We recommend to enable this driver by adding  `services.udev.packages = [ vktablet ]` and to run after login the executable `vktablet` to start the driver. For more details, read the documentation at ${lib.linkToModule "vktablet"}";
}

This field should be reasonably short (in markdown for simplicity?) to provide a quick insight on how to use the program, while pointing to the documentation of the module if it is needed. Later, we could add more specific fields as proposed above to automatically provide example of configuration etc…

If people are worried about duplication of instructions between the manual and these instructions, why not additionnally generating the manual from these instructions? This way packages with one or 2 lines of instructions could directly put the documentation in the meta field, and packages with larger documentation could just add a link in meta.instructions pointing to the manual.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/services-programs-and-packages-oh-my/38029/2

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/gsconnect-does-not-work-with-gdm/46271/5