Nixxing nix

[dirkson]

Pardon the title, couldn't resist the pun.

I spent an hour trying to alternately let simula's script install nix, then to install it via my package manager. Nothing I tried resulted in anything remotely like a usable nix, despite a bunch of sourcing and rebooting and tearing of hair. No consistent error messages either - It's a fresh new error every time!

Could we get some boring old-style binary releases as an option? It may help distros package this thing, which may get more eyes on it.

Cheers,
Dirk

[lboklin]

I understand this issue isn't specifically about the errors you encountered, but would you mind creating an issue about those + details of your system and the errors produced?

Secondly, did you try installing Nix according to the official instructions?

The reason Nix is used as the one and only distribution method is that it works across all Linux distros equally well (disregarding the hacks employed to make graphics work on non-NixOS systems). At least that's the idea - obviously something isn't working in your case.

Other possible distro-agnostic solutions include AppImage and Flatpak, but they're both objectively worse in reproducibility and complexity/overhead for the sole benefit of the user not having to install Nix (assuming they have Flatpak installed - otherwise it's only trading one package manager to install for another).

Distributions are welcome to package the project in their native packaging format - this is not the responsibility of project maintainers. Given the transparent nature of the project's dependencies, it shouldn't be too difficult for a prospective package maintainer to re-package it for their chosen distro.

[lboklin]

Just realised I didn't address the build process, which is also very much Nix-based.

The project has complex dependencies, with a patched version of Godot and an included development environment. Handling this without the use of Nix (say via Docker or somesuch) would be possible but, again, be worse for (1) reproducibility, (2) maintainability and (3) flexibility.

(1) because Docker does not offer the same level of reproducibility as Nix - upstream images can be updated in place without notice, afaiu, and then you're also implicitly relying on the environment of that image. Docker scripts are also more procedural than Nix expressions which are mostly declarative.

(2) because dependencies are simply more manageable and modular in the form of atomic Nix derivations rather than a collection of imperative procedures potentially spread throughout various parts of some shell-script(s).

(3) because Nix derivations can be updated, overridden and refactored easily and atomically because they are (mostly? I'm not up to date on the current state) separated into their own derivations.

Bonus argument: Caching. Builds are cached with Nix, so mostly you'll be downloading binaries (even for the development environment) from cachix.org (or reusing derivations already present in your /nix/store) rather than rebuilding everything from scratch.

[dirkson]

Secondly, did you try installing Nix according to the official instructions?

Nope, never occurred to me! So I tried that first thing this morning - And was fairly surprised to not only have a working nix install, but a fully functional Simula built with it. This is particularly interesting because, as near as I can tell, you basically do a bog-standard nix install from your script.

One of the things I suspect was happening with Nix yesterday was that it was leaving bits of itself everywhere after a failed install (I see several .nix* directories, an entry added to my .bash_profile, and a /nix directory, at the very least. Plus whatever env variables it's actually leaving via that .bash_profile addition.). I suspect that leftover bits were interfering with newly installed bits, thus the errors.

Honestly I think a 'nix uninstall' script would make the nix install process a lot more, well, reproducible. That's not exactly your problem, but you could probably patch around it with your util script if you felt strongly enough. Or one of us could file an issue upstream about that. (Let me know if that's me!)

I understand this issue isn't specifically about the errors you encountered, but would you mind creating an issue about those + details of your system and the errors produced?

After getting the successful build mentioned above, I can no longer make the build fail, even after removing nix fully (maybe) and following the exact steps that failed last time. (i.e the ones listed in your documentation) Nix is truly a crafty opponent.

The issues I had with my own distro's nix are definitely not your guys' problem, so I don't think it makes sense for me to go into the errors I had (Probably still have?) with it.

AppImage and Flatpak [...] Docker

Yeah, I think I agree that these aren't great solutions.

I was imagining nix as solely a build system, but from the way you talk it's clearly more that that - Something akin to a build system, package manager, and set of libraries combined. Understanding that, I can see why distributing the end-result binaries separately might be an exercise in futility - With the build I saw, you'd basically have to tar up the whole nix install and send it along.

Distributions are welcome to package the project in their native packaging format - this is not the responsibility of project maintainers. Given the transparent nature of the project's dependencies, it shouldn't be too difficult for a prospective package maintainer to re-package it for their chosen distro.

It's not your direct responsibility, but if getting Simula into distros is a design goal, it's probably something that you should have plans around - I.e. A package maintainer should have a way to build a binary that depends on their distro's libraries, rather than nix's, and shouldn't have to learn nix fully to do it. Asking every distro who wants to package Simula to understand what nix is doing and write their own build system files for Simula based on that seems like an excellent way to keep it out of most distros.

The patched godot makes that more complicated, I know, particularly if it's something that can't be upstreamed.

But I do understand wanting to ensure a known-working set of libraries with your software - And at such an early stage, that's probably a smart move to have that as the only option, so that you can focus on bugs in Simula rather than tracking down every version incompatibility under the sun. And I love it as a debug build - Ask a user having bugs to install via nix, and you instantly know whether his problem is version incompatibility or something else, and the caching you mention is great for debug builds. So I'm definitely not suggesting you remove the functionality entirely.

Oh, and an utterly unrelated compliment: You even managed to get the Index's cameras working! No other software I've used has managed to do that, including steamvr itself. Good show!

Cheers,
Dirk

[KaneTW]

I think the Index cameras currently show double. I have a AR view planned that combines them stereoscopically and replaces the background with them, but it's on the backburner right now.

[ChristophHaag]

I would try making Archlinux AUR packages with native dependencies and some basic directions what needs to be built and in which version would be very helpful for that.

[oxr463]

I don't love AppImage format (I'm an Alpine / Debian / Gentoo package maintainer), but I appreciate the simplicity of a single file. Perhaps if you had releases of Simula on GitHub Releases and all I had to do was extract and run a binary, that would be a bit easier than the nix approach. I think it's good to have a portable option for trying new software. I don't like the idea of nix leaving "bits" behind should I decide to remove Simula.

[Atemu]

Perhaps if you had releases of Simula on GitHub Releases and all I had to do was extract and run a binary, that would be a bit easier than the nix approach.

An option that could be explored here would be https://github.com/matthewbauer/nix-bundle which even has experimental support for AppImage export now apparently.
I'm not sure how well it'd work w.r.t. the graphics driver dependency though, if at all.

I don't like the idea of nix leaving "bits" behind should I decide to remove Simula.

In theory, Nix shouldn't leave anything behind. You should be able to just rm -rf /nix and that's it.

In practice, Nix needs to put several files in certain places to make the system recognise its paths.
This isn't really Nix' fault actually but rather the inflexibility of most modern UNIX-ish systems and especially ancient tools like (ba)sh.
The Nix intaller has to put a line into your /etc/profile, /etc/bashrc, .profile and whatnot to be able to do add anything to your PATH reliably. There is no other way.

I fully agree that the uninstall process needs be cleaner (See also NixOS/nix#1402) but the things it has to place are designed to be inert without a Nix store present, so it really isn't as big of a deal as you probably think. This is not even close to a sudo make install type situation.
All the actual programs and functionality are entirely self-contained in /nix/. What's placed statefully around the FS is glue code to get the system to recognise what's under /nix/ and doesn't do anything on its own.
We are very aware of system state like this in the Nix community. NixOS actually reduces it to near zero; you can quite comfortably mount a tmpfs as / by changing only a few lines in your config.

[Atemu]

some basic directions what needs to be built and in which version would be very helpful for that

Just have a look at the .nix files. It's a very declarative language and you should be able to get the gist without knowing its syntactic details.

The arguments to the mkDerivation functions should tell you almost everything you need to know for your AUR packages.

Here's the Godot fork's for example:

https://github.com/SimulaVR/godot/blob/746272792542b1a4041a91129f76aba44696b21f/godot.nix#L31

[johnrichardrinehart]

@KaneTW @georgewsinger I think this can be closed.

[Thesola10]

Wouldn't nix-portable work?