Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

preferLocalBuild should not override max-jobs=0 #3810

Closed
grahamc opened this issue Jul 14, 2020 · 8 comments · Fixed by #4443
Closed

preferLocalBuild should not override max-jobs=0 #3810

grahamc opened this issue Jul 14, 2020 · 8 comments · Fixed by #4443
Labels

Comments

@grahamc
Copy link
Member

grahamc commented Jul 14, 2020

Is your feature request related to a problem? Please describe.

The documented behavior of max-jobs defines that even with a value of 0, builds with preferLocalBuild = true still build locally. This is a fairly major policy decision that the daemon makes on behalf of the user, and is a decision the user cannot easily override.

Sometimes a machine should perform absolutely no builds, in which case their options to fully disabling builds is to either delete all the build users (not sure nix-daemon would be happy with this) or set the system to a bogus value and assume/hope nobody will use it.

Describe the solution you'd like

I propose we extend the features feature of remote building to local building, too, and use preferLocalBuild as a listed feature.

Concretely, I want to:

  1. extend the nix.conf to accept a mandatory-features option
  2. treat derivations where preferLocalBuild == true as if they had the "local-preferred" feature
  3. add local-preferred to the default value of system-features

Backwards Compatibility

  • The current behavior of max-jobs=0 can be achieved by setting mandatory-features = local-preferred and max-jobs=auto.
  • Other users may want to set max-jobs=0 and assign the local-preferred feature to a specific builder which is nearby.
@Ericson2314
Copy link
Member

I would be fine just changing the behavior of max-jobs = 0, but the desugaring to local-preferred is neat and I like it :)

This is a fairly major policy decision that the daemon makes on behalf of the user, and is a decision the user cannot easily override.

To be clear, is this a daemon thing in particular? What happens if one is not using the daemon, but has remove builders?

@grahamc
Copy link
Member Author

grahamc commented Jul 14, 2020

To be clear, is this a daemon thing in particular? What happens if one is not using the daemon, but has remove builders?

Same behavior :).

@Ericson2314
Copy link
Member

OK whew. That would have made back-compat much more tricky.

@domenkozar
Copy link
Member

domenkozar commented Jan 11, 2021

One reason the user might be using remote builder / binary cache is to avoid HDD space pollution and there's no way to disable that.

@zupo
Copy link

zupo commented Jan 11, 2021

I'm one of those Nix beginners that was just bitten hard by preferLocalBuild overriding --max-jobs 0. My colleague and I started the day with a simple task: add https://nixbuild.net/ to our 500+ tiny VPSes hosting 70k websites so that when we run nixos-rebuild switch, their memory, CPU and disk don't get completely overused.

So we followed https://docs.nixbuild.net/getting-started/ and nix-building the example in that docs worked fined. But for the life of me, we couldn't get nixos-rebuild switch to use remote builders. We tried setting --max-jobs 0 but the thing would keep on building (locally).

After 6 hours of trying to figure out why --max-jobs 0 is not applied, I gave up, and because I'm born under a lucky star and have @domenkozar on speed dial, I called him up and he cleared up what's going on. I can imagine that if I couldn't randomly call an experienced NixOS person, I'd probably give it another day, and then after wasting 30+ engineering hours, give up.

@NorfairKing
Copy link

I've been burnt by this in the past as well.
It would be nice if there was a clear way to say "build all of this remotely".

@domenkozar
Copy link
Member

I wonder however what does prefer imply in preferLocalBuild? I would expect that in case of --max-jobs=0 that would mean it no longer prefers the local build. In what scenarios does it prefer the local build even?

@rickynils
Copy link
Member

I've implemented the simple resolution of this issue in #4443. It doesn't introduce the system features suggested by @grahamc, but instead makes max-jobs=0 not allow remote building of derivations with preferLocalBuild = true.

edolstra pushed a commit that referenced this issue Jan 13, 2021
This resolves #3810 by changing the behavior of `max-jobs = 0`, so
that specifying the option also avoids local building of derivations
with the attribute `preferLocalBuild = true`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants