Improve UX with CA derivations #875

thufschmitt · 2021-02-23T09:15:38Z

There should be a couple of UI artifacts (because the output paths were showing up in a couple places but we don't know them anymore), but everything seems to be working fine otherwise (I managed to build a couple generations of the nixpkgs-minimal jobset with this).

~~Depends on #874 (could probably be made to work without it, but I built it on top of it and the effectively touch the same code)~~

~~Also depends on NixOS/nix#4477~~
Depends on #1349

nixos-discourse · 2021-03-02T14:29:33Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update-8/11758/1

tomberek

It works!

nixos-discourse · 2021-04-21T11:40:29Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update-10/12587/1

grahamc · 2021-04-28T12:20:02Z

t/content-addressed.t

+ok(evalSucceeds($jobset),                  "Evaluating jobs/content-addressed.nix should exit with return code 0");
+ok(nrQueuedBuildsForJobset($jobset) == 3 , "Evaluating jobs/content-addressed.nix should result in 3 builds");
+
+for my $build (queuedBuildsForJobset($jobset)) {
+    ok(runBuild($build), "Build '".$build->job."' from jobs/content-addressed.nix should exit with code 0");
+    my $newbuild = $db->resultset('Builds')->find($build->id);
+    my $expected = $build->job eq "fails" ? 1 : $build->job =~ /with_failed/ ? 6 : 0;
+    ok($newbuild->finished == 1 && $newbuild->buildstatus == $expected, "Build '".$build->job."' from jobs/content-addressed.nix should have buildstatus $expected");
+}


Check out the other test files: there are other helpers like is which, when the test fails, prints the actual/expected values. Also, please don't check two facts at once (ie: don't use && in a check.)

Oh indeed. This test was actually a copy/paste of the “basic” test, but it didn't follow the overhaul of the test infrastructure.
Fixed in a12704f

grahamc · 2021-04-28T12:21:13Z

t/jobs/content-addressed.nix

+  mkDerivation = args: cfg.mkDerivation ({
+    __contentAddressed = true;
+    outputHashMode = "recursive";
+    outputHashAlgo = "sha256";
+  } // args);


Feel free to extend config.nix.in (and regenerate with make) with more functions, like makeContetAddressedDerivation :)

Right. Done in 8ef36cc

grahamc · 2021-04-28T12:23:42Z

src/lib/Hydra/Controller/Build.pm

+    # XXX: If ca-derivations is enabled then this will always return false
+    # because `$_->path` will be empty


Is this always true, or is it true if it is set AND the derivation is ca-addressed? This feels like a fairly spooky change to the data model of Hydra.

Mh… I see what you mean. It's only true if the derivation is ca. I've updated the comment in ca86783 to make it clearer.

This feels like a fairly spooky change to the data model of Hydra.

It is indeed 😃 . But the path must be nullable, because of the very nature of CA derivations

grahamc · 2021-04-28T12:25:09Z

I should have done this as a review, sorry.

In the C++ I'm seeing a lot of in-place complication. Would it be possible to clean up the campsite a bit, and make some functions and break up the behavior a bit? I'm increasingly uncertain about understanding those pieces of the code.

grahamc · 2021-04-28T12:36:35Z

I also wonder if there would be anything interesting in tests for:

requesting the Builds page. It is fairly easy to test controllers using https://github.com/NixOS/hydra/blob/master/t/Controller/Jobset/channel.t#L15-L16 :` https://github.com/NixOS/hydra/blob/master/t/Controller/Jobset/channel.t#L36
what happens when trying to build a content-addressed derivation but CA is not enabled?
what happens when building a CA derivation but its already been built and pushed to a binary cache? Here is a test that checks similar behavior: https://github.com/NixOS/hydra/blob/master/t/queue-runner/notifications.t
how does Hydra handle it if Hydra is ca-enabled but its builders are not?

thufschmitt · 2021-04-28T13:00:22Z

Thanks for the review @grahamc

In the C++ I'm seeing a lot of in-place complication. Would it be possible to clean up the campsite a bit, and make some functions and break up the behavior a bit? I'm increasingly uncertain about understanding those pieces of the code.

A large part of the complication is annoyingly due to the fact that we need to support both the nix+ca-derivations and nix-without-ca-derivations cases, which means two slightly different code paths. But it should be indeed possible to factor at least part of it out.

I also wonder if there would be anything interesting in tests for:

requesting the Builds page. It is fairly easy to test controllers using https://github.com/NixOS/hydra/blob/master/t/Controller/Jobset/channel.t#L15-L16 :` https://github.com/NixOS/hydra/blob/master/t/Controller/Jobset/channel.t#L36
what happens when trying to build a content-addressed derivation but CA is not enabled?
what happens when building a CA derivation but its already been built and pushed to a binary cache? Here is a test that checks similar behavior: https://github.com/NixOS/hydra/blob/master/t/queue-runner/notifications.t
how does Hydra handle it if Hydra is ca-enabled but its builders are not?

I’d have to think it a bit more, but I think there's a number of things worth testing here indeed.

thufschmitt · 2021-04-28T13:02:29Z

src/hydra-queue-runner/build-remote.cc

-                    basicDrv.inputSrcs.insert(*outPath);
+        BasicDerivation basicDrv;
+        auto outputHashes = staticOutputHashes(*localStore, *step->drv);
+        if (auto maybeBasicDrv = step->drv->tryResolve(*localStore))


Note to self: Reading through the code again, I'm actually wondering whether this isn't messing-up the build of non-ca derivations (because tryResolve changes the output paths, so applying it unconditionally might do some funny things). I'd expect this would cause a test failure (or a hydra crash), but it's worth double-checking first

thufschmitt · 2021-04-28T19:43:50Z

I've added a bunch of tests (and fixed a bunch of related issues by the way).

I didn't test

3. what happens when building a CA derivation but its already been built and pushed to a binary cache? Here is a test that checks similar behavior: https://github.com/NixOS/hydra/blob/master/t/queue-runner/notifications.t

4. how does Hydra handle it if Hydra is ca-enabled but its builders are not?

because

substituting ca derivations isn't supported yet in hydra (and doing so will probably require a bit of work)
A “mixed” network of machines (some ca-enabled, some not) isn't supported by Nix atm (I've just opened Allow mixing ca-enabled and non-ca-enabled builders nix#4755 to track it)

thufschmitt · 2021-04-29T08:44:28Z

The test hydra instance just showed some weird failures with fixed-output derivations (https://hydra.ngi0.nixos.org/build/171 − the cbindgen vendor derivation should build just fine but is marked as failed without anything appearing in the log). This needs a bit of investigation − Might be linked to #875 (comment) because maybe that bit of code also rewrites the output hashes of FO derivations.

thufschmitt · 2021-04-29T10:28:20Z

The test hydra instance just showed some weird failures with fixed-output derivations (https://hydra.ngi0.nixos.org/build/171 − the cbindgen vendor derivation should build just fine but is marked as failed without anything appearing in the log). This needs a bit of investigation − Might be linked to #875 (comment) because maybe that bit of code also rewrites the output hashes of FO derivations.

OK, looks like these are “legitimate” has mismatches. So the only issue is that the error isn’t properly reported − which I think is a Nix issue

grahamc · 2021-04-29T10:41:54Z

It would be really great if whatever the underlying issue is has an accompanying test wherever the fix belongs :) these sort of mystery problems are exhausting 😔

On April 29, 2021, GitHub ***@***.***> wrote: > The test hydra instance just showed some weird failures with fixed- > output derivations (<https://hydra.ngi0.nixos.org/build/171> − the > cbindgen vendor derivation should build just fine but is marked as > failed without anything appearing in the log). This needs a bit of > investigation − Might be linked to #875 (comment) > <#875 (comment)> > because maybe that bit of code also rewrites the output hashes of FO > derivations. > OK, looks like these are “legitimate” has mismatches. So the only issue is that the error isn’t properly reported − which I think is a Nix issue

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#875 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe- auth/AAASXLE7UTSXLSFPVILDEATTLEYFRANCNFSM4YCDMVTQ>.

nixos-discourse · 2021-05-04T14:19:30Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update-11/12886/1

tomberek · 2021-05-11T06:07:00Z

Possible issue with remote builders and hydra. (note: there should be no CA builds occurring)
Using hydra:

 src = pkgs.fetchFromGitHub {
    owner = "regnat";
    repo = "hydra";
    rev = "2090d3a0899db4b63d3fbf93545be0020385a165";
    sha256 = "sha256-GrYR/xX/kWxxB0VoHKj0c3Pw7A6RLPexDkVNNVa7kn4=";
};

Getting errors like:

possibly transient failure building ‘/nix/store/XYZ.drv’ on ‘[email protected]’: error: dependency '/nix/store/3fs5wqlj9ipv7wjx4av81skmpcdjjh3d-jq-1.6-dev' of '/nix/store/XYZ.drv' does not exist, and substitution is disabled

Reverting to package = hydra-unstable; restores behavior as expected with regards to remote building. All builds have __contentAddressed = false. Any thoughts, leads? I attempted setting use-substitutes, useSubstitutes, and builders-use-substitutes to no effect. Only solution was to revert.

thufschmitt · 2021-05-15T06:38:27Z

Getting errors like:

possibly transient failure building ‘/nix/store/XYZ.drv’ on ‘[email protected]’: error: dependency '/nix/store/3fs5wqlj9ipv7wjx4av81skmpcdjjh3d-jq-1.6-dev' of '/nix/store/XYZ.drv' does not exist, and substitution is disabled

We’ve also started getting the same on https://hydra.ngi0.nixos.org/build/202 .

That might (quite ironically) be related to d7a1b6b

Mindavi · 2021-05-29T12:59:09Z

Running this branch now with ca-derivation builds, but getting errors in the UI like this: Caught exception in Hydra::Controller::Build->build "�[31;1merror:�[0m path '�[33;1m/1vbp723njr8wq6m404hzkfmqck0vk73gcgr6pjc78il65mh6p9mn�[0m' is not in the Nix store at /nix/store/62znwr88yc5mi5v0qxmzp9rd84km68q6-hydra-2021-05-03/libexec/hydra/lib/Hydra/Controller/Build.pm line 73.". Seems to be building anyway though, which wasn't the case with hydra-unstable.

Not sure if that's to be addressed in a future patch or if I hit some edge case.

Edit: I made a new jobset and now it seems solved (for that new jobset).

thufschmitt · 2021-05-31T13:46:15Z

@tomberek can you try 854dcec ? I think it should fix your issue.

@Mindavi there were some known issues with earlier commits in this branch where the database wasn’t properly filled with all the required informations, causing this kind of issue (and which persisted after an upgrade because the incomplete data was still there). Any chance that’s the case for you?

Mindavi · 2021-05-31T16:21:05Z

I guess I broke it with trying to build with ca-derivations first and only then finding out hydra doesn't support it and applying thid patchset.

It seems to be ok now though, with a fresh jobset, apart from the (already expected) ui artifacts (notably, empty package names).

Mindavi · 2021-08-06T07:07:46Z

This has now been running pretty smoothly for a while for me. However, there's one thing that's now starting to happen:

    Aborted: error: Trying to register a realisation of 'sha256:5a7f41bac1b8abb186eb4f0ec1a76ee90809fabbdd620bde7ea0ddb66cb96e6a!out', but we already have another one locally.
      Local: /nix/store/2489p9qk8i8wi627maxix9w15li2y890-systemd-247.6
      Remote: /nix/store/dvbv3d914ii22vnr277bslzq8lnwz68x-systemd-247.6

I already have systemd built locally, but it appears to be trying to substitute it anyways? And the remote appears to have another hash as well. I'm not sure what's happening here, but it shouldn't be trying to substitute it, I guess.

This has happened for some different packages over the last few days, e.g. also for glibc.

I have the binary cache for the ngi0 machine that's building the ca derivations enabled, I guess those might also have something to do with this?

Versions:

nix (Nix) 2.4pre20210707_02dd6bb
hydra: d20639a (latest commit on this PR as of today)

In particuilar, make all - `tryResolve` - `queryerivationOutputMap` - `queryPartialDerivationOutputMap` look in dest store. The local store will just be used for the derivations themselves. Progress towards not keeping outputs and realisations in the local store at all.

No outputs, no resolutions

They do now! This reverts commit a6b6c5a.

Mindavi · 2023-12-11T18:50:56Z

t/jobs/dir-with-file-builder.sh

+#! /bin/sh
+
+# Workaround for https://github.com/NixOS/nix/pull/6051
+echo "some output"


If I read upstream correctly, this workaround shouldn't be needed anymore.

SuperSandro2000 · 2023-12-30T02:14:34Z

Could it be that we are missing the remote builder check implemented here https://github.com/NixOS/nix/pull/4905/files#diff-e89a2224d1d4c39bf5a2d448b7b2fac1e3e75c7e08e414ee448846266c8bfeab ?

Because I am experiencing that hydra tries to use remote builders for ca-derivations that do not have that feature enabled in the daemon or the machines file.

Edit:

Hydra reported the following error error: experimental Nix feature 'ca-derivations' is disabled; use '--extra-experimental-features ca-derivations' to override

The builder it tried to run the build on is the correct system but doesn't support that feature.

When building manually it is confirmed, that the derivation requires ca-derivation. Why did hydra schedule that on that build?

 ➜ command nix-build /nix/store/h29mvdz2fk3zbsdpiy8gsssa80l6m5zy-unit-systemd-udevd.service.drv -v
this derivation will be built:
  /nix/store/h29mvdz2fk3zbsdpiy8gsssa80l6m5zy-unit-systemd-udevd.service.drv
resolved derivation: '/nix/store/h29mvdz2fk3zbsdpiy8gsssa80l6m5zy-unit-systemd-udevd.service.drv' -> '/nix/store/fa6f9l3fjjdgknx4dhrlf4xzk2ipbknc-unit-systemd-udevd.service.drv'...
warning: Ignoring setting 'impure-env' because experimental feature 'configurable-impure-env' is not enabled
Failed to find a machine for remote build!
derivation: fa6f9l3fjjdgknx4dhrlf4xzk2ipbknc-unit-systemd-udevd.service.drv
required (system, features): (aarch64-linux, [ca-derivations])

Edit2:

I really don't have much of clue but I think the error is here https://github.com/thufschmitt/hydra/blob/master/src/hydra-queue-runner/state.hh#L281C18-L281C35

My clues are that either the check code is wrong or the steps don't have the required step set correct.

Other machines with the wrong arch log this line https://github.com/thufschmitt/hydra/blob/master/src/hydra-queue-runner/dispatcher.cc#L250C28-L250C35

Edit3: I think I found the difference in the hydra code https://github.com/thufschmitt/hydra/blob/nix-ca/src/hydra-queue-runner/queue-monitor.cc#L464-L473 and https://github.com/NixOS/nix/blob/master/src/libstore/parsed-derivations.cc#L96-L97

Ericson you are so right, that this code needs to be deduplicated which would fix a ton of bugs like this one.

SuperSandro2000 · 2024-01-01T18:13:07Z

I think SuperSandro2000@27b1935 solved that issue but I am not yet 100% sure.

Edit: I am not sure actually.

NixOS/hydra#875 is the next thing to test. Flake lock file updates: • Updated input 'hydra': 'github:NixOS/hydra/d45e14fd4381d1476a606c20ef3573215067472c' (2024-01-24) → 'github:thufschmitt/hydra/002a77a954259ddb49fd262607ba44fb2c3b34a9' (2024-01-24)

Per NixOS/hydra#838 we want to test that NixOS/hydra#875 works both with and without it enabled, so we can deploy without it first.

Ericson2314 · 2024-01-25T23:45:54Z

(For anyone else reading this, the scheduling issues @SuperSandro2000 brings up are now fixed on master.)

Ericson2314 · 2024-01-26T16:19:55Z

Renamed because since I factored out #1349, what is left just in this PR is just UI changes (and the extra column that those UI changes leverage).

SuperSandro2000 · 2024-03-13T12:56:36Z

nix_2_21 is unhappy with this PR/hydra in generell:

hydra> g++ -DHAVE_CONFIG_H -I. -I../..    -std=c++2a -I/nix/store/90ns4amhyqqgb9lnx3kvqsl6vjigpzww-nix-2.21.0-dev/include/nix -I/nix/store/h5g13jk10ziz3xfnpdvgcsc0brzxmkkw-boehm-gc-8.2.4-dev/include -Wall -I ../libhydra -Wno-deprecated-declarations -g -O2 -include nix/config.h -c -o hydra_evaluator-hydra-evaluator.o `test -f 'hydra-evaluator.cc' || echo './'`hydra-evaluator.cc
hydra> hydra-evaluator.cc: In member function 'std::string JobsetId::display() const':
hydra> hydra-evaluator.cc:41:20: error: 'format' was not declared in this scope
hydra>    41 |         return str(format("%1%:%2% (jobset#%3%)") % project % jobset % id);
hydra>       |                    ^~~~~~
hydra> hydra-evaluator.cc:41:20: note: suggested alternatives:
hydra> In file included from /nix/store/7vc88ixc6yca2hwgszjk51dh23j3g2nr-gcc-13.2.0/include/c++/13.2.0/bits/chrono_io.h:39,
hydra>                  from /nix/store/7vc88ixc6yca2hwgszjk51dh23j3g2nr-gcc-13.2.0/include/c++/13.2.0/chrono:3330,
hydra>                  from /nix/store/n2v9lm5vvam9v847qvpdrjik0rk1qz02-libpqxx-7.7.5/include/pqxx/time.hxx:12,
hydra>                  from /nix/store/n2v9lm5vvam9v847qvpdrjik0rk1qz02-libpqxx-7.7.5/include/pqxx/pqxx:26,
hydra>                  from ../libhydra/db.hh:3,
hydra>                  from hydra-evaluator.cc:1:
hydra> /nix/store/7vc88ixc6yca2hwgszjk51dh23j3g2nr-gcc-13.2.0/include/c++/13.2.0/format:3764:5: note:   'std::format'
hydra>  3764 |     format(const locale& __loc, wformat_string<_Args...> __fmt,
hydra>       |     ^~~~~~
hydra> In file included from /nix/store/n2v9lm5vvam9v847qvpdrjik0rk1qz02-libpqxx-7.7.5/include/pqxx/result.hxx:26,
hydra>                  from /nix/store/n2v9lm5vvam9v847qvpdrjik0rk1qz02-libpqxx-7.7.5/include/pqxx/binarystring.hxx:22,
hydra>                  from /nix/store/n2v9lm5vvam9v847qvpdrjik0rk1qz02-libpqxx-7.7.5/include/pqxx/pqxx:5:
hydra> /nix/store/n2v9lm5vvam9v847qvpdrjik0rk1qz02-libpqxx-7.7.5/include/pqxx/types.hxx:80:12: note:   'pqxx::format'
hydra>    80 | enum class format : int
hydra>       |            ^~~~~~
hydra> In file included from include/boost/format.hpp:34,
hydra>                  from /nix/store/90ns4amhyqqgb9lnx3kvqsl6vjigpzww-nix-2.21.0-dev/include/nix/fmt.hh:4,
hydra>                  from /nix/store/90ns4amhyqqgb9lnx3kvqsl6vjigpzww-nix-2.21.0-dev/include/nix/error.hh:21,
hydra>                  from /nix/store/90ns4amhyqqgb9lnx3kvqsl6vjigpzww-nix-2.21.0-dev/include/nix/util.hh:5,
hydra>                  from ../libhydra/db.hh:6:
hydra> include/boost/format/format_fwd.hpp:27:37: note:   'boost::format'
hydra> hydra-evaluator.cc:41:16: error: 'str' was not declared in this scope
hydra>    41 |         return str(format("%1%:%2% (jobset#%3%)") % project % jobset % id);
hydra>       |                ^~~
hydra> hydra-evaluator.cc:41:16: note: suggested alternatives:
hydra> In file included from include/boost/format.hpp:53:
hydra> include/boost/format/free_funcs.hpp:22:38: note:   'boost::str'
hydra> include/boost/format/free_funcs.hpp:22:38: note:   'boost::str'
hydra> make[3]: *** [Makefile:443: hydra_evaluator-hydra-evaluator.o] Error 1
hydra> make[3]: Leaving directory '/build/source/src/hydra-evaluator'
hydra> make[2]: *** [Makefile:363: all-recursive] Error 1
hydra> make[2]: Leaving directory '/build/source/src'
hydra> make[1]: *** [Makefile:423: all-recursive] Error 1
hydra> make[1]: Leaving directory '/build/source'
hydra> make: *** [Makefile:351: all] Error 2

Mindavi · 2024-04-04T19:56:19Z

I think currently (at least) a database migration is missing here.

NixOS/hydra#875 is the next thing to test. Flake lock file updates: • Updated input 'hydra': 'github:NixOS/hydra/d45e14fd4381d1476a606c20ef3573215067472c' (2024-01-24) → 'github:thufschmitt/hydra/002a77a954259ddb49fd262607ba44fb2c3b34a9' (2024-01-24)

Per NixOS/hydra#838 we want to test that NixOS/hydra#875 works both with and without it enabled, so we can deploy without it first.

SuperSandro2000 · 2024-05-03T16:16:06Z

(For anyone else reading this, the scheduling issues @SuperSandro2000 brings up are now fixed on master.)

Not sure if I ever confirmed this, but lately I have noticed that sometimes builds couldn't be scheduled because the feature was missing, so from my perspective things seem to work as expected 🎉

gador · 2024-11-02T11:57:09Z

When building packages and system configurations with this PR, it all seems to work with and without content-addressed derivations.
Only when I tried to build one of my system configurations, I get:

Nov 02 12:33:34 hel hydra-queue-runner[255887]: performing step ‘/nix/store/dxzgs08hhmcvm48pflwv1lfmimcls7gx-tmpfiles.d.drv’ 1 times on ‘localhost’ (needed by build 198 and 0 others)
Nov 02 12:33:34 hel hydra-queue-runner[255887]: warning: output out of input /nix/store/qp9sqj34pl45l65hlslz1hidjh8vb9l7-nixos-tmpfiles.d.drv missing, aborting the resolving

Note: The derivation dxzgs08hhmcvm48pflwv1lfmimcls7gx-tmpfiles.d.drv is present on the machine, as well as the build output. When I look at it, I see:

nix build --print-out-paths "/nix/store/dxzgs08hhmcvm48pflwv1lfmimcls7gx-tmpfiles.d.drv^*"
/nix/store/f3a9xyvjh2g98xdnfl1z01vp66r2hwrr-tmpfiles.d
ls -la /nix/store/f3a9xyvjh2g98xdnfl1z01vp66r2hwrr-tmpfiles.d
total 25064
dr-xr-xr-x     2 root root      16 Jan  1  1970 .
drwxrwxr-t 14919 root nixbld 92640 Nov  2 12:49 ..
lrwxrwxrwx     2 root root      89 Jan  1  1970 00-nixos.conf -> /nix/store/55bhwpm4zk1p25m73lbsv3bchxkc1czm-nixos-tmpfiles.d/lib/tmpfiles.d/00-nixos.conf
lrwxrwxrwx     5 root root      86 Jan  1  1970 home.conf -> /nix/store/zqmkf68sb4j70v89j1hhsa44b0fb0sdv-systemd-256.6/example/tmpfiles.d/home.conf
lrwxrwxrwx     5 root root      95 Jan  1  1970 journal-nocow.conf -> /nix/store/zqmkf68sb4j70v89j1hhsa44b0fb0sdv-systemd-256.6/example/tmpfiles.d/journal-nocow.conf
lrwxrwxrwx     3 root root      81 Jan  1  1970 lvm2.conf -> /nix/store/w0w4mcvrma1b11kv2fxi6x02dcnqxkyz-lvm2-2.03.26/lib/tmpfiles.d/lvm2.conf
lrwxrwxrwx     4 root root      85 Jan  1  1970 nix-daemon.conf -> /nix/store/s94nymfmjg1bknwfbbmz2r4k8qr7g964-nix-2.24.9/lib/tmpfiles.d/nix-daemon.conf
lrwxrwxrwx     5 root root      91 Jan  1  1970 portables.conf -> /nix/store/zqmkf68sb4j70v89j1hhsa44b0fb0sdv-systemd-256.6/example/tmpfiles.d/portables.conf
lrwxrwxrwx     5 root root     106 Jan  1  1970 static-nodes-permissions.conf -> /nix/store/zqmkf68sb4j70v89j1hhsa44b0fb0sdv-systemd-256.6/example/tmpfiles.d/static-nodes-permissions.conf
lrwxrwxrwx     5 root root      89 Jan  1  1970 systemd.conf -> /nix/store/zqmkf68sb4j70v89j1hhsa44b0fb0sdv-systemd-256.6/example/tmpfiles.d/systemd.conf
lrwxrwxrwx     5 root root      97 Jan  1  1970 systemd-nologin.conf -> /nix/store/zqmkf68sb4j70v89j1hhsa44b0fb0sdv-systemd-256.6/example/tmpfiles.d/systemd-nologin.conf
lrwxrwxrwx     5 root root      96 Jan  1  1970 systemd-nspawn.conf -> /nix/store/zqmkf68sb4j70v89j1hhsa44b0fb0sdv-systemd-256.6/example/tmpfiles.d/systemd-nspawn.conf
lrwxrwxrwx     5 root root      93 Jan  1  1970 systemd-tmp.conf -> /nix/store/zqmkf68sb4j70v89j1hhsa44b0fb0sdv-systemd-256.6/example/tmpfiles.d/systemd-tmp.conf
lrwxrwxrwx     5 root root      85 Jan  1  1970 tmp.conf -> /nix/store/zqmkf68sb4j70v89j1hhsa44b0fb0sdv-systemd-256.6/example/tmpfiles.d/tmp.conf
lrwxrwxrwx     5 root root      85 Jan  1  1970 var.conf -> /nix/store/zqmkf68sb4j70v89j1hhsa44b0fb0sdv-systemd-256.6/example/tmpfiles.d/var.conf
lrwxrwxrwx     5 root root      85 Jan  1  1970 x11.conf -> /nix/store/zqmkf68sb4j70v89j1hhsa44b0fb0sdv-systemd-256.6/example/tmpfiles.d/x11.conf

So the derivation, as well as the output, is present. Restarting the build (or wait for the queue-runner to do it again), results in the exact same error message. So does repeating or restarting the build. It it impossible to build the system config right now with hydra.

This happens on the localhost builder just as well as on remote builders.
This only happens with content-addressed derivations and only when I build the system configuration. All other packages ( I maintain) build fine as CA's.

The only related issue I found was NixOS/nix#6700 which did happen to me i the past, but got fixed later on. I also don't get the error: unexpected end-of-file message, but just the above.

The nix-daemon.service doesn't report any errors either, which is why I think it has something to do with how hydra build the CA derivations (ergo this and the related PR's)

Also, I already did build the system configuration with hydras own reproduce locally command just fine. Once on a remote builder, once on localhost. So this seems hydra-specific.

thufschmitt marked this pull request as draft February 23, 2021 09:15

thufschmitt force-pushed the nix-ca branch 3 times, most recently from e0b9c88 to 74eb4e4 Compare April 9, 2021 12:50

thufschmitt marked this pull request as ready for review April 9, 2021 12:51

thufschmitt force-pushed the nix-ca branch 2 times, most recently from 4cb5eee to ff0a9b5 Compare April 15, 2021 05:41

tomberek approved these changes Apr 20, 2021

View reviewed changes

thufschmitt force-pushed the nix-ca branch from 5da543c to 01019b6 Compare April 22, 2021 11:30

thufschmitt mentioned this pull request Apr 28, 2021

The testsuite never checks what happens when destStore != localStore #938

Closed

grahamc reviewed Apr 28, 2021

View reviewed changes

thufschmitt commented Apr 28, 2021

View reviewed changes

Ericson2314 added 6 commits December 11, 2023 12:39

Don't write things to the local store that don't belong

17c8d51

No outputs, no resolutions

Beef of content addressing test

7d7a2a3

Merge branch 'ca-derivations-prep' into nix-ca

ba70b4e

Merge branch 'ca-derivations-prep' into nix-ca

25a5e4e

Revert "Revert query -- those columns don't exist yet!"

bfe3567

They do now! This reverts commit a6b6c5a.

Mindavi reviewed Dec 11, 2023

View reviewed changes

Merge branch 'ca-derivations-prep' into nix-ca

7f3e2c8

Ericson2314 added 2 commits January 24, 2024 18:15

Merge branch 'master' into nix-ca

002a77a

Merge branch 'master' into nix-ca

0b632ec

Ericson2314 added a commit to ngi-nix/ngi0-infra that referenced this pull request Jan 25, 2024

Disable ca-derivations experimental feature

40fdf9e

Per NixOS/hydra#838 we want to test that NixOS/hydra#875 works both with and without it enabled, so we can deploy without it first.

Ericson2314 added 2 commits January 25, 2024 19:46

Merge branch 'master' into nix-ca

ee6c465

Merge branch 'ca-no-new-col' into nix-ca

c87a066

Ericson2314 changed the title ~~Allow building content-addressed derivations with hydra~~ Improve UX with CA derivations Jan 26, 2024

Merge branch 'ca-no-new-col' into nix-ca

91c641d

Ericson2314 added 2 commits January 26, 2024 11:28

Fix mistake in merge

2b1d277

Merge branch 'master' into nix-ca

d7e7bee

lorenzleutgeb pushed a commit to ngi-nix/ngipkgs that referenced this pull request Apr 26, 2024

Disable ca-derivations experimental feature

b110ae6

Per NixOS/hydra#838 we want to test that NixOS/hydra#875 works both with and without it enabled, so we can deploy without it first.

Merge remote-tracking branch 'upstream/master' into nix-ca

b207f20

benaryorg mentioned this pull request Jul 17, 2024

transient failure building due to unresolved derivations when building ca-derivations with an s3 store #1393

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve UX with CA derivations #875

Improve UX with CA derivations #875

thufschmitt commented Feb 23, 2021 •

edited by Ericson2314

Loading

nixos-discourse commented Mar 2, 2021

tomberek left a comment

nixos-discourse commented Apr 21, 2021

grahamc Apr 28, 2021

thufschmitt Apr 28, 2021

grahamc Apr 28, 2021

thufschmitt Apr 28, 2021

grahamc Apr 28, 2021

thufschmitt Apr 28, 2021

grahamc commented Apr 28, 2021

grahamc commented Apr 28, 2021

thufschmitt commented Apr 28, 2021

thufschmitt Apr 28, 2021

thufschmitt commented Apr 28, 2021

thufschmitt commented Apr 29, 2021

thufschmitt commented Apr 29, 2021

grahamc commented Apr 29, 2021 via email

nixos-discourse commented May 4, 2021

tomberek commented May 11, 2021

thufschmitt commented May 15, 2021

Mindavi commented May 29, 2021 •

edited

Loading

thufschmitt commented May 31, 2021

Mindavi commented May 31, 2021

Mindavi commented Aug 6, 2021

Mindavi Dec 11, 2023

SuperSandro2000 commented Dec 30, 2023 •

edited

Loading

SuperSandro2000 commented Jan 1, 2024 •

edited

Loading

Ericson2314 commented Jan 25, 2024

Ericson2314 commented Jan 26, 2024

SuperSandro2000 commented Mar 13, 2024

Mindavi commented Apr 4, 2024

SuperSandro2000 commented May 3, 2024

gador commented Nov 2, 2024

		# XXX: If ca-derivations is enabled then this will always return false
		# because `$_->path` will be empty

Improve UX with CA derivations #875

Are you sure you want to change the base?

Improve UX with CA derivations #875

Conversation

thufschmitt commented Feb 23, 2021 • edited by Ericson2314 Loading

nixos-discourse commented Mar 2, 2021

tomberek left a comment

Choose a reason for hiding this comment

nixos-discourse commented Apr 21, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

grahamc commented Apr 28, 2021

grahamc commented Apr 28, 2021

thufschmitt commented Apr 28, 2021

Choose a reason for hiding this comment

thufschmitt commented Apr 28, 2021

thufschmitt commented Apr 29, 2021

thufschmitt commented Apr 29, 2021

grahamc commented Apr 29, 2021 via email

nixos-discourse commented May 4, 2021

tomberek commented May 11, 2021

thufschmitt commented May 15, 2021

Mindavi commented May 29, 2021 • edited Loading

thufschmitt commented May 31, 2021

Mindavi commented May 31, 2021

Mindavi commented Aug 6, 2021

Choose a reason for hiding this comment

SuperSandro2000 commented Dec 30, 2023 • edited Loading

SuperSandro2000 commented Jan 1, 2024 • edited Loading

Ericson2314 commented Jan 25, 2024

Ericson2314 commented Jan 26, 2024

SuperSandro2000 commented Mar 13, 2024

Mindavi commented Apr 4, 2024

SuperSandro2000 commented May 3, 2024

gador commented Nov 2, 2024

thufschmitt commented Feb 23, 2021 •

edited by Ericson2314

Loading

Mindavi commented May 29, 2021 •

edited

Loading

SuperSandro2000 commented Dec 30, 2023 •

edited

Loading

SuperSandro2000 commented Jan 1, 2024 •

edited

Loading