hadoop: add aarch64 support #158613

ConnorBaker · 2022-02-08T13:27:08Z

This commit also changes the names of the tests for Hadoop so they use dashes instead of dots,
and makes the default hadoop test what would have been hadoop-all after the rename.

This change should mean that we're able to run

nix build github:nixos/nixpkgs/master#nixosTests.hadoop

which I was unable to do prior to this change.

I've built this change on aarch64-darwin, aarch64-linux, and x86_64-linux. I've run the NixOS tests for Hadoop on aarch64-linux, and x86_64-linux. On aarch64-darwin I just ran hadoop version and hadoop checknative.

I made a flake to spin up an AWS EC2 instance on a bare-metal instance (required because NixOS tests need KVM), create a ramdisk (to avoid provisioning a larger amount of disk space), and build and test the change there: https://github.com/ConnorBaker/nix-hadoop-aarch64-test.

I've seen some posts discussing getting the native libraries working for macOS, but I wasn't sure how to do that. I'm not familiar with patchelf or install_name_tool or how linking works, so any help would be appreciated!

Motivation for this change

Spark requires Hadoop, so adding Hadoop's aarch64 distribution allows us to use Spark on that platform.

Things done

jonringer

diff looks good to me.

How expensive are the tests? the x86_64-linux machine has pretty slow cores

ConnorBaker · 2022-02-08T20:52:53Z

I used an m6g.metal instance (about $2.50 an hour) for the aarch64-linux tests and a c5n.metal instance (about $4.00 an hour) for the x86_64-linux tests.

I'm running the tests again on m6g.metal and c5n.metal to get you the numbers on how long it'll take -- I don't remember exactly, but I thought it was less than 20 or 30 minutes.

jonringer · 2022-02-08T21:44:08Z

cc @grahamc about resource usage

jonringer · 2022-02-08T21:45:22Z

pkgs/applications/networking/cluster/hadoop/default.nix

+    let
+      platformUrlSuffix =
+        if stdenv.isx86_64 && (stdenv.isLinux || stdenv.isDarwin)
+        then ""
+        else
+          if stdenv.isAarch64 && (stdenv.isLinux || stdenv.isDarwin)
+          then "-aarch64"
+          else throw "Hadoop does not currently support ${stdenv.system}.";
+    in


generally we don't bother with doing platform radix, as that's captured in the .drv

[13:26:04] jon@nixos ~/projects/nixpkgs (staging-next) $ nix-instantiate -A hadoop warning: you did not specify '--add-root'; the result might be removed by the garbage collector /nix/store/smsf7vdmscgj1fqdvc7b14fw186pn7kn-hadoop-3.3.1.drv [13:47:28] jon@nixos ~/projects/nixpkgs (staging-next) $ nix show-derivation /nix/store/smsf7vdmscgj1fqdvc7b14fw186pn7kn-hadoop-3.3.1.drv | grep system "system": "x86_64-linux", "system": "x86_64-linux",

Does a24835e handle it better?

jonringer · 2022-02-08T21:46:40Z

pkgs/applications/networking/cluster/hadoop/default.nix

+        x86_64-linux = "1b3v16ihysqaxw8za1r5jlnphy8dwhivdx2d0z64309w57ihlxxd";
+        x86_64-darwin = "1b3v16ihysqaxw8za1r5jlnphy8dwhivdx2d0z64309w57ihlxxd";
+        aarch64-linux = "sha256-v1Om2pk0wsgKBghRD2wgTSHJoKd3jkm1wPKAeDcKlgI=";
+        aarch64-darwin = "sha256-v1Om2pk0wsgKBghRD2wgTSHJoKd3jkm1wPKAeDcKlgI=";


there should be some instruction on how to generate the shas

E.g. nix-prefetch-url ...

Whoops! It should be fixed in 0aaa8f7.

ConnorBaker · 2022-02-09T00:09:24Z

Uh oh, I'm getting a failed build for nixosTests.hadoop, though not until about 19m in. Both hadoop-yarn and hadoop-hdfs seem to be fine. I'll test against current master to see if I can reproduce it there.

Info follows below:

c5n.metal running, NixOS 21.11 on the nixos-unstable channel, building in a ramdisk:

$ lscpu
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         46 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  72
  On-line CPU(s) list:   0-71
Vendor ID:               GenuineIntel
  Model name:            Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
    CPU family:          6
    Model:               85
    Thread(s) per core:  2
    Core(s) per socket:  18
    Socket(s):           2
    Stepping:            4
    CPU max MHz:         3500.0000
    CPU min MHz:         1200.0000
    BogoMIPS:            6000.00
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_
                         tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpu
                         id_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq r
                         dseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req pku ospke
                          md_clear flush_l1d
Virtualization features: 
  Virtualization:        VT-x
Caches (sum of all):     
  L1d:                   1.1 MiB (36 instances)
  L1i:                   1.1 MiB (36 instances)
  L2:                    36 MiB (36 instances)
  L3:                    49.5 MiB (2 instances)
NUMA:                    
  NUMA node(s):          2
  NUMA node0 CPU(s):     0-17,36-53
  NUMA node1 CPU(s):     18-35,54-71
Vulnerabilities:         
  Itlb multihit:         KVM: Mitigation: VMX disabled
  L1tf:                  Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
  Mds:                   Mitigation; Clear CPU buffers; SMT vulnerable
  Meltdown:              Mitigation; PTI
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl and seccomp
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Full generic retpoline, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling
  Srbds:                 Not affected
  Tsx async abort:       Mitigation; Clear CPU buffers; SMT vulnerable

$ lsmem
RANGE                                  SIZE  STATE REMOVABLE BLOCK
0x0000000000000000-0x000000307fffffff  194G online       yes  0-96

Memory block size:         2G
Total online memory:     194G
Total offline memory:      0B

$ time nix build github:connorbaker/nixpkgs#nixosTests.hadoop-hdfs --out-link hadoop-hdfs

real	2m35.571s
user	2m38.870s
sys	1m39.416s

$ time nix build github:connorbaker/nixpkgs#nixosTests.hadoop-yarn --out-link hadoop-yarn

real	0m27.148s
user	0m39.548s
sys	0m20.513s

$ time nix build github:connorbaker/nixpkgs#nixosTests.hadoop --out-link hadoop
error: builder for '/nix/store/kxzchmmkwh64ain16k7jsqwgxdmpg256-vm-test-run-unnamed.drv' failed with exit code 1;
       last 10 log lines:
       > kill vlan (pid 16)
       > kill vlan (pid 15)
       > kill vlan (pid 14)
       > kill vlan (pid 13)
       > kill vlan (pid 12)
       > kill vlan (pid 11)
       > kill vlan (pid 10)
       > kill vlan (pid 9)
       > kill vlan (pid 8)
       > kill vlan (pid 7)
       For full logs, run 'nix log /nix/store/kxzchmmkwh64ain16k7jsqwgxdmpg256-vm-test-run-unnamed.drv'.

real	18m2.483s
user	20m10.371s
sys	7m27.104s

Logs for the failed tests are here: https://gist.github.com/ConnorBaker/ec83223ed31256d9ba637d40f0ee4296

I'm seeing a lot of

java.io.IOException: Cannot run program "/run/wrappers/yarn-nodemanager/bin/container-executor": error=2, No such file or directory�

repeated in the logs.

EDIT: Kind of unrelated, but is there a way to get nix log to strip ANSI color codes when outputting to a file?

ConnorBaker · 2022-02-09T01:44:36Z

On the same c5n.metal instance as above. I blew away the nix store in the ramdisk prior to running these. (Note: I had to get a copy of nixpkgs because I wasn't able to figure out how to use nix build with a flake when there are nested attribute sets).

It works, so I think I may have messed up by setting openssl = null; and conditionally updating the attribute set when the platform is Linux, but I don't know enough Nix to say that's what did it. I also made sure to do the same tests against the repo just prior to my changes, to make sure that the copy of master I was working off of didn't have the same issue before I started. ~~I'll do some further investigating later tonight!~~ I'll be looking at this over the weekend.

$ curl -L --silent https://github.com/NixOS/nixpkgs/archive/refs/heads/master.tar.gz | tar -xz
$ cd nixpkgs-master/
$ time nix build -f ./nixos/tests/hadoop/hdfs.nix --out-link hadoop-hdfs

real	1m44.698s
user	2m23.901s
sys	0m55.611s

$ time nix build -f ./nixos/tests/hadoop/yarn.nix --out-link hadoop-yarn

real	0m28.016s
user	0m41.740s
sys	0m19.863s

$ time nix build -f ./nixos/tests/hadoop/hadoop.nix --out-link hadoop

real	5m2.715s
user	7m42.968s
sys	3m52.261s

jonringer · 2022-02-10T03:54:12Z

pkgs/applications/networking/cluster/hadoop/default.nix

let

probably going to need to make an assertion

attribute 'i686-linux' missing, at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/mr-est/ofborg-evaluator-0/pkgs/applications/networking/cluster/hadoop/default.nix:31:18

What would that assert look like? Do you have any pointers/references to things I could look at?

ConnorBaker · 2022-02-10T04:59:36Z

As of the latest, I was able to build the tests successfully on aarch64 and on x86_64.

Here it is on aarch64:

$ time nix build github:/connorbaker/nixpkgs/master#nixosTests.{hadoop,hadoop-yarn,hadoop-hdfs} --store ~/ramdisk

real	5m49.831s
user	8m3.129s
sys	4m53.577s

and on x86_64 I got:

real	4m51.494s
user	8m24.644s
sys	4m47.754s

While debugging, I found that some of the problems I was having were a result of specifying store in ~/.config/nix/nix.conf (or, at the least, they went away when I specified store as a command-line argument).

After modifying store in ~/.config/nix/nix.conf, is there anything I need to do to make the changes take effect?

This commit also changes the names of the tests for Hadoop so they use dashes instead of dots, and makes the default `hadoop` test what would have been `hadoop-all` after the rename. This change should mean that we're able to run `nix build github:nixos/nixpkgs/master#nixosTests.hadoop` which I was unable to do prior to this change.

…ommon

ConnorBaker · 2022-02-22T15:57:09Z

Volth and @illustris, would you also be willing to review?

Volth, I can't find your GitHub user -- if you're still maintaining Hadoop, can you update maintainers-list.nix?

illustris · 2022-02-23T04:15:07Z

Diff looks good. Also verified that NixOS tests succeed on aarch64 and x86_64. Thanks!
I may not be of much help when it comes to MacOS, but can you share the details of any errors you encountered when using libPatches as-is on darwin?

…to simplify expression

ConnorBaker · 2022-02-24T04:05:47Z

Sure!

See ConnorBaker@799d743 for the latest.

(I had to wrap patchelf in an optionalString because that doesn't work on Darwin.)

So here's the latest build on an x86_64-linux:

$ nix build github:connorbaker/nixpkgs#hadoop --print-build-logs
hadoop> unpacking sources
hadoop> unpacking source archive /nix/store/57vfasjl07by1fb4krlh9sn4hmz4w5cs-hadoop-3.3.1.tar.gz
hadoop> source root is hadoop-3.3.1
hadoop> setting SOURCE_DATE_EPOCH to timestamp 1623737898 of file hadoop-3.3.1/share/doc/hadoop/project-reports.html
hadoop> patching sources
hadoop> configuring
hadoop> no configure script, doing nothing
hadoop> building
hadoop> no Makefile, doing nothing
hadoop> running tests
hadoop> no Makefile or custom checkPhase, doing nothing
hadoop> installing
hadoop> post-installation fixup
hadoop> shrinking RPATHs of ELF executables and libraries in /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1
hadoop> shrinking /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/libnativetask.so.1.0.0
hadoop> shrinking /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/libhdfs.so.0.0.0
hadoop> shrinking /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/libhadoop.so.1.0.0
hadoop> shrinking /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/libhdfspp.so.0.1.0
hadoop> shrinking /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/examples/wordcount-nopipe
hadoop> shrinking /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/examples/wordcount-part
hadoop> shrinking /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/examples/wordcount-simple
hadoop> shrinking /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/examples/pipes-sort
hadoop> shrinking /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/bin/test-container-executor
hadoop> shrinking /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/bin/container-executor
hadoop> shrinking /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/bin/oom-listener
hadoop> strip is /nix/store/v5a649mvjkgwcxsjiz6ngw03xhnh7isp-binutils-wrapper-2.35.2/bin/strip
hadoop> stripping (with command strip and flags -S) in /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib  /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/bin
hadoop> patching script interpreter paths in /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/resourceestimator/bin/stop-estimator.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/resourceestimator/bin/start-estimator.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/resourceestimator/bin/estimator.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/sls/bin/rumen2sls.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/sls/bin/slsrun.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-workload/bin/parse-start-timestamp.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-workload/bin/start-workload.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-blockgen/bin/generate-block-lists.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-infra/bin/upload-fsimage.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-infra/bin/parse-metrics.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-infra/bin/create-slim-hadoop-tar.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-infra/bin/start-dynamometer-cluster.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/start-all.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/stop-yarn.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/start-dfs.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/stop-dfs.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/hadoop-daemon.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/mr-jobhistory-daemon.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/stop-all.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/kms.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/yarn-daemons.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/stop-secure-dns.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/distribute-exclude.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/workers.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/start-yarn.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/refresh-namenodes.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/start-secure-dns.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/start-balancer.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/stop-balancer.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/yarn-daemon.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/httpfs.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/hadoop-daemons.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/hdfs-config.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/hadoop-config.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-streaming.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-distcp.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-archive-logs.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-gridmix.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-mapreduce.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-azure.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-s3guard.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-kms.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-azure-datalake.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-httpfs.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-openstack.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-yarn.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-aws.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-kafka.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-rumen.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-archives.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-extras.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-hdfs.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-aliyun.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/mapred-config.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/hadoop-functions.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-streaming.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-distcp.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-archive-logs.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-gridmix.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-sls.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-resourceestimator.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-dynamometer-blockgen.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-dynamometer-workload.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-aws.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-rumen.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-archives.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-extras.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-dynamometer-infra.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/hadoop-layout.sh.example: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/bin/mapred: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/bin/yarn: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/bin/hadoop: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/bin/hdfs: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/4nmqxajzaf60yjribkgvj5j54x9yvr1r-bash-5.1-p12/bin/bash"
hadoop> checking for references to /build/ in /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1...
hadoop> automatically fixing dependencies for ELF files
hadoop> searching for dependencies of /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/libhdfspp.so.0.1.0
hadoop>   libprotobuf.so.18 -> found: /nix/store/n7wm7y0jlppd8hrmbn1gd7f9ji7qrkpg-protobuf-3.7.1/lib/libprotobuf.so.18
hadoop>   libcrypto.so.1.1 -> found: /nix/store/2aprvflf41z1dfma87mzd3md51xm2g71-openssl-1.1.1m/lib/libcrypto.so.1.1
hadoop>   libsasl2.so.2 -> found: /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/libsasl2.so.2
hadoop>   libstdc++.so.6 -> found: /nix/store/ndnqiz3nnifj1blhg9q626xlmkqq1nmh-gcc-10.3.0-lib/lib/libstdc++.so.6
hadoop> setting RPATH to: /nix/store/n7wm7y0jlppd8hrmbn1gd7f9ji7qrkpg-protobuf-3.7.1/lib:/nix/store/2aprvflf41z1dfma87mzd3md51xm2g71-openssl-1.1.1m/lib:/nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native:/nix/store/ndnqiz3nnifj1blhg9q626xlmkqq1nmh-gcc-10.3.0-lib/lib
hadoop> searching for dependencies of /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/libhadoop.so.1.0.0
hadoop> searching for dependencies of /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/libhdfs.so.0.0.0
hadoop>   libjvm.so -> found: /nix/store/kylb9qkyg5njrkln7fpx5lpb4qidnjvg-openjdk-headless-11.0.12+7/lib/openjdk/lib/server/libjvm.so
hadoop> setting RPATH to: /nix/store/kylb9qkyg5njrkln7fpx5lpb4qidnjvg-openjdk-headless-11.0.12+7/lib/openjdk/lib/server
hadoop> searching for dependencies of /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/libnativetask.so.1.0.0
hadoop>   libz.so.1 -> found: /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/libz.so.1
hadoop>   libsnappy.so.1 -> found: /nix/store/4i9d1kj13qw0fm1f05zcga715ld3cfp9-snappy-1.1.9/lib/libsnappy.so.1
hadoop>   libjvm.so -> found: /nix/store/kylb9qkyg5njrkln7fpx5lpb4qidnjvg-openjdk-headless-11.0.12+7/lib/openjdk/lib/server/libjvm.so
hadoop>   libstdc++.so.6 -> found: /nix/store/ndnqiz3nnifj1blhg9q626xlmkqq1nmh-gcc-10.3.0-lib/lib/libstdc++.so.6
hadoop> setting RPATH to: /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native:/nix/store/4i9d1kj13qw0fm1f05zcga715ld3cfp9-snappy-1.1.9/lib:/nix/store/kylb9qkyg5njrkln7fpx5lpb4qidnjvg-openjdk-headless-11.0.12+7/lib/openjdk/lib/server:/nix/store/ndnqiz3nnifj1blhg9q626xlmkqq1nmh-gcc-10.3.0-lib/lib
hadoop> searching for dependencies of /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/examples/pipes-sort
hadoop>   libcrypto.so.1.1 -> found: /nix/store/2aprvflf41z1dfma87mzd3md51xm2g71-openssl-1.1.1m/lib/libcrypto.so.1.1
hadoop>   libstdc++.so.6 -> found: /nix/store/ndnqiz3nnifj1blhg9q626xlmkqq1nmh-gcc-10.3.0-lib/lib/libstdc++.so.6
hadoop> setting RPATH to: /nix/store/2aprvflf41z1dfma87mzd3md51xm2g71-openssl-1.1.1m/lib:/nix/store/ndnqiz3nnifj1blhg9q626xlmkqq1nmh-gcc-10.3.0-lib/lib
hadoop> searching for dependencies of /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/examples/wordcount-simple
hadoop>   libcrypto.so.1.1 -> found: /nix/store/2aprvflf41z1dfma87mzd3md51xm2g71-openssl-1.1.1m/lib/libcrypto.so.1.1
hadoop>   libstdc++.so.6 -> found: /nix/store/ndnqiz3nnifj1blhg9q626xlmkqq1nmh-gcc-10.3.0-lib/lib/libstdc++.so.6
hadoop> setting RPATH to: /nix/store/2aprvflf41z1dfma87mzd3md51xm2g71-openssl-1.1.1m/lib:/nix/store/ndnqiz3nnifj1blhg9q626xlmkqq1nmh-gcc-10.3.0-lib/lib
hadoop> searching for dependencies of /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/examples/wordcount-part
hadoop>   libcrypto.so.1.1 -> found: /nix/store/2aprvflf41z1dfma87mzd3md51xm2g71-openssl-1.1.1m/lib/libcrypto.so.1.1
hadoop>   libstdc++.so.6 -> found: /nix/store/ndnqiz3nnifj1blhg9q626xlmkqq1nmh-gcc-10.3.0-lib/lib/libstdc++.so.6
hadoop> setting RPATH to: /nix/store/2aprvflf41z1dfma87mzd3md51xm2g71-openssl-1.1.1m/lib:/nix/store/ndnqiz3nnifj1blhg9q626xlmkqq1nmh-gcc-10.3.0-lib/lib
hadoop> searching for dependencies of /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/examples/wordcount-nopipe
hadoop>   libcrypto.so.1.1 -> found: /nix/store/2aprvflf41z1dfma87mzd3md51xm2g71-openssl-1.1.1m/lib/libcrypto.so.1.1
hadoop>   libstdc++.so.6 -> found: /nix/store/ndnqiz3nnifj1blhg9q626xlmkqq1nmh-gcc-10.3.0-lib/lib/libstdc++.so.6
hadoop> setting RPATH to: /nix/store/2aprvflf41z1dfma87mzd3md51xm2g71-openssl-1.1.1m/lib:/nix/store/ndnqiz3nnifj1blhg9q626xlmkqq1nmh-gcc-10.3.0-lib/lib
hadoop> searching for dependencies of /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/bin/oom-listener
hadoop> searching for dependencies of /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/bin/container-executor
hadoop>   libcrypto.so.1.1 -> found: /nix/store/2aprvflf41z1dfma87mzd3md51xm2g71-openssl-1.1.1m/lib/libcrypto.so.1.1
hadoop> setting RPATH to: /nix/store/2aprvflf41z1dfma87mzd3md51xm2g71-openssl-1.1.1m/lib
hadoop> searching for dependencies of /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/bin/test-container-executor
hadoop>   libcrypto.so.1.1 -> found: /nix/store/2aprvflf41z1dfma87mzd3md51xm2g71-openssl-1.1.1m/lib/libcrypto.so.1.1
hadoop> setting RPATH to: /nix/store/2aprvflf41z1dfma87mzd3md51xm2g71-openssl-1.1.1m/lib

And here's the same build on the latest on aarch64-darwin:

$ nix build .#hadoop --print-build-logs 
hadoop> unpacking sources
hadoop> unpacking source archive /nix/store/7i751wf2lyx1rfkm2207rb51vbckzch5-hadoop-3.3.1-aarch64.tar.gz
hadoop> source root is hadoop-3.3.1
hadoop> setting SOURCE_DATE_EPOCH to timestamp 1623758040 of file hadoop-3.3.1/share/doc/hadoop/project-reports.html
hadoop> patching sources
hadoop> updateAutotoolsGnuConfigScriptsPhase
hadoop> configuring
hadoop> no configure script, doing nothing
hadoop> building
hadoop> no Makefile, doing nothing
hadoop> running tests
hadoop> no Makefile or custom checkPhase, doing nothing
hadoop> installing
hadoop> post-installation fixup
hadoop> strip is /nix/store/km02igh4pshp20d0wn89rf5jjfxcm8v5-clang-wrapper-11.1.0/bin/strip
hadoop> stripping (with command strip and flags -S) in /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib  /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/bin
hadoop> patching script interpreter paths in /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/bin/mapred: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/bin/yarn: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/bin/hadoop: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/bin/hdfs: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-extras.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-distcp.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-archive-logs.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-sls.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-resourceestimator.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-dynamometer-infra.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-rumen.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-dynamometer-blockgen.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-streaming.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-aws.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-dynamometer-workload.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-archives.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/tools/hadoop-gridmix.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/hadoop-config.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/hadoop-layout.sh.example: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/hadoop-functions.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-extras.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-s3guard.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-distcp.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-kms.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-openstack.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-archive-logs.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-mapreduce.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-httpfs.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-azure-datalake.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-azure.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-rumen.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-yarn.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-streaming.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-aws.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-kafka.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-aliyun.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-hdfs.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-archives.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/shellprofile.d/hadoop-gridmix.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/hdfs-config.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/libexec/mapred-config.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/yarn-daemon.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/yarn-daemons.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/distribute-exclude.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/start-yarn.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/start-all.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/mr-jobhistory-daemon.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/kms.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/workers.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/stop-dfs.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/start-secure-dns.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/stop-balancer.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/stop-secure-dns.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/start-dfs.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/httpfs.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/stop-yarn.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/start-balancer.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/hadoop-daemons.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/hadoop-daemon.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/refresh-namenodes.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/sbin/stop-all.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/sls/bin/rumen2sls.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/sls/bin/slsrun.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-infra/bin/upload-fsimage.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-infra/bin/create-slim-hadoop-tar.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-infra/bin/parse-metrics.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-infra/bin/start-dynamometer-cluster.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-blockgen/bin/generate-block-lists.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-workload/bin/start-workload.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-workload/bin/parse-start-timestamp.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/resourceestimator/bin/stop-estimator.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/resourceestimator/bin/estimator.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"
hadoop> /nix/store/g1bkm13bdn7a2im98ijwkgx7vbwfsgl3-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/resourceestimator/bin/start-estimator.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p12/bin/bash"

On x86_64-linux we see

$ ./result/bin/hadoop checknative -a
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
2022-02-24 03:57:41,976 INFO  [main] bzip2.Bzip2Factory (Bzip2Factory.java:isNativeBzip2Loaded(61)) - Successfully loaded & initialized native-bzip2 library system-native
2022-02-24 03:57:41,978 INFO  [main] zlib.ZlibFactory (ZlibFactory.java:loadNativeZLib(59)) - Successfully loaded & initialized native-zlib library
2022-02-24 03:57:41,981 WARN  [main] erasurecode.ErasureCodeNative (ErasureCodeNative.java:<clinit>(49)) - Loading ISA-L failed: Failed to load libisal.so.2 (libisal.so.2: cannot open shared object file: No such file or directory)
2022-02-24 03:57:41,981 WARN  [main] erasurecode.ErasureCodeNative (ErasureCodeNative.java:<clinit>(55)) - ISA-L support is not available in your platform... using builtin-java codec where applicable
2022-02-24 03:57:42,002 INFO  [main] nativeio.NativeIO (NativeIO.java:isPmdkAvailable(192)) - The native code was built without PMDK support.
Native library checking:
hadoop:  true /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/libhadoop.so.1.0.0
zlib:    true /nix/store/bqjh8dc6m5plswxrmmn9x4cpxrrw94lv-zlib-1.2.11/lib/libz.so.1
zstd  :  true /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/libzstd.so.1
bzip2:   true /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/libbz2.so.1
openssl: true /nix/store/i9pg1y2ki390xp6ypcpglzqk92slr6bd-hadoop-3.3.1/lib/hadoop-3.3.1/lib/native/libcrypto.so
ISA-L:   false Loading ISA-L failed: Failed to load libisal.so.2 (libisal.so.2: cannot open shared object file: No such file or directory)
PMDK:    false The native code was built without PMDK support.
2022-02-24 03:57:42,007 INFO  [main] util.ExitUtil (ExitUtil.java:terminate(210)) - Exiting with status 1: ExitException

while on aarch64-darwin we have

$ ./result/bin/hadoop checknative -a
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
2022-02-23 23:03:11,804 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(60)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Native library checking:
hadoop:  false 
zlib:    false 
zstd  :  false 
bzip2:   false 
openssl: false 
ISA-L:   false 
PMDK:    false 
2022-02-23 23:03:11,924 INFO  [main] util.ExitUtil (ExitUtil.java:terminate(210)) - Exiting with status 1: ExitException

I've talked (a little bit) previously with people about how to do the same patching on macOS but I can't find my notes on it at the moment. If you happen to know what commands would be needed to help Hadoop find the libraries, let me know!

illustris · 2022-02-24T11:55:07Z

From the logs, it looks like autoPatchelfHook isn't running at all on darwin.

https://github.com/ConnorBaker/nixpkgs/blob/799d7434e2a49407b6506cecb21d3f1fd3d3ae4c/pkgs/applications/networking/cluster/hadoop/default.nix#L38

optional (stdenv.isLinux && (nativeLibs != [ ] || libPatches != "")) [ autoPatchelfHook ];

You'll just have to drop the stdenv.isLinux here.

ConnorBaker · 2022-02-24T13:48:49Z

I removed it as suggested and it errored (I thought patchelf didn't work on Darwin)?

Here are the last few lines of the log (the rest of which is identical to the log before the change):

/nix/store/qx36hf946n1bcdb60nw8iambf1500r28-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-blockgen/bin/generate-block-lists.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40
/nix/store/qx36hf946n1bcdb60nw8iambf1500r28-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-workload/bin/start-workload.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9
/nix/store/qx36hf946n1bcdb60nw8iambf1500r28-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/dynamometer/dynamometer-workload/bin/parse-start-timestamp.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c4
/nix/store/qx36hf946n1bcdb60nw8iambf1500r28-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/resourceestimator/bin/stop-estimator.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5
/nix/store/qx36hf946n1bcdb60nw8iambf1500r28-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/resourceestimator/bin/estimator.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-5.1-p1
/nix/store/qx36hf946n1bcdb60nw8iambf1500r28-hadoop-3.3.1/lib/hadoop-3.3.1/share/hadoop/tools/resourceestimator/bin/start-estimator.sh: interpreter directive changed from "#!/usr/bin/env bash" to "/nix/store/m0nsx4ai768kjp9c40vhkpj9pxlii46m-bash-
automatically fixing dependencies for ELF files
/nix/store/a4q92isrpiy004njsk6x66rsahw62am3-auto-patchelf-hook/nix-support/setup-hook: line 333: -l: command not found

I tried again after also removing the optional in

optionalString stdenv.isLinux "patchelf --add-rpath ${jdk.home}/lib/server $out/lib/${untarDir}/lib/native/libnativetask.so.1.0.0";

to see if that helped (it would make sense to use patchelf if we're using autoPatchelfhook right?), but it didn't:

$ nix log /nix/store/s4jpnbvmmfk4kgk7rl2292kis1n2gd35-hadoop-3.3.1.drv
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/7i751wf2lyx1rfkm2207rb51vbckzch5-hadoop-3.3.1-aarch64.tar.gz
source root is hadoop-3.3.1
setting SOURCE_DATE_EPOCH to timestamp 1623758040 of file hadoop-3.3.1/share/doc/hadoop/project-reports.html
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "updateAutotoolsGnuConfigScriptsPhase" }
updateAutotoolsGnuConfigScriptsPhase
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
no configure script, doing nothing
@nix { "action": "setPhase", "phase": "buildPhase" }
building
no Makefile, doing nothing
@nix { "action": "setPhase", "phase": "checkPhase" }
running tests
no Makefile or custom checkPhase, doing nothing
@nix { "action": "setPhase", "phase": "installPhase" }
installing
/nix/store/0qlq4s175mzlx9g0aqcdk95j92g1z7mf-stdenv-darwin/setup: line 1378: patchelf: command not found

Any ideas?

illustris · 2022-02-24T14:28:52Z

ah, my bad... Yes, patchelf doesn't work on darwin. I'm not sure if hadoop with native libs can even work in darwin with the way we're packaging now.

[illustris@desktop:/dev/shm/hadoop-3.3.1/lib/native]$ file libhadoop.so.1.0.0
libhadoop.so.1.0.0: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[sha1]=c08694691c54cd5ce768dea281674437c23afaf4, with debug_info, not stripped

The native lib files bundled with hadoop are ELF files. Apache doesn't release builds with .dylib files for darwin, and it isn't supported on darwin. It might still be possible to build native libs for darwin, but it might not be a good idea to do that in nixpkgs.

ConnorBaker · 2022-02-24T16:27:28Z

I actually looked into building Hadoop from source, but all the Java tooling threw me... maybe I'll try again?

As long as the builds are cached, end-users shouldn't have to compile it themselves, right? And we can always offer the binary distribution alongside it.

illustris · 2022-02-24T16:59:33Z

That's how it was done before #141143

Building from source is not a good idea because

maven central does not guarantee the same checksum for an artifact pulled from different mirrors
the build process for hadoop will download multiple dynamically linked binaries that will get used at build time. I tried looping mvn build and patchelf, but some of these binaries are overwritten by a fresh copy of the unpatched binary during the subsequent run of maven
if the end user wants to change their java version, it will force a very expensive full rebuild

ConnorBaker · 2022-02-24T17:29:05Z

Yikes, no kidding!

So without finding a way to link the libraries on macOS (which I absolutely lack the knowledge to do) I guess this is as good as it'll get.

In terms of the scope of this pull request, do you think it's good? Are there other things you'd like to see done, or ways I can clean it up further? I'm still learning about most of nix's builtins and the nixpkg's lib, so any advice would be appreciated!

illustris

The PR looks good. I just have a few minor non-blocking suggestions.

pkgs/applications/networking/cluster/hadoop/default.nix

ConnorBaker · 2022-02-26T00:23:04Z

Great feedback!

I had to keep the isLinux check to prevent autoPatchelfHook from running on darwin (which causes a crash). If we don't have that check, then even with the patchelf call in libPatches guarded, we still get this error:

$ nix log /nix/store/3xmyasz6bxsfq0608krxv7pbs9kbx4by-hadoop-3.3.1.drv
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/7i751wf2lyx1rfkm2207rb51vbckzch5-hadoop-3.3.1-aarch64.tar.gz
source root is hadoop-3.3.1
setting SOURCE_DATE_EPOCH to timestamp 1623758040 of file hadoop-3.3.1/share/doc/hadoop/project-reports.html
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "updateAutotoolsGnuConfigScriptsPhase" }
updateAutotoolsGnuConfigScriptsPhase
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
no configure script, doing nothing
@nix { "action": "setPhase", "phase": "buildPhase" }
building
no Makefile, doing nothing
@nix { "action": "setPhase", "phase": "checkPhase" }
running tests
no Makefile or custom checkPhase, doing nothing
@nix { "action": "setPhase", "phase": "installPhase" }
installing
/nix/store/0qlq4s175mzlx9g0aqcdk95j92g1z7mf-stdenv-darwin/setup: line 1378: patchelf: command not found

github-actions bot added the 6.topic: nixos label Feb 8, 2022

ConnorBaker force-pushed the master branch from 075c9df to 43490e2 Compare February 8, 2022 13:27

ofborg bot added 8.has: clean-up 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Feb 8, 2022

jonringer approved these changes Feb 8, 2022

View reviewed changes

jonringer reviewed Feb 8, 2022

View reviewed changes

ConnorBaker marked this pull request as draft February 9, 2022 01:44

ConnorBaker force-pushed the master branch from 6bafe88 to 6fe168d Compare February 10, 2022 02:33

jonringer reviewed Feb 10, 2022

View reviewed changes

ConnorBaker added 7 commits February 14, 2022 08:20

hadoop: Don't case-split on supported architectures

2bb25db

hadoop: Use sha256 from nix-prefetch-url for aarch64 builds

f389702

hadoop: Make x86_64 settings default and conditionally override

d321b39

hadoop: aarch64-linux has native libraries, darwin does not

50c8089

hadoop: Group attrset update to ensure completion before passing to c…

e752dfc

…ommon

hadoop: Consolidate sha256 attrs and add assert for supported platforms

1e5ffe1

ConnorBaker force-pushed the master branch from 6d20952 to 1e5ffe1 Compare February 14, 2022 14:24

ConnorBaker marked this pull request as ready for review February 14, 2022 16:01

ConnorBaker mentioned this pull request Feb 15, 2022

spark: init 3.2.1 and test on aarch64-linux #160075

Merged

13 tasks

Artturin requested a review from jonringer February 21, 2022 21:48

hadoop: add back dots in names of nixos tests

e10eed8

hadoop: link native libraries on Darwin even though they're not used …

799d743

…to simplify expression

illustris reviewed Feb 25, 2022

View reviewed changes

hadoop: Minor simplifications to default.nix

2cf9686

illustris mentioned this pull request Feb 27, 2022

nixos/hadoop: module and package improvements #154004

Merged

13 tasks

garbas merged commit 43a7654 into NixOS:master Mar 1, 2022

Janik-Haag added the 12. first-time contribution label Jun 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

hadoop: add aarch64 support #158613

hadoop: add aarch64 support #158613

ConnorBaker commented Feb 8, 2022 •

edited

Loading

jonringer left a comment

ConnorBaker commented Feb 8, 2022

jonringer commented Feb 8, 2022

jonringer Feb 8, 2022

jonringer Feb 8, 2022

ConnorBaker Feb 8, 2022

jonringer Feb 8, 2022

ConnorBaker Feb 8, 2022

ConnorBaker commented Feb 9, 2022 •

edited

Loading

ConnorBaker commented Feb 9, 2022 •

edited

Loading

jonringer Feb 10, 2022

ConnorBaker Feb 10, 2022

ConnorBaker commented Feb 10, 2022

ConnorBaker commented Feb 22, 2022

illustris commented Feb 23, 2022 •

edited

Loading

ConnorBaker commented Feb 24, 2022

illustris commented Feb 24, 2022

ConnorBaker commented Feb 24, 2022

illustris commented Feb 24, 2022

ConnorBaker commented Feb 24, 2022

illustris commented Feb 24, 2022

ConnorBaker commented Feb 24, 2022

illustris left a comment

ConnorBaker commented Feb 26, 2022

hadoop: add aarch64 support #158613

hadoop: add aarch64 support #158613

Conversation

ConnorBaker commented Feb 8, 2022 • edited Loading

Motivation for this change

Things done

jonringer left a comment

Choose a reason for hiding this comment

ConnorBaker commented Feb 8, 2022

jonringer commented Feb 8, 2022

jonringer Feb 8, 2022

Choose a reason for hiding this comment

jonringer Feb 8, 2022

Choose a reason for hiding this comment

ConnorBaker Feb 8, 2022

Choose a reason for hiding this comment

jonringer Feb 8, 2022

Choose a reason for hiding this comment

ConnorBaker Feb 8, 2022

Choose a reason for hiding this comment

ConnorBaker commented Feb 9, 2022 • edited Loading

ConnorBaker commented Feb 9, 2022 • edited Loading

jonringer Feb 10, 2022

Choose a reason for hiding this comment

ConnorBaker Feb 10, 2022

Choose a reason for hiding this comment

ConnorBaker commented Feb 10, 2022

ConnorBaker commented Feb 22, 2022

illustris commented Feb 23, 2022 • edited Loading

ConnorBaker commented Feb 24, 2022

illustris commented Feb 24, 2022

ConnorBaker commented Feb 24, 2022

illustris commented Feb 24, 2022

ConnorBaker commented Feb 24, 2022

illustris commented Feb 24, 2022

ConnorBaker commented Feb 24, 2022

illustris left a comment

Choose a reason for hiding this comment

ConnorBaker commented Feb 26, 2022

ConnorBaker commented Feb 8, 2022 •

edited

Loading

ConnorBaker commented Feb 9, 2022 •

edited

Loading

ConnorBaker commented Feb 9, 2022 •

edited

Loading

illustris commented Feb 23, 2022 •

edited

Loading