nwaku: init at 0.16.0 #193679

jakubgs · 2022-09-30T10:02:19Z

Description of changes

Waku is the communication layer for Web3. Decentralized communication that scales.

Things done

nixos/modules/services/security/nwaku.nix

pkgs/tools/security/nwaku/default.nix

jakubgs · 2023-03-06T16:05:47Z

I have tested this using the following config.nix:

{
  services.nwaku = {
    enable = true;
    tcpPort = 8888;
    protocols = ["store" "relay" "filter" "swap" "lightpush"];
    nodekey = "abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234";
    nat = "extip:12.34.56.78";
    rest = {
      enable = true;
      admin = true;
      port = 5052;
    };
    dnsDiscovery = {
      enable = true;
      url = "enrtree://AOGECG2SPND25EEFMAJ5WF3KSGJNSGV356DSTL2YVLLZWIV6SAYBM@prod.nodes.status.im";
    };
    storeMessage = {
      retentionPolicy = "time:${toString (7*24*60*60)}"; # 7 days
    };
  };

  services.getty.autologinUser = "root";
  systemd.services.nwaku.serviceConfig.StandardOutput = "journal+console";
}

And ran it using:

nixos-generate -f vm-nogui -c $PWD/config.nix -I nixpkgs=$HOME/work/nixpkgs

Which worked without issues.

nixos-discourse · 2023-03-06T17:07:02Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/1910

pkgs/top-level/all-packages.nix

pkgs/tools/security/nwaku/default.nix

nixos/modules/services/security/nwaku.nix

SuperSandro2000 · 2023-03-06T18:50:42Z

nixos/modules/services/security/nwaku.md

+  };
+};
+```
+This node will store 30 days worth of messages it will receive under `/var/run/nwaku`.


/var/run is a symlink to /run which is a tmpfs.

That's a good point. Is there any stock NixOS way of creating a /var/lib directory for a service?

Oh, it just works:

[root@nixos:~]# ls -l /var/lib/nwaku/ total 4648 -rw-r--r-- 1 nwaku nwaku 598016 Mar 6 22:54 store.sqlite3 -rw-r--r-- 1 nwaku nwaku 32768 Mar 6 22:57 store.sqlite3-shm -rw-r--r-- 1 nwaku nwaku 4124152 Mar 6 22:57 store.sqlite3-wal

We usually use tmpfiles.d

But those are not temporary files. That's the store of historical messages.

jakubgs · 2023-03-06T23:02:24Z

Thanks for reviewing this. Appreciate it.

SuperSandro2000

linux-aarch64 fails with:

gcc: error: unrecognized command line option '-msse3'
stack trace: (most recent call last)
/nix/store/86y677haw24vwq0idczq6y340fhyrfss-nim-unwrapped-1.6.10/nim/lib/system/nimscript.nim(429, 18)
/build/source/waku.nimble(69, 15) wakunode2Task
/build/source/waku.nimble(35, 8) buildBinary
/nix/store/86y677haw24vwq0idczq6y340fhyrfss-nim-unwrapped-1.6.10/nim/lib/system/nimscript.nim(273, 7) exec
/nix/store/86y677haw24vwq0idczq6y340fhyrfss-nim-unwrapped-1.6.10/nim/lib/system/nimscript.nim:273:7 Error: unhandled exception: FAILED: nim c --out:build/wakunode2 -d:chronicles_log_level=TRACE -d:disableMarchNative --verbosity:0 --hints:off -d:git_version="v0.15.0" -d:release apps/wakunode2/wakunode2.nim [OSError]
make: *** [Makefile:144: wakunode2] Error 1
error: builder for '/nix/store/mnm84bfkbc0swfj4czs3b150i49wbxcp-nwaku-0.15.0-e12b7cb4.drv' failed with exit code 2;

pkgs/tools/security/nwaku/default.nix

jakubgs · 2023-03-14T14:54:23Z

Opened issue for ARM64 build issues:

bug: unable to build on arm64 due to msse3 feature waku-org/nwaku#1607

Will be fixed in next release, for now I'll remove support for that platform.

jakubgs · 2023-03-15T10:05:52Z

Oh, we might get a 0.16.0 release today with the fixes, so we can skip the patching.

jakubgs · 2023-03-15T14:26:35Z

I've upgraded to 0.16.0 which should resolve ARM build issues. But I had to re-add the Nim compiler compilation, since without the submodule pinned version of 1.6.11(b1a0467f) it fails with:

/build/source/vendor/nim-confutils/confutils.nim:905:28 Error: invalid pragma: warning[BareExcept]: off

So we can't make use of USE_SYSTEM_NIM=1, at least for this release.

Waku is the communication layer for Web3. Decentralized communication that scales. Signed-off-by: Jakub Sokołowski <[email protected]>

Signed-off-by: Jakub Sokołowski <[email protected]>

jakubgs · 2023-03-20T09:59:16Z

Actually, since this PR was just merged:

nim: 1.6.10 -> 1.6.12 #220540

We get Nimx 1.6.12 by default, and that means the build works without having to compile the compiler again for this package.

SuperSandro2000 · 2023-03-30T14:31:52Z

pkgs/tools/security/nwaku/default.nix

+assert nim.version == "1.6.12";
+


Suggested change

assert nim.version == "1.6.12";

No, that is staying there. The Nim compiler version check is necessary to ensure stability.

This will break when nim is updated and will likely be removed by the next person because of that

Then I can lock Nim version at all-packages.nix level as I do for nimbus-beacon-node package.

Is this really necessary? Are nim updates really that unstable? Nixpkgs is a distribution and normally a patch update for a compiler should not break all applications and we would normally apply that to all programs.

Yes. Status as a company has probably the most Nim developers, including Nim compiler contributors, among its staff than any other organization, and yes, the use of specific Nim compiler versions is required to avoid very weird issues.

I'll get to locking it up later this week.

SuperSandro2000 · 2023-03-30T14:32:32Z

pkgs/tools/security/nwaku/default.nix

+  commit = "03f0de36";
+  name = "${pname}-${version}-${commit}";


Suggested change

commit = "03f0de36";

name = "${pname}-${version}-${commit}";

No, we include the commit in the version of the node.

We don't add that to name.

I can remove it from name, but I will keep commit anyway for the sake of fake git for git rev-parse.

The contributing guide should mention how pname and version and name should be handled and we normally don't write commit hashes into name/version if we are using a release tag.

That's fine. I can remove it, but I will still keep it for fake git rev-parse, as I said.

SuperSandro2000 · 2023-03-30T14:32:37Z

pkgs/tools/security/nwaku/default.nix

+
+  buildInputs = let
+    # Fix for Nim compiler calling 'git rev-parse' and 'lsb_release'.
+    fakeGit = writeScriptBin "git" "echo $commit";


Suggested change

fakeGit = writeScriptBin "git" "echo $commit";

fakeGit = writeScriptBin "git" "echo v${version}";

we can cheat that easily

But I don't want the version. I want the commit. Why would I change it like that?

it would be easier to update and maintain and only adds very minimal information

I'm going to be maintaining this and I'll somehow cope with the immense burden of updating the commit attribute.

github-actions bot added 6.topic: nixos 8.has: changelog 8.has: documentation 8.has: module (update) labels Sep 30, 2022

ofborg bot added 8.has: package (new) 11.by: package-maintainer 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Sep 30, 2022

ilmotta reviewed Nov 7, 2022

View reviewed changes

jakubgs force-pushed the init-nwaku branch from 371f4fc to 031c516 Compare November 16, 2022 12:52

jakubgs mentioned this pull request Mar 6, 2023

fix: clean and normalize module imports waku-org/nwaku#1591

Merged

jakubgs changed the title ~~nwaku: init at 0.11.0~~ nwaku: init at 0.15.0 Mar 6, 2023

jakubgs force-pushed the init-nwaku branch 7 times, most recently from 52e8bdf to e8e2d68 Compare March 6, 2023 16:05

jakubgs marked this pull request as ready for review March 6, 2023 16:09

jakubgs force-pushed the init-nwaku branch 2 times, most recently from 34b5d30 to 7dd2d02 Compare March 6, 2023 17:28

SuperSandro2000 requested changes Mar 6, 2023

View reviewed changes

jakubgs force-pushed the init-nwaku branch from 7dd2d02 to cd855de Compare March 6, 2023 22:29

github-actions bot added the 6.topic: python label Mar 6, 2023

jakubgs force-pushed the init-nwaku branch from cd855de to f0d3a04 Compare March 6, 2023 22:30

jakubgs removed the 6.topic: python label Mar 6, 2023

jakubgs force-pushed the init-nwaku branch from f0d3a04 to 66fee4e Compare March 6, 2023 23:01

jakubgs force-pushed the init-nwaku branch from 66fee4e to 7f6dcf3 Compare March 6, 2023 23:01

jakubgs force-pushed the init-nwaku branch from 7f6dcf3 to b785a21 Compare March 6, 2023 23:20

jakubgs mentioned this pull request Mar 7, 2023

feat: allow providing node key using a file waku-org/nwaku#1593

Open

SuperSandro2000 reviewed Mar 13, 2023

View reviewed changes

pkgs/tools/security/nwaku/default.nix Outdated Show resolved Hide resolved

pkgs/tools/security/nwaku/default.nix Show resolved Hide resolved

jakubgs mentioned this pull request Mar 14, 2023

bug: unable to build on arm64 due to msse3 feature waku-org/nwaku#1607

Closed

jakubgs force-pushed the init-nwaku branch from b785a21 to 502dc25 Compare March 14, 2023 18:39

jakubgs force-pushed the init-nwaku branch from 502dc25 to dc9ae4f Compare March 15, 2023 14:18

jakubgs changed the title ~~nwaku: init at 0.15.0~~ nwaku: init at 0.16.0 Mar 15, 2023

jakubgs force-pushed the init-nwaku branch from dc9ae4f to 0ca6c44 Compare March 15, 2023 14:25

jakubgs force-pushed the init-nwaku branch from 0ca6c44 to 7786262 Compare March 20, 2023 09:14

jakubgs added 2 commits March 20, 2023 10:51

nwaku: init at v0.16.0

5a61ea3

Waku is the communication layer for Web3. Decentralized communication that scales. Signed-off-by: Jakub Sokołowski <[email protected]>

nwaku: init service definition

b59ce88

Signed-off-by: Jakub Sokołowski <[email protected]>

jakubgs force-pushed the init-nwaku branch from 7786262 to b59ce88 Compare March 20, 2023 09:52

SuperSandro2000 reviewed Mar 30, 2023

View reviewed changes

jakubgs requested a review from SuperSandro2000 April 3, 2023 07:24

wegank added 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 2.status: merge conflict labels Mar 19, 2024

stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 20, 2024

wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nwaku: init at 0.16.0 #193679

nwaku: init at 0.16.0 #193679

jakubgs commented Sep 30, 2022

jakubgs commented Mar 6, 2023

nixos-discourse commented Mar 6, 2023

SuperSandro2000 Mar 6, 2023

jakubgs Mar 6, 2023

jakubgs Mar 6, 2023

SuperSandro2000 Mar 13, 2023

jakubgs Mar 15, 2023

jakubgs commented Mar 6, 2023

SuperSandro2000 left a comment

jakubgs commented Mar 14, 2023 •

edited

Loading

jakubgs commented Mar 15, 2023

jakubgs commented Mar 15, 2023

jakubgs commented Mar 20, 2023

SuperSandro2000 Mar 30, 2023

jakubgs Apr 3, 2023

SuperSandro2000 Apr 14, 2023

jakubgs Apr 15, 2023

SuperSandro2000 Apr 30, 2023

jakubgs May 4, 2023

SuperSandro2000 Mar 30, 2023

jakubgs Apr 3, 2023

SuperSandro2000 Apr 14, 2023

jakubgs Apr 15, 2023

SuperSandro2000 Apr 30, 2023

jakubgs May 4, 2023

SuperSandro2000 Mar 30, 2023

jakubgs Apr 3, 2023

SuperSandro2000 Apr 14, 2023

jakubgs Apr 15, 2023

	fakeGit = writeScriptBin "git" "echo $commit";
	fakeGit = writeScriptBin "git" "echo v${version}";

nwaku: init at 0.16.0 #193679

Are you sure you want to change the base?

nwaku: init at 0.16.0 #193679

Conversation

jakubgs commented Sep 30, 2022

Description of changes

Things done

jakubgs commented Mar 6, 2023

nixos-discourse commented Mar 6, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakubgs commented Mar 6, 2023

SuperSandro2000 left a comment

Choose a reason for hiding this comment

jakubgs commented Mar 14, 2023 • edited Loading

jakubgs commented Mar 15, 2023

jakubgs commented Mar 15, 2023

jakubgs commented Mar 20, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakubgs commented Mar 14, 2023 •

edited

Loading