Replace custom require loader with process.dlopen and using static path to node.napi.node

[CMCDragonkai]

Specification

Native bindings is currently loaded using requireBinding. This is integrated into our native addon packaging style using optional dependencies.

However this won't work as part of ESM. So it has to be changed to using dlopen in preparation for ESM migration.

Furthermore, require supports the package.json specification of main. Which we are currently setting to node.napi.node. However instead we just fix statically where our .node files will be, and that it will always be in @matrixai/quic-linux-x64/node.napi.node.

This means dlopen should work better, and plus it becomes simpler to use when it comes to integrating into esbuild bundling and pkg.

In particular it means:

• esbuild needs to specify @matrixai/quic-linux-x64 as an external package, since esbuild cannot bundle such a thing - this is done by setting it in optionalDependencies and passing it to the external option in await esbuild.build(esbuildOptions). - The reason it is in optionalDependencies is because this allows it to be installed into the node_modules so when a nix-build occurs, it will in fact exist when esbuild is running. This we cannot really automate, and we have to do this for all native addons. See package.json in Polykey-CLI.
• pkg needs to specify an asset as node_modules/@matrixai/quic-linux-x64/node.napi.node. - Currently we're using node-gyp-build to auto-find native modules, however this does not work for our custom optional dependency structure. So currently this is also manually specified.

I think it should be possible to update our native module finding code to look for node.napi.node that is within our direct dependencies too, by just iterating through the root directory of each package, then it should be possible to make that automatic as well. That would reduce this:

  "pkg": {
    "assets": [
      "node_modules/@matrixai/quic-linux-x64/node.napi.node",
      "node_modules/@matrixai/quic-linux-x64/package.json",
      "node_modules/@matrixai/quic-darwin-x64/node.napi.node",
      "node_modules/@matrixai/quic-darwin-x64/package.json",
      "node_modules/@matrixai/quic-darwin-arm64/node.napi.node",
      "node_modules/@matrixai/quic-darwin-arm64/package.json",
      "node_modules/@matrixai/quic-win32-x64/node.napi.node",
      "node_modules/@matrixai/quic-win32-x64/package.json",
      "node_modules/@matrixai/mdns-linux-x64/node.napi.node",
      "node_modules/@matrixai/mdns-linux-x64/package.json"
    ],
    "scripts": [
      "dist/polykeyWorker.js"
    ]
  },

Down to:

  "pkg": {
    "assets": [],
    "scripts": [
      "dist/polykeyWorker.js"
    ]
  },

Since it would be auto-found.

Additional context

• ci: merge staging to master Polykey-CLI#4 (comment)

Tasks

1. ...
2. ...
3. ...

[CMCDragonkai]

If this is done, we should also be updating @matrixai/db and the other native code as much as possible. @amydevs