Should we run .../node-packages/generate.sh in batches instead of on every new package?

[tomodachi94]

Should we run pkgs/development/node-packages/generate.sh in batches instead of on every new package?

Currently:

• The script takes about four hours to run (source).
• It can cause merge conflicts whenever a new nodePackages package is merged. (look at all of these closed nodePackages PRs that had merge conflicts!)
• The same updates are performed many times when multiple PRs adding to nodePackages are filed, wasting compute time.

If this is a bad idea, please tell me :) I'm not very familiar with nodePackages.

Proposal

Run node-packages/generate.sh on a weekly (or other time period) basis (either manually or automatically), instead of instructing developers to run it when adding a new package.

Discourage developers from running the script when adding a new package.

Considerations

Would the Node team become overwhelmed by this?

Should we automate the proposed process (sans merging)?

Is this even worth formalizing, considering that we are actively phasing out packages in nodePackages?

Maintainers

Pinging @NixOS/node.

[tomodachi94]

We could also modify generate.sh to support adding/updating specific attributes, but I'm not sure how difficult this would be to implement.

[pyrox0]

I don't think this should be formalized, as we should be working to remove as many packages as possible from node-packages into the main set, or drop those that are unused. For instance, why is bower still listed? It's been unmaintained for many years now(last commit was 2 years ago), and thus I feel it should be dropped. Doing any sort of regular maintainence on the set is a hassle, and I think setting up CI for it only encourages the migration to take longer than it has to. Also, you have to run the script to add it to node-packages.nix anyways, so it would have to be run whether or not it's autorun in CI.

Edit 1: I honestly think that the node-packages package set should be frozen, and dependencies shouldn't be updated, as it causes a massive amount of churn. Packages should just be migrated out as necessary, but the ones there currently should not be updated, as that also encourages migrating packages away if people want them to be updated.

[SuperSandro2000]

node2nix doesn't support running this in batches or parallel.

Also see previous comment

[tomodachi94]

node2nix doesn't support running this in batches or parallel.

Yes, I'm aware.

Proposal

Run node-packages/generate.sh on a weekly (or other time period) basis (either manually or automatically), instead of instructing developers to run it when adding a new package.

But I agree that we should probably freeze the package set (except for package removals).