Add Homebrew Bundle module

[malob]

Overview and motivation

This module adds the ability for nix-darwin to manage packages/apps installed by Homebrew Bundle. This gives nix-darwin users the ability to not only manage packages/apps installed via Homebrew, but also apps from the Mac App Store via mas, and Docker containers via whalebrew.

While I wish I could manage installing all my packages/apps directly through Nix via nix-darwin there are a sizable number of cases where that currently just isn't feasible at present. I expect many other users are in the same boat. I think this module, or something like it, provides a good compromise, by allowing users to move more of their macOS system configuration into Nix while leveraging Homebrew Bundle under the hood.

Here's an example of this module in use in my current config:
https://github.com/malob/nixpkgs/blob/master/darwin/homebrew.nix

It's been working quite well in my testing so far, both on my system, and my GitHub workflow.

Suggestions for changes/improvements are of course welcome.

Implementation details

The options,

• homebrew.taps,
• homebrew.brews,
• homebrew.casks,
• homebrew.masApps,
• homebrew.whalebrews,

are used to generate a Brewfile in the Nix store which defines which packages/apps Homebrew Bundle will install/upgrade. (The homebrew.extraConfig option is also provided to allow users to include more complex Brewfile entries.)

The activation script runs brew bundle using the generated Brewfile, with the HOMEBREW_NO_AUTO_UPDATE environment variable set to ensure idempotence. The homebrew.autoUpdate option is provided for users who want to allow Homebrew to auto-update during activation.

The homebrew.cleanup option is provided to allow the user to enable Homebrew Bundle uninstall packages/apps that are currently installed but aren't present in the generated Brewfile. I made the default "none" so that new users of this module would be less likely to uninstall packages/apps accidentally.

More detailed documentation is provided in the module.

(Edit * 3: Updated to reflect changes since creating PR)

[malob]

Alrighty, this seems like it's in a good state for initial review. I'll add some review comments with thoughts/questions I have.

[malob]

See my review comments for specific thoughts/questions.

I expect we'll also want test for this module. However, I'm not sure what they should be since it's my understanding that activation scripts aren't run when running tests, and most of the things I'd want to test are that things were successfully installed etc.

If there are suggestions for what should be tested. I'm happy write the test up.

[malob]

I should note that I actually haven't tested Homebrew Bundles whalebrew integration, since I'm not currently using Docker on my macOS machine. I've based the implementation of this option on the Homebrew Bundle, and whalebrew documentation.

[LnL7]

I have not used homebrew on my system for quite some time so I don't really have much feedback here. I think it's best that the people that will actually use it give their suggestions, looks good to me in general if everybody is happy with it.

[TheOptimist]

This is a great answer to my comment, thank you. I realized much of this (especially the tying to brew bundle) after writing. Then saw the uselessness of my comment (not adding anything).

[malob]

@LnL7, just want to flag up front that I don't feel confident I understand exactly what you were getting at. Apologies if my responses below aren't on target.

As far as I can tell, Homebrew has no plans to use Brewfile.lock.json as a "proper" lockfile, due to fundamental limitations of how Homebrew works. From the PR that added the ability for Homebrew Bundle to output the file:

Starting with a caveat: we will never have a "proper" lockfile in Homebrew/homebrew-bundle that installs all your dependencies at exactly the versions you want because Homebrew doesn't work that way (and never will because e.g. pinning OpenSSL at the same version forever is a bad idea).

There could be some value in having access to the lockfile for the last successful invocation of brew bundle [install], since, if a subsequent invocation fails, one could have a look at the file in the process of debugging the failure. However, I don't currently see a good way to incorporate the lockfile into this module.

From brew bundle --help:

brew bundle will output a Brewfile.lock.json in the same directory as the Brewfile if all dependencies are installed successfully.

That poses a couple challenges:

• The Brewfile this module generates is located in the Nix store, and so during activation brew bundle [install] can't generate a lockfile when using the generated Brewfile, since it will try to write it to the Nix store, which it can't.
• Even if this wasn't the case, I don't see a good way to save the lockfile into the Nix store, given that it would be generated during activation.

This is kind of a bummer since, if brew bundle [install] fails during activation, there isn't a lockfile around to help with debugging, but unless I'm missing something, I don't see any good way for this module to track or work with the Brewfile.lock.json file.

The global HOMEBREW_* variables also mean that any other usecases, like a Brewfile in a repo won't work as expected, I'm not sure if this is a thing people do but keeping things contained to activation by default would be a better default I think. Your workflow can still be handled by exposing the brewfile in an option so it can be referenced in other modules

{ config, lib, pkgs, ... }:
{
    environment.variables.HOMEBREW_BUNDLE_FILE = config.homebrew.brewfile;
}

It does seem to me that setting HOMEBREW_BUNDLE_FILE and HOMEBREW_BUNDLE_NO_LOCK globally, could be controversial for the reasons you mentioned. My sense is that there are two primary ways people use Brewfiles, to manage the things installed on their system for their own day-to-day use, and included with a project to make it easy to install dependencies for that project (I'm not sure how common the latter is, but I've definitely seen it).

For the former case, I do think that having these variables set leads to a more intuitive experience, and that the illustrative examples are representative of a broader class of use-cases that I expect to be fairly common. I also agree that with these variables set globally, it may be confusing/cumbersome for folks to use a Brewfile provided with a project (e.g., they'd have to run brew bundle --file=Brewfile rather than just brew bundle when in the project root, and they'd have to unset HOMEBREW_BUNDLE_NO_LOCK if they wanted to generate a lockfile).

Given my understanding of your concerns, here are what seem to me like the best ways to move forward:

• Don't set the variables at all in this module, add a (readonly or private?) option homebrew.brewfile which gets set to the path of the Brewfile in the store, and add to the description of homebrew.enable that some users may want to use environment.variables to set the variables themselves.
• Add back in options like homebrew.global.brewfile and homebrew.global.noLock, so that there's a quick way using this module for users to choose what behavior they want when running brew bundle commands themselves. My inclination would be for these options to be enable by default, but I'd be happy to have them be disabled by default if you disagree.

I'd prefer the latter, but would settle for the former in order to get this merged.

[malob]

@LnL7, I've gone ahead and implemented the latter of the two paths forward I listed with the options disabled by default (to make things quicker if you're happy with it). Of course, if you'd really prefer the other option, or I completely misunderstood your concerns, just let me know, and I'll make the required changes.

[ethancedwards8]

I would love for this to get merged. I've recently needed to install a homebrew package and I would love a nix/declarative way to do it such as this.

[kclejeune]

I would love for this to get merged. I've recently needed to install a homebrew package and I would love a nix/declarative way to do it such as this.

If you use niv or flakes to pin dependencies, you can point it to the version from this fork until it gets merged.

[ethancedwards8]

I would love for this to get merged. I've recently needed to install a homebrew package and I would love a nix/declarative way to do it such as this.

If you use niv or flakes to pin dependencies, you can point it to the version from this fork until it gets merged.

I don't use flakes yet, that's on my TODO list :)

[malob]

In case it's helpful, my Nix configs repo is a flake and it has and output darwinModules that contains an up to date version of this module: https://github.com/malob/nixpkgs/blob/master/flake.nix

[LnL7]

@LnL7, I've gone ahead and implemented the latter of the two paths forward I listed with the options disabled by default (to make things quicker if you're happy with it). Of course, if you'd really prefer the other option, or I completely misunderstood your concerns, just let me know, and I'll make the required changes.

Mostly just wanted to make sure I got the idea across. I leave the decision up to the people that will use this.

[kclejeune]

@LnL7, I've gone ahead and implemented the latter of the two paths forward I listed with the options disabled by default (to make things quicker if you're happy with it). Of course, if you'd really prefer the other option, or I completely misunderstood your concerns, just let me know, and I'll make the required changes.

Mostly just wanted to make sure I got the idea across. I leave the decision up to the people that will use this.

I'm happy with this implementation - I think this looks ready to merge.

[ethancedwards8]

@LnL7, I've gone ahead and implemented the latter of the two paths forward I listed with the options disabled by default (to make things quicker if you're happy with it). Of course, if you'd really prefer the other option, or I completely misunderstood your concerns, just let me know, and I'll make the required changes.

Mostly just wanted to make sure I got the idea across. I leave the decision up to the people that will use this.

I'm happy with this implementation - I think this looks ready to merge.

I agree, I've forked this and changed my flake input to my fork. Everything's working as expected. One thing I think could improve is if homebrew isn't installed it should install it automatically.

[kclejeune]

I agree, I've forked this and changed my flake input to my fork. Everything's working as expected. One thing I think could improve is if homebrew isn't installed it should install it automatically.

We had a discussion about this and ultimately decided that's not a good idea for reasons cited here: #262 (comment). If memory serves, this will print a warning and do nothing if brew isn't found in the path.

[malob]

Yeah, during activation a (non-fatal) error message gets printed if Hombrew isn't installed:

nix-darwin/modules/homebrew.nix

Lines 206 to 211 in 2a0b9a9

	echo >&2 "Homebrew bundle..."
	if [ -f /usr/local/bin/brew ]; then
	PATH=/usr/local/bin:$PATH ${brew-bundle-command}
	else
	echo -e "\e[1;31merror: Homebrew is not installed, skipping...\e[0m" >&2
	fi

While I do agree it would be nice for this module to install Homebrew, this module is already pushing the boundries of the usual Nix zen, and given that I expect that installation of Homebrew would need to happen during activation, that would push things even further. Regardless, I think we should move forward with merging this initial version of the module, and leave potential additions like adding Homebrew installation to future discussions/PRs.

[ethancedwards8]

Yeah, during activation a (non-fatal) error message gets printed if Hombrew isn't installed:

nix-darwin/modules/homebrew.nix

Lines 206 to 211 in 2a0b9a9

	echo >&2 "Homebrew bundle..."
	if [ -f /usr/local/bin/brew ]; then
	PATH=/usr/local/bin:$PATH ${brew-bundle-command}
	else
	echo -e "\e[1;31merror: Homebrew is not installed, skipping...\e[0m" >&2
	fi

While I do agree it would be nice for this module to install Homebrew, this module is already pushing the boundries of the usual Nix zen, and given that I expect that installation of Homebrew would need to happen during activation, that would push things even further. Regardless, I think we should move forward with merging this initial version of the module, and leave potential additions like adding Homebrew installation to future discussions/PRs.

Okay cool yeah, I must've missed that comment.

[bphenriques]

👋 Wanted to provide feedback that it is working as intended, thank you! 😄 For now I am setting my inputs as follows:

    darwin.url = "github:malob/nix-darwin/brew-bundle"; # Replace with lnl7/nix-darwin once https://github.com/LnL7/nix-darwin/pull/262 is merged.

One less thing to manage via shell scripts 🙇 It is not deterministic due to the lack of ability to pin versions but I think it is a good compromise for MacOS users.

Edit: I found out a (good?) quirk where I can include different files with different sets of homebrew.casks and they are all installed as if Nix is merging all the lists (e.g., importing this and that results in all of them being installed) 🤔 This is more of a Nix question and is not specific to this forum.

[kclejeune]

👋 Wanted to provide feedback that it is working as intended, thank you! 😄 For now I am setting my inputs as follows:

    darwin.url = "github:malob/nix-darwin/brew-bundle"; # Replace with lnl7/nix-darwin once https://github.com/LnL7/nix-darwin/pull/262 is merged.

One less thing to manage via shell scripts 🙇 It is not deterministic due to the lack of ability to pin versions but I think it is a good compromise for MacOS users.

Edit: I found out a (good?) quirk where I can include different files with different sets of homebrew.casks and they are all installed as if Nix is merging all the lists (e.g., importing this and that results in all of them being installed) 🤔 This is more of a Nix question and is not specific to this forum.

This is expected behavior and is not actually unique to this module! Nix merges lists and attribute sets if they're defined in more than one location. This is helpful to design modular configurations :)

[LnL7]

There hasn't really been an new feedback so merging.

[frontsideair]

Yeah, during activation a (non-fatal) error message gets printed if Hombrew isn't installed:

nix-darwin/modules/homebrew.nix

Lines 206 to 211 in 2a0b9a9

	echo >&2 "Homebrew bundle..."
	if [ -f /usr/local/bin/brew ]; then
	PATH=/usr/local/bin:$PATH ${brew-bundle-command}
	else
	echo -e "\e[1;31merror: Homebrew is not installed, skipping...\e[0m" >&2
	fi

While I do agree it would be nice for this module to install Homebrew, this module is already pushing the boundries of the usual Nix zen, and given that I expect that installation of Homebrew would need to happen during activation, that would push things even further. Regardless, I think we should move forward with merging this initial version of the module, and leave potential additions like adding Homebrew installation to future discussions/PRs.

Are there any developments on this front? I would really like nix to transparently use homebrew instead of me, as a user, installing it beforehand.

[malob]

I'm not aware of any work on this front. I'd be happy to give feedback on a PR that implemented Homebrew installation (though it's not my call on whether it gets merged).