Create "binarydir" (/usr/local by default) automatically & check permissions/ownership

[radeksimko]

Homebrew can be installed in different locations, although it may not happen very often and I'd personally discourage people from doing so, it happens in boxen.

This causes issues for Cask, because it expects existence of binarydir and existence of the parent dir - /usr/local which doesn't exist by default in a fresh OS X installation.

Debug: Executing 'brew cask install sourcetree'
Error: Execution of 'brew cask install sourcetree' returned 1: ==> Downloading https://downloads.atlassian.com/software/sourcetree/SourceTree_2.0.3.dmg
==> Symlinking App 'SourceTree.app' to '/Users/vagrant/Applications/SourceTree.app'
Error: Permission denied - /usr/local
==> Symlinking Binary 'stree' to '/usr/local/bin/stree'

It seems that there were only some historical reasons behind the exotic homebrew location and it should be fixed, but I think that Cask should not rely on the default location anyway.

I would send a fix that will check existence of default binarydir + tries mkdir -p before even trying to symlink casks into it (either with any cask installation or on homebrew-cask installation), if you agree?

[vitorgalvao]

Homebrew-cask isn’t using /usr/local because it’s homebrew’s default location. In fact, you should be able to find issues where users who use a non-standard location ask for homebrew-cask to install cli tools where their homebrew installation does. The conclusion in these has always been that such feature won’t be implemented, since we install to /usr/local because it is the appropriate system directory, and some apps we have no control over (pkg-based) also treat it like the appropriate system directory for those tools.

Now, I’m not disagreeing with your proposal. We should check for the existence of the directory (and create it) before trying to do anything to it, naturally. The previous paragraph is solely to explain that such directory is not what it is due to homebrew at all.

[rolandwalker]

@radeksimko yes. I noted the need to mkdir -p /usr/local in #7481 but then accidentally closed the issue, which I meant to keep open as a reminder.

[DomT4]

I wonder if the Cask could grab the binarydir dynamically from Homebrew at install time and then check periodically afterwards. Something like:

binarydir = $(shell brew --prefix)+"bin"

[vitorgalvao]

@DomT4 No. See my comment and follow @rolandwalker’s link, to multiple discussions where it was decided that was not the best course of action.

[DomT4]

@vitorgalvao Apologies, Roland's comment wasn't there when I hit the comment button, and I was scrolled far enough down the page that I didn't see yours'. You could always insert a simple check for /usr/local in that case, either in shell or something like

path = "/usr/local"
assert !path.exist?

[rolandwalker]

Yes, in my interface all three of our comments appeared at the same time.

[radeksimko]

Homebrew-cask isn’t using /urs/local because it’s homebrew’s default location. In fact, you should be able to find issues where users who use a non-standard location ask for homebrew-cask to install cli tools where their homebrew installation does.

ok, good to know, thanks for explanation.

The conclusion in these has always been that we install to /usr/local because it is the appropriate system directory, and some apps we have no control over (pkg-based) also treat it like the appropriate system directory for those tools.

Agreed, but I assume that even that the other (pkg-based) apps treat it like the appropriate system dir, I assume they create it if it doesn't exist.

I will send the PR then, but an important question needs to be answered - should we be checking it with each cask installation & upgrade or only while installing homebrew-cask itself?

While checking on each (related = with binary stanza) cask:

Pros:

• prepared for changes even after installation (which won't happen in most cases)

Cons:

• a little performance overhead, but running a simple stat shouldn't be too costly

[vitorgalvao]

Ideally we should be checking it every time a cask with binary is installed. It may run a few more times, but it shouldn’t make it that much slow, and it not only prevents in cases the directory might’ve been deleted or changed (via the --binarydir option, for example), it also makes it so it is only created if is is really needed (i.e., if a user never installs a cask with a binary, why should we be creating this directory?).

[radeksimko]

@vitorgalvao Good point, 100% agree.

[rolandwalker]

Great. Just so the reference is here, rather than buried in other links: we use /usr/local because it is defined in the FHS as the appropriate default for installing Unixy software. We have no problem with people choosing a different directory, but this is the right choice for a default.

Incidentally, /opt/local is a common hack adopted to avoid having too much collide in /usr/local. But that hack is contra-FHS, which reserves /opt to be organized by application: /opt/appname/…. Our use of /opt/homebrew-cask/ for the staging area follows the standard.

(Also: Wikipedia FHS)

[radeksimko]

I modified title to address occasional issues with permissions.
See boxen/puppet-brewcask#9

I think it is most likely package-dependent issue as brewcask itself does not touch /usr/local unless it's told to do by any particular cask.

There was a naughty cask in the past that has been changing permissions of the mentioned directory.

What we may eventually do are some checks on permissions before touching any path containing /usr/local, but I'm not entirely sure whether we should try to fix it nor what is the "correct" state as that directory does not exist on OSX by default. I know that homebrew does some checks on permissions in this directory though.

[radeksimko]

@salimane could you please provide more details which may help us to replicate the issue?

Output from these commands may help:

brew cask list

stat -f '%Sp %Su %Sg' /usr/local

stat -f '%Sp %Su %Sg' /usr/local/bin

stat -f '%Sp %Su %Sg' /usr/local/bin/github

Providing a cask name that you installed as a last one before trying to install github would also help a lot.

[salimane]

@radeksimko here you go :)

salimane at Salimanes-MacBook-Pro (git:salimane.com)  in /opt/boxen/repo
± brew cask list                                                                      
adium         clipmenu      hipchat       lastpass      osxfuse       skype
appcleaner    dash          imageoptim        libreoffice   pgadmin3      spectacle
archiver      dropbox       imgur         licecap       pivotalbooster    spotify
arduino       gimp          intellij-idea-ce  magican       qq        torbrowser
caffeine      google-hangouts   invisionsync      nmap      quicksilver   vlc
calibre       google-notifier   istat-menus       omnigraffle   rubymine
ccleaner      gpgtools      iterm2        onyx      sequel-pro
cinch         heroku-toolbelt   jenkins-menu      opera     skitch

salimane at Salimanes-MacBook-Pro (git:salimane.com)  in /opt/boxen/repo
± stat -f '%Sp %Su %Sg' /usr/local                                                    
drwxr-xr-x root wheel

salimane at Salimanes-MacBook-Pro (git:salimane.com)  in /opt/boxen/repo
± stat -f '%Sp %Su %Sg' /usr/local/bin                                                 
drwxr-xr-x root wheel

salimane at Salimanes-MacBook-Pro (git:salimane.com)  in /opt/boxen/repo
± stat -f '%Sp %Su %Sg' /usr/local/bin/github                                          
stat: /usr/local/bin/github: stat: No such file or directory

salimane at Salimanes-MacBook-Pro (git:salimane.com)  in /opt/boxen/repo
± ./script/boxen                                                                       
Boxen's master branch is out of sync, won't auto-update!
====> Installing gem dependencies
Notice: Compiled catalog for salimanes-macbook-pro.local in environment production in 4.72 seconds
[1;31mError: Execution of 'brew cask install diffmerge --appdir=/Applications' returned 1: ==> Caveats
Use "diffmerge --nosplash" to hide the splash screen when using
diffmerge with external tools such as git.

==> Downloading http://download-us.sourcegear.com/DiffMerge/4.2.0/DiffMerge.4.2.0.697.intel.stable.dmg
Already downloaded: /Library/Caches/Homebrew/diffmerge-4.2.0.697.dmg
==> It seems there is already an App at '/Applications/DiffMerge.app'; not linking.
==> Symlinking Binary 'DiffMerge' to '/usr/local/bin/diffmerge'
Error: Command failed to execute!

==> Failed command:
["/bin/ln", "-hfs", "--", "#<Pathname:/opt/homebrew-cask/Caskroom/diffmerge/4.2.0.697/DiffMerge.app/Contents/MacOS/DiffMerge>", "#<Pathname:/usr/local/bin/diffmerge>"]

==> Output of failed command:


==> Exit status of failed command:
#<Process::Status: pid 99942 exit 1>
==> ln: /usr/local/bin/diffmerge: Permission denied[0m
[1;31mError: /Stage[main]/Boxen::Personal/Package[diffmerge]/ensure: change from absent to present failed: Execution of 'brew cask install diffmerge --appdir=/Applications' returned 1: ==> Caveats
Use "diffmerge --nosplash" to hide the splash screen when using
diffmerge with external tools such as git.

==> Downloading http://download-us.sourcegear.com/DiffMerge/4.2.0/DiffMerge.4.2.0.697.intel.stable.dmg
Already downloaded: /Library/Caches/Homebrew/diffmerge-4.2.0.697.dmg
==> It seems there is already an App at '/Applications/DiffMerge.app'; not linking.
==> Symlinking Binary 'DiffMerge' to '/usr/local/bin/diffmerge'
Error: Command failed to execute!

==> Failed command:
["/bin/ln", "-hfs", "--", "#<Pathname:/opt/homebrew-cask/Caskroom/diffmerge/4.2.0.697/DiffMerge.app/Contents/MacOS/DiffMerge>", "#<Pathname:/usr/local/bin/diffmerge>"]

==> Output of failed command:


==> Exit status of failed command:
#<Process::Status: pid 99942 exit 1>
==> ln: /usr/local/bin/diffmerge: Permission denied[0m
[1;31mError: Execution of 'brew cask install smartgit --appdir=/Applications' returned 1: ==> Downloading http://www.syntevo.com/download/smartgit/smartgit-macosx-6_5_2.dmg
Already downloaded: /Library/Caches/Homebrew/smartgit-6.5.2.dmg
==> It seems there is already an App at '/Applications/SmartGit.app'; not linking.
==> Symlinking Binary 'SmartGit' to '/usr/local/bin/SmartGit'
Error: Command failed to execute!

==> Failed command:
["/bin/ln", "-hfs", "--", "#<Pathname:/opt/homebrew-cask/Caskroom/smartgit/6.5.2/SmartGit.app/Contents/MacOS/SmartGit>", "#<Pathname:/usr/local/bin/SmartGit>"]

==> Output of failed command:


==> Exit status of failed command:
#<Process::Status: pid 99988 exit 1>
==> ln: /usr/local/bin/SmartGit: Permission denied[0m
[1;31mError: /Stage[main]/Boxen::Personal/Package[smartgit]/ensure: change from absent to present failed: Execution of 'brew cask install smartgit --appdir=/Applications' returned 1: ==> Downloading http://www.syntevo.com/download/smartgit/smartgit-macosx-6_5_2.dmg
Already downloaded: /Library/Caches/Homebrew/smartgit-6.5.2.dmg
==> It seems there is already an App at '/Applications/SmartGit.app'; not linking.
==> Symlinking Binary 'SmartGit' to '/usr/local/bin/SmartGit'
Error: Command failed to execute!

==> Failed command:
["/bin/ln", "-hfs", "--", "#<Pathname:/opt/homebrew-cask/Caskroom/smartgit/6.5.2/SmartGit.app/Contents/MacOS/SmartGit>", "#<Pathname:/usr/local/bin/SmartGit>"]

==> Output of failed command:


==> Exit status of failed command:
#<Process::Status: pid 99988 exit 1>
==> ln: /usr/local/bin/SmartGit: Permission denied[0m
[1;31mError: Execution of 'brew cask install oclint --appdir=/Applications' returned 1: ==> Downloading http://archives.oclint.org/releases/0.7/oclint-0.7-x86_64-apple-darwin-10.tar.gz
Already downloaded: /Library/Caches/Homebrew/oclint-0.7.tar.gz
==> Symlinking Binary 'oclint' to '/usr/local/bin/oclint'
Error: Command failed to execute!

==> Failed command:
["/bin/ln", "-hfs", "--", "#<Pathname:/opt/homebrew-cask/Caskroom/oclint/0.7/oclint-0.7-x86_64-apple-darwin-10/bin/oclint>", "#<Pathname:/usr/local/bin/oclint>"]

==> Output of failed command:


==> Exit status of failed command:
#<Process::Status: pid 120 exit 1>
==> ln: /usr/local/bin/oclint: Permission denied[0m
[1;31mError: /Stage[main]/Boxen::Personal/Package[oclint]/ensure: change from absent to present failed: Execution of 'brew cask install oclint --appdir=/Applications' returned 1: ==> Downloading http://archives.oclint.org/releases/0.7/oclint-0.7-x86_64-apple-darwin-10.tar.gz
Already downloaded: /Library/Caches/Homebrew/oclint-0.7.tar.gz
==> Symlinking Binary 'oclint' to '/usr/local/bin/oclint'
Error: Command failed to execute!

==> Failed command:
["/bin/ln", "-hfs", "--", "#<Pathname:/opt/homebrew-cask/Caskroom/oclint/0.7/oclint-0.7-x86_64-apple-darwin-10/bin/oclint>", "#<Pathname:/usr/local/bin/oclint>"]

==> Output of failed command:


==> Exit status of failed command:
#<Process::Status: pid 120 exit 1>
==> ln: /usr/local/bin/oclint: Permission denied[0m
[1;31mError: Execution of 'brew cask install packer --appdir=/Applications' returned 1: ==> Downloading https://dl.bintray.com/mitchellh/packer/packer_0.7.5_darwin_amd64.zip
Already downloaded: /Library/Caches/Homebrew/packer-0.7.5.zip
==> Symlinking Binary 'packer' to '/usr/local/bin/packer'
Error: Command failed to execute!

==> Failed command:
["/bin/ln", "-hfs", "--", "#<Pathname:/opt/homebrew-cask/Caskroom/packer/0.7.5/packer>", "#<Pathname:/usr/local/bin/packer>"]

==> Output of failed command:


==> Exit status of failed command:
#<Process::Status: pid 155 exit 1>
==> ln: /usr/local/bin/packer: Permission denied[0m
[1;31mError: /Stage[main]/Boxen::Personal/Package[packer]/ensure: change from absent to present failed: Execution of 'brew cask install packer --appdir=/Applications' returned 1: ==> Downloading https://dl.bintray.com/mitchellh/packer/packer_0.7.5_darwin_amd64.zip
Already downloaded: /Library/Caches/Homebrew/packer-0.7.5.zip
==> Symlinking Binary 'packer' to '/usr/local/bin/packer'
Error: Command failed to execute!

==> Failed command:
["/bin/ln", "-hfs", "--", "#<Pathname:/opt/homebrew-cask/Caskroom/packer/0.7.5/packer>", "#<Pathname:/usr/local/bin/packer>"]

==> Output of failed command:


==> Exit status of failed command:
#<Process::Status: pid 155 exit 1>
==> ln: /usr/local/bin/packer: Permission denied[0m
[1;31mError: Execution of 'brew cask install github --appdir=/Applications' returned 1: ==> Downloading https://central.github.com/mac/latest
Already downloaded: /Library/Caches/Homebrew/github-latest
==> It seems there is already an App at '/Applications/GitHub.app'; not linking.
==> Symlinking Binary 'github_cli' to '/usr/local/bin/github'
Error: Command failed to execute!

==> Failed command:
["/bin/ln", "-hfs", "--", "#<Pathname:/opt/homebrew-cask/Caskroom/github/latest/GitHub.app/Contents/MacOS/github_cli>", "#<Pathname:/usr/local/bin/github>"]

==> Output of failed command:


==> Exit status of failed command:
#<Process::Status: pid 190 exit 1>
==> ln: /usr/local/bin/github: Permission denied[0m
[1;31mError: /Stage[main]/Boxen::Personal/Package[github]/ensure: change from absent to present failed: Execution of 'brew cask install github --appdir=/Applications' returned 1: ==> Downloading https://central.github.com/mac/latest
Already downloaded: /Library/Caches/Homebrew/github-latest
==> It seems there is already an App at '/Applications/GitHub.app'; not linking.
==> Symlinking Binary 'github_cli' to '/usr/local/bin/github'
Error: Command failed to execute!

==> Failed command:
["/bin/ln", "-hfs", "--", "#<Pathname:/opt/homebrew-cask/Caskroom/github/latest/GitHub.app/Contents/MacOS/github_cli>", "#<Pathname:/usr/local/bin/github>"]

==> Output of failed command:


==> Exit status of failed command:
#<Process::Status: pid 190 exit 1>
==> ln: /usr/local/bin/github: Permission denied[0m

[radeksimko]

@salimane Great, thanks for that.

I'm curious how did your /usr/local got drwxr-xr-x root wheel... When you normally first install Homebrew, it creates this directory with drwxrwxr-x root admin, so the group name & group write permission are most likely the problem.

Nonetheless when you install it via boxen to an alternative location, it doesn't get created and something somehow at some point must create it, so the question is what and when.

I will dig into the provided list of casks you have installed using pkgutil --bom and lsbom. Hopefully I will find the bad guy.

We could eventually run this check on all casks and do it automatically either with each installation or at least as part of the automated PR tests for each new cask.

[radeksimko]

So it turned out to be more of a permission problem: #9225 I will run a few more tests on my VMWare/boxen setup to double check I'm right and then probably move onto testing/fixing the permissions problem.

[salimane]

@radeksimko thanks

[radeksimko]

@salimane Just FYI, I tested the whole provided list of casks above - installed them, launched each and then uninstalled. Both /usr/local and /usr/local/bin still have the same (correct) permissions so this conflict was either caused by something you installed outside of brewcask or by manually using/updating any app on that list or by boxen itself or any of its modules.

Either way I think, that brewcask could run stat on these directories each time user installs/uninstalls any cask and if the process changes anything, encourage the user to report such cask here. This way we can at least make sure that it's never caused by (un)installation process.

btw. the "correct" permissions given after installing homebrew & cask are following:

drwxrwxr-x root admin /usr/local
drwxr-xr-x %USER% admin /usr/local/bin

[halo]

Just wanted to briefly mention how I experienced this on a fresh installation of El Capitan with a fresh homebrew installed in a custom location (in my home dir).

The first thing I tried was to install the VLC cask, which apparently has a binary it would like to place into /usr/local/bin which, as mentioned before, does not exist.

So it just plainly fails. I might be missing something, but had expected that the (apparently optional) executable is simply skipped. Is that not an option? Instead of trying to create that directory.

I also wanted to point out Apple's SIP which states about the /usr directory:

System Integrity Protection is designed to allow modifications of these protected parts only by processes that are signed by Apple...

Not sure whether that affects the desired implementation, but I thought I just mention it.

This is the output, just in case.

$ HOMEBREW_CASK_OPTS="--caskroom=/Users/me/caskroom --appdir=/Users/me/Applications" \ 
  /Users/me/homebrew/bin/brew cask install vlc

Downloading https://get.videolan.org/vlc/2.2.3/macosx/vlc-2.2.3.dmg
  ==> Verifying checksum for Cask vlc
  ==> Symlinking App 'VLC.app' to '/Users/me/Applications/VLC.app'
  ==> Symlinking Binary 'vlcwrapper' to '/usr/local/bin/vlc'

Error: Permission denied - /usr/local/bin
Most likely, this means you have an outdated version of homebrew-cask... (etc...)

$ brew cask list
Warning: nothing to list

$ stat -f '%Sp %Su %Sg' /usr/local
drwxr-xr-x root wheel

$ stat -f '%Sp %Su %Sg' /usr/local/bin
stat: /usr/local/bin: stat: No such file or directory

[vitorgalvao]

Since we’ll be merging with Homebrew, we should actually close this and just read whatever directory homebrew uses. Will open a discussion.