Rewrite in Swift

[thesoftwarephilosopher]

This is related to #5080 (become separate from Homebrew) and #390 (use JSON instead of Ruby).

Rationale

• Without being attached to Homebrew, there's no infrastructural need for Ruby
• We can load JSON files from Swift trivially

Benefits

• The benefits of Separate the Ruby backend from internal Homebrew dependencies #5080 and Change casks to use a data format, not code #390 are inherited
• With JSON files, cask would be faster
• In Swift, cask would be faster
• Third-parties could read cask's data files
• Could offer opportunities to greatly simplify development of the codebase

In particular, I wanted to make a GUI wrapper for cask, that would make it much more convenient to use cask. (Not sure if there's an open issue for this; too many use the word "gui" for a quick search to be helpful). But when I last tried this, it was too slow, due to reading configuration files by executing lots of Ruby code.

Downsides

• Conversion of existing casks would be time-consuming
• Existing users would have incompatible local cask installations
• Would require 10.9 if the Swift language is chosen

Proposal variation

Swift is a suggestion, for a few reasons: it will soon be built-in, it's very fast, it's generally type-safe and memory-safe, and it's purportedly convenient to write programs in.

But despite the title of this issue, Swift isn't inherent to this issue, it's just a suggestion. Mainly I was trying to avoid this issue degrading into another pointless language war and never being resolved.

There are other languages that have some or all of these qualifications, such as Go. But I suggested Swift because it seems to have more if these good qualities than other options.

But if Swift is out of the question, I'd suggest Python, since a reasonably high version is built into the system already and has been for quite a few versions of OS X.

[rolandwalker]

@sdegutis several of the maintainers here are fans of your work and use your superb tools daily; it is unlikely you would get the WONTFIX brushoff as was mentioned in IRC. If you were unable to build a tool on top of homebrew-cask for performance reasons, we should address that, one way or the other. Being hackable has always been one of the project goals here.

Performance

And performance is important for other reasons. We are now well over 2,500 Casks across all of our repos, and growing fast. Now is the time to prepare for 10,000 Casks. If we wait until we get there, it will be too late. So, we have been working on perf a little bit lately.

In a recent analysis of large-scale operations such as brew cask list, we found that the biggest performance hit came from unnecessary stats rather than object instantiation. The fact that instantiation wasn't the biggest drag was a surprise to me, and the speedup in #4434 was substantial.

If you go back even farther, there were some inefficient uses of conditionals within Casks. As the project became larger, the lag caused by those constructs became noticeable, and those various individual Casks were amended.

There is also an unstated performance agenda behind the syntax change in #5365. After the DSL transition period, the intention is to drop all the per-Cask Ruby classes and just have Cask instances. Furthermore, on first read of a Cask file, we would only load the name, check the DSL version, and swallow the rest as a single do block. We would then defer evaluating that block until/unless actually needed.

It's possible this would all be faster in Swift; I don't know enough to argue the point and am not really inclined to disagree anyway. I just see that your concerns are valid, and that we haven't yet hit the wall on what we can do for perf with Ruby.

Hackability and Interop

Having our own dynamic DSL places limits on interop with third parties. That's a fact. If interop became our overriding concern, then yes, we would simply have to change. And perhaps there are already missed opportunities in that space, as we would like to interop with mackup (#1135) or AppCleaner (#1005) but so far that has seemed too difficult to get rolling.

However, in the interest of hackability, we have recently been adding some undocumented command verbs such as brew cask _stanza (#5025) and brew cask _dump, which dumps a Cask in YAML (#5011) We would be happy to add JSON on request, and settle on a spec, though obviously that would never be as performant as static JSON.

We also added an interface for external commands in #2594, but that has not seen much uptake.

Team and Culture

Most of our core maintainers are Rubyists, and as it happens I joined the project for the sole purpose of learning Ruby. Then again, most of us would probably enjoy learning Swift, so I'm not certain that's an absolute barrier.

However (and once again this is not exactly intended as an argument, just a relevant issue) the human factor is an important element of engineering. The design that @phinze laid out was very straightforward for many people, and this project went on to attract the input of 749 contributors to date. Now seems like a good moment to revisit #2925 (current results below). Our project has an unusual degree of dependence on casual contributors. "First-time Contributor" is the project leader, and the sine qua non.

$ ./developer/bin/the_long_tail
Commits Contributors
---------------------
1      435  ....................................................................................
2      148  ............................
3       82  ...............
4       54  ..........
5       44  ........
6       28  .....
7       15  ..
8       12  ..
9       15  ..
10's    52  ..........
100's    9  .
1000's   0

85% of contributors are "occasional" (with <= 5 commits)

48% of contributors commit only once

65% - 75% of Casks depend on an occasional contributor

89% - 100% of Casks depend on a contributor who is not a maintainer

[thesoftwarephilosopher]

@rolandwalker

The reason I expected a 'wontfix' is because my proposal is basically saying "hey why don't you rewrite this whole project from scratch?" which is usually a pretty unreasonable suggestion. But I suggested it anyway, knowing that I personally have had much success with this technique.

Performance

You're right, performance can indeed be tweaked to be reasonable using Ruby, or most languages actually. Performance wasn't my main reason behind opening this issue.

Hackability and Interop

It's my own personal preference to lean towards easier analysis from outside tools, thus my suggestion for static configuration files. This is probably the second highest benefit of my proposal, in my opinion. But opinions never have to be agreed upon.

Team and Culture

Learning a new programming language is great, no matter which one it is. I'm glad to hear Cask is providing an opportunity for this. A little disappointed that it's Ruby, but not as much as I thought I'd be.

Ramblings

There's a whole lot I don't know about package management. But I do know that 99% of problems projects have are created by their own design decisions. That's why most of my projects failed and are in the dustbin as we speak.

I opened this issue mostly for the sake of completeness, to just throw the idea out there of greatly simplifying the configuration file format of Cask formulae (and, if I know my Ruby, probably Cask's implementation too). Because I know that simpler usually means easier to fix and add to and extend and change.

But I'm keenly aware that it would involve a massive rewrite that would be quite tedious and unreasonable to ask of anyone, and would break compatibility and basically break everything for everyone. That's why I didn't expect it to be given this kind of attention.

Anyway, thanks for your time.

[passcod]

10,000 Casks? Too bad the C10k abbreviation is already taken! ;-)

Some notes:

• Last I looked, Python's interpreter's slower than Ruby's.
• Changing the language and changing the cask format are two separate issues: switching to JSON while keeping Ruby as the language would still provide a speed boost, for example. Change casks to use a data format, not code #390 should be reopened and discussion on the matter of code vs data should happen there, I think. Historical info: at the time I actually spent the day thinking it through. In the conclusion you can see me rejecting it because "phinze started it that way", which is a weak argument. I actually thought it would be neat to have a static, fast, standard format, but the context (youth, rapid changes and strong Homebrew ties) was incompatible. Now that the project is much more mature and quite evolved in its nature, I am quite happy to see this issue discussed again!
• Swift would probably be somewhat annoying for interop as it's Mac-native and proprietary (doesn't run anywhere else, smaller community, etc).
• Swift and other compiled languages would require binaries to be pre-built and distributed (or a Homebrew formula to be built from source).
• JSON isn't the only option (and probably shouldn't even be an option, given its limitedness). YAML, TOML and plain text (see the foo.cask example in Change casks to use a data format, not code #390) come to mind.
• Another option would be to actually parse (instead of evaling) the cask DSL instead of converting to another syntax. Using the full Ruby parser is really slow (compared to eval), but writing a smaller grammar shouldn't be too hard.

[thesoftwarephilosopher]

@passcod

1. Yeah Python isn't too great either.
2. They are separate, and honestly I prefer Change casks to use a data format, not code #390 over this issue, but changing to Swift necessitates not using a DSL since it doesn't have eval, hence I brought Change casks to use a data format, not code #390 back up.
3. I don't see how Swift would be more annoying for interop. It's not like I plan to dynamically evaluate stuff inside Cask. By interop, I just mean reading its configuration files and maybe sometimes shelling out to cask info bla to see if it's installed, etc. Although honestly it would be awesome if all cask's info was stored in parsable static files (i.e. JSON, YAML, etc), so that I would never have to shell out to it.
4. Yeah, it'd need to distribute binaries. Not ideal, agreed.
5. YAML is nice too. I just suggested JSON because Swift (via Foundation.framework) has a native JSON parser already. But honestly I don't care, I just mean something that other languages can read without using a programming language lexer or interpreter.
6. Having a configuration file that looks like Ruby but isn't really Ruby is a recipe for disaster! With Clojure this might make more sense, since that'd basically be EDN, but Ruby is quite a different beast.

[thesoftwarephilosopher]

So, sticking with Ruby is obviously the resolution to this specific issue. But I hope #390 can be reopened.