explore fixing #5187 via lispy bitops

[Sacha0]

The crux of #5187: & and | conflate logical and bitwise operations. The natural precedence of & and | for logical operations is low, whereas that for bitwise operations is high. This tension forces an unsatisfying precedence compromise.

One approach to resolving this tension is to syntactically separate logical and bitwise operations, for example by giving the latter names such as bitor, bitand, bitnot, and bitxor. Having additional upsides (see below), this approach at one point or another received support from (e.g.) multiple(1) core(2) contributors(3) on github(4), slack, and triage (though a recent triage was less enthusiastic).

So I gave this approach a shot. This branch (somewhat outdated, incomplete, and incorrect where things BitArray are concerned so please ignore those changes) is the result of that experiment. A few unexpected findings:

1. I was surprised by how frequently I could not tell from (local) context whether &/| were intended to be bitwise or logical.

2. I was also surprised by how infrequently I had to add parentheses to (compound) bit twiddling expressions when rewriting with bitor/bitand/bitnot/bitxor: In such compound bit twiddling expressions, defensive parens seemed the rule rather than the exception.

3. I was also surprised that, despite having the precedence table open (or by some point memorized) and knowing the associativity rules, I occasionally had to resort to checking the result of certain compound bit twiddling expressions at the repl to figure out precisely what they were doing.

The second and third observations perhaps support the notion that something is amiss with precedence as it stands (#5187), or perhaps support the notion that relying upon precedence and associativity in bit twiddling expressions is suboptimal in any case (too difficult to remember, too difficult to mentally parse, or what have you).

What are the tradeoffs of this approach?

Arguments in favor:

With bitor/bitand/bitnot/bitxor, you never need the precedence table and/or associativity rules on hand (or memorized) to grok a bit twiddling expression, nor need you interrogate particularly complex expressions at the repl.

bitor/bitand/bitnot/bitxor are semantically clear independent of which language(s) you came from, whereas expectations regarding the semantics, precedence, and associativity of &/|/~ vary quite a bit depending on which language(s) you came from. In other words, bitor/bitand/bitnot/bitxor are more broadly accessible, understandable, and unambiguous.

With bitor/bitand/bitnot/bitxor, whether an expression is logical or bitwise is always evident locally.

Moving to bitnot immediately frees the precious ASCII territory ~. Uses for free ASCII appear in every discussion of feature introduction or extension (think views, partial evaluation and currying, traits/protocols, missingness, getfield overloading, things namespacing related, piping/chaining, et cetera).

Arguments against:

To those of us who came from C(/-influenced languages), &/|/~(/^) feel natural for bitwise operations / bit twiddling.

Expressions in bitor/bitand/bitnot/bitxor are more verbose and less cute than in &/|/~.

Short op= expressions are particularly cute, and e.g. bitor= would be a bit less cute.

The advantages of bitor/bitand/bitnot/bitxor hold primarily in compound / complex bit twiddling expressions. In short such expressions, the downsides of &/|/~ are weaker or absent and the upsides stronger, and one could argue that any expression where the downsides of &/|/~ become significant should simply be refactored.

Other considerations worth bearing in mind:

Though a good fraction of those of us active on github came from C and do / have done a nontrivial amount of bit twiddling, chances are we represent a minority in the broader Julia userbase (and likely will be a shrinking minority with future uptake). Naturalness/intuitiveness for us may not be a good predictor of naturalness/intuitiveness for the broader userbase, perhaps to the contrary.

Typing speed is rarely if ever a productivity rate limiter when bit twiddling, whereas existing-code-grokking speed frequently is.

If this approach gains support, I should be able to polish this branch off in a few hours. If not, cheers :). Thoughts? Thanks all!

[quinnj]

Excellent research/findings as usual @Sacha0! I find the point about "precedence confusion" to be particularly compelling, so I'm in favor the bit* family of functions (even as someone who recently implemented a whole feature around some bit-twiddling at the core).

[ararslan]

I'm in favor of fixing the precedence of & and | to something more sensible but I'm not in favor of removing them as bitwise "and" and "or." I've always found the fact that Julia lets you do these kind of low-level operations in a convenient, concise, and familiar manner even though it's a high level language to be one of the big draws for Julia as a "one-size-fits-all" kind of language. Having to do bitand(a, b) feels very second-class, plus a = bitand(a, b) is significantly less ergonomic than a &= b.

[KristofferC]

I'm not sure how a non infix function can be called "second class". Is sin second class?

[ararslan]

sin is standard, as is &. It has nothing to do with being infix.

[StefanKarpinski]

I'm 100% with @ararslan here.

[Sacha0]

Having slept on this, I came up with another approach with a better set of tradeoffs. I will post a pull request exploring that approach (hopefully) later today. Thanks all! :)