consolidate ~ and !

[Sacha0]

& represents both logical and bitwise and. Similarly, | represents both logical and bitwise or. In contrast, ! and ~ separately represent logical and bitwise not. Triage liked the idea of consolidating logical and bitwise not into !, freeing the precious ASCII territory that is ~ for other uses. This pull request implements that consolidation, deprecating ~ to ! for bitwise not. Best!

[JeffBezanson]

This should be avoided.

[Sacha0]

Good catch! Fixed. Thanks! :)

[Sacha0]

(The AV i686 and Travis i686 test failures appear unrelated.)

[garrison]

People coming from other languages (e.g. Javascript or C++) will often expect !!x to convert x to a Bool. (I think this change has potential to be polarizing.)

EDIT: after thinking more, I no longer think this is such a big deal. Something to consider, though.

[Sacha0]

Rebased out conflict. Marking triage for discussion. Best!

[StefanKarpinski]

I've decided I like it.

[nalimilan]

I really hate the conflation of logical and bitwise operators...

Can we claim ~ for model formulas in exchange? :-p

[JeffBezanson]

I really hate the conflation of logical and bitwise operators...

Why? They're the same thing if a Bool is considered one bit. Is it just to help catch bugs?

[nalimilan]

Is it just to help catch bugs?

Yes. Also, I think most scientists have never heard about bitwise operations, so seeing things like !1 === -2 will just make them think Julia is weird or too complex for them.

[StefanKarpinski]

We are already very strict about boolean context only accepting true or false, I don't really see what is gained by ! also refusing any argument but booleans. Presumably if you use that in a context where only a boolean is ok, then you'll still get an error.

I'm not sure that anyone who is fine with ~1 == -2 currently is going to be that thrown by !1 == -2. I feel like the audience of people for whom the former made any sense in the first place is mostly people who will be fine with the latter if you just explain that ! is bitwise not in Julia.

[Sacha0]

Rebased out conflicts. Given the perhaps surprising uniformity of support this pull request has received and last week's triage's favor, perhaps triaging again is unnecessary? Best!

[JeffBezanson]

I'm mostly in favor, but I think the strongest objection would be the history of ! meaning "iszero". That makes mapping non-zero values to other non-zero values surprising.

[JeffBezanson]

Triage is largely in favor. @KristofferC want to make a final plea?

[StefanKarpinski]

Everyone on the triage call is in favor of this, but we wanted to give the dissenters to give some input. The general feeling is:

• Coming from C it's really weird to use !x for bitwise not of x.
• It feels incredibly wasteful to have two ASCII operators that mean almost the same thing.

[ararslan]

I kind of like the idea of replacing ~1 with something like flipbits(1). My background isn't even in C and find !1 to be pretty odd.

As another data point, we use flipbits! for BitArrays, so flipbits for scalars (and copying for BitArrays) seems nice and consistent.

[iamed2]

I think the biggest risk is with people doing something like !0 == true and expecting that to evaluate to true because they're used to a coercing ! operator. I can't say if that will actually trip people up in practice though (I have a hunch it won't).

[ararslan]

I've created #25514 to explore a replacement of ~ with flipbits.

[simonbyrne]

Since we've appropriated ⊻/\xor we could also use ¬/\neg?

[garrison]

!x means x != 0 in C, but in C-like languages you can also use x for !iszero(x) in a Boolean context. [...] Since we are strict about the latter, it makes no sense to me to use !x for iszero(x).

I think many people against this change will agree even with this statement. As I understand, the question is whether !x for non-Boolean x should be unimplemented and thus error (friendliest to developers coming from other languages; prevents "unexpected" behavior), or instead perform a bitwise not (saves a symbol for use elsewhere; provides "consistency").

Another data point: Rust chose ! for bitwise not as well.

This is an important argument in favor of this change. While it seems like it easily could have been controversial, I've yet to find a single blog post deriding the Rust developers for this choice (or even a single complaint, anywhere). Of course, Julia is meant to be more approachable than Rust, but many Rust developers will be coming from C where !x means iszero(x), so if it has not been confusing there, perhaps it will not be here either.

[vtjnash]

!x means x != 0 in C

Just fwiw, this isn't what it means in C++, and various style guides may mention not to write out x when used as in an inferred boolean expression as x != 0 to avoid surprises in template code. See also "The Safe Bool Idiom", since after the number of ways you could overload or implement the conversions that could be applied to this expression, it's pretty hard to know what if (x) means without checking the documentation for the type of x.

[StefanKarpinski]

Another reason we may want the ! and ~ separation would be three-valued logic (3VL): it seems ok to have !missing == missing but having ~missing == missing seems sketchier – but maybe it is ok on the premise that the bitwise negation of a missing value is another missing value?

[nalimilan]

I don't think that's a problem. We should probably add ~missing === missing if we don't merge ~ and !.

[Sacha0]

But why is ! the only function that autocomposes like that?

Small digression: ! autocomposition's resounding success makes general autocomposition of unaries an intriguing prospect. I wonder whether the former's utility is privileged given !'s privilege, or whether instead general unary autcomposition would prove similarly useful. Best!

[JeffBezanson]

There are very few unary operators, and it seems to me none are in the same ballpark of utility as inverting predicates, by a long shot. - comes close, e.g. minimizing -f instead of maximizing f, but still doesn't come up nearly as often as !f.

[andyferris]

Count me as one of the dissenters.

I like to think about interfaces - clear, consistent, reliable interfaces. Most users will be exposed to think of ! as an interface that represents "logical not", because that's how it will be used in 99.9% of the code they read. We have in Base (and will probably have more in the package ecosystem) other "logical" types now like missing (well, missing is many things, but it does participate in 3VL - but there are other "fuzzy logics" to consider). Eventually, these "logical" types may even contain data (bits) inside and people will want to use "logical not", "logical or" and "logical and" on them.

While ! happens to be both biwise not and logical not for Bool (and missing) I feel it would be terrible to not know if !fuzzybool does a logical operation or a bitwise operation. There's no need to put two distinct concepts into the one interface - we can provide two interfaces. It really won't be that much effort to use a function for bitwise not in the small number of cases it comes up - in fact, it would be much, much clearer for people not used to low-level C stuff. Some of the best things about Julia are its readability and easy, generic programming - separating helps both.

Also, providing an interface for a very low-level operation like "bitwise not" using one of our precious ASCII operators (~) seems illogical to me. I'd vote to preserve the meaning of ! as "logical not", and deprecate ~x in favor of bitnot(x) or something. I'd also do the same for "bitwise or" and "bitwise and" for exactly the same reason that & and | should work without ambiguity for logical abstractions that contain data (bits) inside.

[JeffBezanson]

I sense enough controversy that we should perhaps leave well enough alone.

[nalimilan]

Maybe we should introduce .&& and .|| operators, and say that &&/||/! and their element-wise variants are the standard logical operators, with &/|/~ being bitwise (and only coincidentally logical, just because that happens to be the same for booleans)? That would be consistent with @StefanKarpinski's summary from above.

Then && and || would be short-circuiting operators just because that makes sense for scalar logical operations. There would actually be no inconsistency with .&& and .||, since whether they short-circuit or not is not observable for arrays.

Previous suggestions to add .&& and .||: #5187 and #6915. Places where users expected it to work:
https://groups.google.com/forum/#!topic/julia-users/tmadZn5pc-A
https://groups.google.com/forum/#!topic/julia-users/dVFT4a4ps7w
https://discourse.julialang.org/t/zeroing-elements-of-matrix-a-depending-on-matrix-b-fast/6849

[rfourquet]

While ! happens to be both biwise not and logical not for Bool (and missing) I feel it would be terrible to not know if !fuzzybool does a logical operation or a bitwise operation

Is it expected that a FuzzyBool type exists where "bitwise not" and "logical not" don't coincide? otherwise, I don't see why "bitwise not" and "logical not" should be considered as separate concepts.
(I also don't really understand why the other possible "generalization" of "logical not" for !, i.e. iszero, would be rejected, if the merge doesn't happen).

I'm personally in favor of the merge of ! and ~ : has ~ been considered instead of ! for the unified meaning? (because != means "not equal", ! seems more adequate, but it may depend on what we would use the freed operator for).

[andyferris]

Is it expected that a FuzzyBool type exists where "bitwise not" and "logical not" don't coincide?

Yes, definitely.

In theory, the data bits could be any representation at all, like a floating-point number of probability. Here are a couple examples examples of many-valued logic on wikipedia; there's also fuzzy logics with infintely-many values (e.g. probability of truth - which really won't work with iszero for logical not). For 3-valued logic, we could use true, false and missing or we could use a new primitive type ThreeValueLogical 2 end with two bits with valid values 0b00 (false), 0b01 (true) and 0b10 (missing). What is !ThreeValueLogical(0b00)? It should be true (0b01), but actually if it's bitwise it's not even in the set, it's 0b11. This is why I say we can't describe ! as both a logical operator and bitwise one.

From memory I think @JeffreySarnoff might be more familiar with this stuff than me? (Forgive me if I'm mistaken).

[Sacha0]

@andyferris, re. #25435 (comment), #25156 and #25180 might interest you. (Edit: Re. "I feel it would be terrible to not know if !fuzzybool does a logical operation or a bitwise operation.", how is this any different from & and |?) Best!

[yurivish]

& and | are just defined as bitwise operations, so you'd know that they were bitwise.

help?> &
search: &

  &(x, y)

  Bitwise and.

help?> |
search: | |>

  |(x, y)

Bitwise or.

But I'm not sure what "logical and" and "logical or" functions you would define if that's what you wanted to do.

[andyferris]

how is this any different from & and |?

Thanks Sacha. It's not different... my position is they're all conflating logical and bitwise (and I feel it would be nice to do something about all of them).

[vtjnash]

primitive type ThreeValueLogical 2 end with two bits with valid values 0b00 (false), 0b01 (true) and 0b10 (missing). What is !ThreeValueLogical(0b00)? It should be true (0b01), but actually if it's bitwise it's not even in the set, it's 0b11

I don't see that this should pose any difficulty, just add a third column to the truth table:

    | miss | true | false |
 p  | 0b10 | 0b01 | 0b00  |
!p  | 0b10 | 0b00 | 0b01  |

I was initially very hesitant about this PR, since seeing, e.g., !3 was mentally a bit surprising after spending so many years memorizing C idiom trivia. But now I think I've come around. I agree that it's rather weird to have this one operator (!) that alone makes some sort of distinction between "logical" and "bitwise"; a distinction that's entirely superfluous given Julia's past decision not to endow random objects with "truthiness", and (more recently) to generally just disallow casting other objects like numbers to Bool. And after some days to reflect, I don't think I find the new syntax for bitflip that odd anymore.

[andyferris]

just add a third column to the truth table

My point is that that truth table is not flipping all the bits. What exactly does the bitwise ! interface guarantee, if not flipping all the bits of a primitive type?

[vtjnash]

Sorry, I suppose my truth table wasn't very clear, as I was mixing some of the implementation details with the definition. Here instead is an executable version of IEEE 1164 standard for full-on 9-value logic (3VL is just the subset X01):

    -- truth table for "not" function
    CONSTANT not_table: stdlogic_1d := 
    --  -------------------------------------------------
    --  |   U    X    0    1    Z    W    L    H    -   |
    --  -------------------------------------------------
         ( 'U', 'X', '1', '0', 'X', 'X', '1', '0', 'X' ); 

(copied from std_logic_1164.vhd)

We can number these however we want, so all that's important is the symbol and the mapping – the bits themselves used to implement this in binary hardware are irrelevant.

[andyferris]

the bits themselves used to implement this in binary hardware are irrelevant

OK, I'm trying to figure out what ! is meant to do. All the conversation above seems to indicate that we want it to flip bits. I assumed these are real, actual bits somewhere on a computer, not imaginary bits.

What you seem to be suggesting here is that we should implement ! as an abstract logical operator, that doesn't care about the bit representation on the computer. (Which is exactly what I was saying.)

So what the pithy one-liner for describing !x? It flips the real bits of a concrete type (like x::UInt8)? It does a logical negation of x? Or, maybe we are choosing to pun both into the one operator, and we choose to live with that?

Sorry everyone for going on (and on) about this - I'm just a little confused what the underlying story is here.

[StefanKarpinski]

To clarify why I ❤️ and 😕 @vtjnash's post, I don't really understand it, but I love it.

Sorry everyone for going on (and on) about this - I'm just a little confused what the underlying story is here.

No worries, I think we're all trying to get clarity here.

[vtjnash]

So what the pithy one-liner for describing !x?

It's the bitflip of x. You're right though that I don't think it makes sense to define it based on the limitations of commodity processor hardware.

In binary logic the bitflip is just:

 p | 0  1 |
!p | 1  0 | 

But if you're making your own hardware following the IEEE 1164 standard, there's 9 possible states for each bit, so the truth table for bitflip is larger:

 p  |   U    X    0    1    Z    W    L    H    -   |
!p  |   U    X    1    0    X    X    1    0    X   |
U = undefined (could be meta-stable)
X = don't know, but either 0 or 1
0 = logical zero / false
1 = logical one / true
Z = high-impedience
W = don't know, but either L or H
L = weak logical zero / false
H = weak logical one / true
- = dont care

[martinholters]

If ! is "bitflip", does it flip a single bit or all the bits in something? By making the meaning of ! include negation/bitflipping for boolean/fuzzy/364245-valued/whatever logic, !42 feels like implicitly treating the integer as a vector of bits and auto-broadcasting the !. That sounds rather un-Julian to me.

BTW, I'm not at all against this change, but having a coherent explanation for why it makes sense would be nice 😄

[StefanKarpinski]

I agree that viewing !42 as broadcasting logical negation is a bit iffy since we've gotten rid of autobroadcasting behaviors everywhere in the language. But that's the not the perspective that suggests unifying these – the perspective is that we define !to mean bitwise negation, of which boolean negation is a special case if you view Bool as a 1-bit integer type (which it is). Perhaps it's clearer to describe the change as "merge ! into ~ as a special case, then rename ~ to !".

[andyferris]

Jameson wrote:

... so the truth table for bitflip is larger:

I think the the word "bitflip" is being used here to mean the two things I am trying to say are distinct. I would say that this is the truth table for logical negation.

I still see the word bitflipping / bitwise NOT as doing something physical to some RAM, cache or register bits (i.e. "binary digits"). The existing methods for ~ seem like the correct set of low-level operations which is "safe" for users - I don't see how this set of operations naturally extends to an abstract function suitable for non-standard logics, for example.

[JeffreySarnoff]

I am familiar with this stuff.

The more frequent uses of a computational logic are to resolve something[s] more quickly than would occur using another approach (e.g. constraint based reasoning) aor to allow easy expression of not-always-simple constructs (e.g. fuzzy reasoning). The first prefers bitwise operations and set valued functions over multibit value representations. In my experience, using one bit (1-of-8 in a byte, etc) per value allows a faster implementation of higher level logic-based operators. Even with fuzzy logic, where the valuations are naturally ordered and easily represented as small integers or floats in (not at all so) 0.. 0.5 (somewhat so) .. 1 (entirely so), it can help to work from distinguished bits and map them to floats as needed.

Modal logics require additional structure -- something that our parameters could support very cleanly.

[StefanKarpinski]

Triage declines.

[JeffBezanson]

Triage thinks it's a bit too late to put in a change that seems to be this contentious. Closing.