[Proposal] Improving scenarios that break backwards compatibility

[bondsbw]

Problem

Backwards compatibility is a big deal for any language design team. The language needs to evolve, but the team never wants to break existing code.

C# has become better every version. The number of good ideas seems to be endless, but many of them suffer from backwards compatibility. For example, #1303 proposes fixing the "billion dollar mistake":

string? n; // nullable string
string! s; // non-nullable string

But it is impossible to enforce the developer to apply ? to every nullable variable, because it would break an insurmountable amount of existing code.

Proposal

Let's move past these limitations but in a way that doesn't actually break backwards compatibility, by using preprocessor directives.

At the top of my C# file, I add a directive like this:

#explicit_nullability

using System;
// and so on...

Later in that file, I might write the following code:

string? n; // nullable string, OK
string! s; // non-nullable string, OK
string x;  // ERROR, must specify nullability when using #explicit_nullability directive

Other C# files in the project would continue to work normally, i.e. without explicit nullability.

C# v.Future

The above proposal is targeted at features which are not universally loved. Many developers would love explicit nullability, but some certainly would not see the value of the extra ? everywhere.

Those features may not be good for the main C# language. But some features will have a general consensus, and those can be combined with other breaking changes at once (e.g., C# version 8 might be selected to break backwards compatibility).

The project file could declare which version of C# (7 vs. 8+) is utilized by default for all .cs files. Older projects would default to C# 7 compatibility, and the programmer could use the directive #csharp8 on individual files as they are updated to the new specifications.

(EDIT: The title may have been misleading, so I updated it.)

[HaloFour]

Breaking compatibility is a non-starter. It's all too easy to find instances where language designers have attempted to reboot and break compatibility between major versions only to find the decision reviled and no willingness to adopt the breaking changes.

The inability to "fix" "non-nullable reference types" has little to do with C#. It's not a problem of syntax, it's a problem of the fact that the CLR has zero concept of "non-nullable reference types" and absolutely requires a "zero" state for all types, which has to be null for any reference type.

[bondsbw]

@HaloFour, thank you for your feedback.

Nullability is just an example case for breaking backwards compatibility, one of many posed in comments here, on CodePlex, and elsewhere. Plus I don't want to immediately assume that CLR changes are out of scope for some unnamed, unplanned future release.

(Sorry for the original post... I wrote "mutability" everywhere I meant "nullability". It has been edited.)

[paulomorgado]

If you break backward compatibility, you might as well throw it all away and start all over again.

And as @HaloFour states, there's nothing the language can do about non nullable reference types. Once and if the CLR solves this, than the language can leverage it - in a non backward compatibility way, obviously.

[SolalPirelli]

The only major language that broke backwards compatibility is Python. That did not turn out well.

[bondsbw]

Keep in mind that I'm not suggesting a huge Python-style break. I'm suggesting two things:

1. An F# #light-like syntax to tweak the syntax of the current file only.
2. The ability to change the entire project to use that syntax without needing the preprocessor directive on each file, e,g. #csharp8, when it makes sense to evolve the base language.

Another thing, not all backwards compatibility breaks are in syntax. #csharp8 could simply be improving syntax in scenarios where it might break a few people who currently use new keywords as identifiers.

[HaloFour]

@bondsbw

I don't see why any directives like that would be necessary and I would argue that they do little but make the language more confusing. The language thus far has been quite good at preventing any new feature from breaking existing syntax through careful application of contextual keywords. Code written in C# 1.0 with class names or variable names of var, yield, async, await, etc., all compile perfectly fine in C# 6.0.

As for any potential #light, DO NOT WANT. Plenty of CLR languages exist that have syntax of that nature, all of which can be cross-referenced from C# or merged via build tools. There's no reason to turn C# into F#, either optionally or otherwise. If we're talking features from F#, that's fine, where the syntax can adapt appropriate to the expected C# grammar. But to modify the syntax for the sake of modifying syntax is a large effort for absolutely no benefit whatsoever.

[bondsbw]

@HaloFour

I don't see why any directives like that would be necessary and I would argue that they do little but make the language more confusing.

But to modify the syntax for the sake of modifying syntax is a large effort for absolutely no benefit whatsoever.

No, I'm not suggesting change for the sake of change. Some major, beneficial features are incompatible with today's C#. I discussed explicit nullability in the original post, it's a feature that a lot of people would find useful. No matter if the CLR is updated to add non-nullability to reference types, the C# language couldn't require explicit nullability without breaking existing code.

But if you add #explicit_nullability to the top of a C# file, the compiler would know that the user expects the compiler to break compatibility, in a known way, for that code file.

Even if the C# team decides that building such features into the primary language is a bad idea (or at least not a high enough priority), it might be a nice generalized approach for selectively turning on compiler extensions we create, or that we pick up from the community.

[HaloFour]

@bondsbw

In my opinion adding directives to enable/disable language features opens a massive can of worms and I would prefer that the language goes to some extreme lengths to avoid doing so. To add one opens the potential for adding many and the entire concept of a stable language that evolves carefully over time goes right out the window.

Your argument is that such flags would allow such breaking evolution to happen. My argument is that such breaking evolution shouldn't happen.

[gafter]

We're not going to break backward compatibility, and we avoid creating dialects.