Proposal: "with" expressions for record types

[gafter]

This is a proposed enhancement to the proposal for records in #206.

A new expression form is proposed:

primary-expression:
    with-expression:
with-expression:
    primary-expression with { with-initializer-list }
with-initializer-list:
    with-initializer
    with-initializer , with-initializer-list
with-initializer:
    identifier = expression

The token with is a new context-sensitive keyword.

Semantics

The with-expression is translated into a primary constructor invocation that copies members from the primary-expression on the left-hand-side into constructor's parameters, but with some of those replaced by values from the initializer list. Because it depends on the presence of a primary constructor, this is only defined for record types (#206).

(This needs to be described in more detail, including specifying that the left-hand-side expression is evaluated once, the constraints on its type, that identifiers in a with-intiializer bind to a property (or field) of that type, and that the correspondence between constructor parameters and properties are used according to the spec for records. Similarly it needs to give definite assignment rules, order of evaluation, etc.)

Example:

class Person(string FirstName, string LastName);
...
    Person p = new Person(FirstName: "Neil", LastName: "Gafter");
    Person q = p with { FirstName = "Neal" };

The latter is translated into

    Person q = new Person(FirstName: "Neal", LastName: p.LastName);

To be clear, the Person declaration's expansion includes the following:

class Person
{
    public string FirstName { get; }
    public string LastName { get; }
    public Person(string FirstName, string LastName)
        { this.FirstName = FirstName; this.LastName = LastName; }
    // as well as other members not relevant to this issue
}

Because the semantics of this new expression form require a language-defined mapping between constructor arguments and properties of the type, and a primary constructor, it is only defined for record types as specified in #206. Expanding this construct to other types may be the subject of a separate proposal.

[gafter]

@MadsTorgersen I believe you were interested in extending this to a pattern-based construct.

[pawchen]

Nice. Maybe the curly braces can be removed?

[Eyas]

Why not define a compiler-generated/language-specified similar to

Point With(int? X = null, int? Y = null, int? Z = null) =>
    new Point(X ?? this.x, Y ?? this.y, Z ?? this.z);

Nullable isn't quite right as it requires value types, but I wonder why the language needs to be extended here. Methods and named parameters seem to be quite enough here:

class Point(int X, int Y, int Z);
...
    Point p = ...;
    Point q = p.With(Y: 2 );

[gafter]

@Eyas How would that work when the type is a nullable or reference type? You say that your proposal isn't quite right, but what do you have in mind that is quite right?

[Eyas]

Not quite there yet; rather wondering why something like this wasn't pursued. Is there something about extending the language with a brand-new construct that is actually good/needed here? Or was there something explicit about how a method-like approach is unworkable?

How I saw it, worse comes to worse, the compiler could treat a RecordType.With(...) as syntactic sugar and transform it exactly as you proposed.

[pawchen]

@Eyas Isn't it more efficient for compiler to pass the members directly than performing checks for every member?

[gafter]

@Eyas Doing it without additional language support may be possible but awkward. One approach is to define an Optional type with an implicit conversion

public struct Optional<T>
{
    public readonly bool HasValue;
    public readonly T ValueOrDefault;
    public static readonly Default = default(Optional<T>);
    public static implicit operator Optional<T>(T value) { ... }
    /// and a constructor too...
}

Now you can define With as follows

    public Point With(Optional<XType> X = default(Optional<XType>), ...) { ... }

Here are some of the disadvantages

• You now have a public type Optional<T>, and some of these fields can be of type Optional<Something>, which makes the meaning all the more confusing.
• The implementation of this With method must necessarily be much slower than the proposed with language construct because it is doing at runtime what the proposal does at compile-time.
• The syntax is not very pretty.

Plus, you still have to wire it into the language if you want the language to produce these With methods for you, so you're not reducing the concept count by much.

On the other hand, with the proposed with feature, you probably want to have some way for programmers to enable the feature for non-record types. Thus my comment to @MadsTorgersen above.

[Eyas]

@gafter That's very fair. I had thought of the Optional solution, but agree its not worth pursuing for this purpose (might be for the CoreFX folks).

What I floated in my previous post was whether a .With(...) can be a 'reserved' method for record types that desugars into a constructor call with no runtime checks at compile time. Upon sleeping on it, I don't think I'm a fan of this either, anymore.

[MgSam]

I'm glad there's finally a formal proposal for with, as it is sorely needed when working with immutable types. That being said, I don't like tying the feature to record types/primary constructors. Why shouldn't I be able to automatically define a with operator for any type? I frequently have classes that I want a copy constructor for, and it's a pain to write and to maintain.

I think auto-implementation of with, ToString, HashCode, and Equals is orthogonal to record types. Yes, they're certainly desirable for record types, but I might want these things for any type, or I might want them for only certain properties of a type.

I think the autoimplementation should be triggered by a separate keyword, and there should even be a way of annotating which properties should participate in the autogeneration of these identity-type methods.

Currently when you add new properties to a class you likely have to go to 5 different places and make updates and hope you didn't forget one or else your type's sense of identity will get out of whack. While having auto-implementation of these methods for records will help we could do better by making the feature more general and more widely applicable.

[gafter]

@MgSam: My note to @MadsTorgersen was to trigger his work on extending support for the with operator to "other types" by basing it on some kind of programming pattern or convention.

The with operator needs some kind of language-based indication of which constructor it should use, and some language-based indication of the mapping between the constructor parameters and the properties. Those are precisely the things that the primary constructor syntax provides. It can be used "for any type," and the primary constructor can be written to provide values for only "certain properties of a type".

I can believe that some other syntax might provide all of the things needed for the with operator, and yet be applicable to scenarios that are somehow not served well by the primary constructor syntax. I look forward to what @MadsTorgersen or anyone else comes up with.

[ghord]

How about requiring the type to have method of the form:

Type With<PropertyName>(PropertyType value);

So that we can use with keyword even with interfaces like this:

public interface IJob
{
    DateTimeOffset Start { get; }
    DateTimeOffset End { get; }
    IJob WithStart(DateTimeOffset start);
    IJob WithEnd(DateTimeOffset end);
}

public class Job : IJob
{
    public Job(DateTimeOffset start, DateTimeOffset end)
    {
        Start = start;
        End = end;
    }
    public DateTimeOffset Start { get; }
    public DateTimeOffset End { get; }
    public IJob WithStart(DateTimeOffset start)  => new Job(start, End);
    public IJob WithEnd(DateTimeOffset end) => new Job(Start, end);
}

And use it like this:

var job = GetJob(); //returns some IJob
var job2 = job with { 
    Start = DateTimeOffset.Now,
    End = DateTimeOffset.Now.AddHours(1) 
}; // calls job.WithStart(...).WithEnd(...)

[HaloFour]

@ghord That would be interesting. Would there be specific requirements on the return type of the method? Let's say in your example GetJob() actually returned a Job and not an IJob, could you use with in that case? What if Job defined additional With methods? Could they be used in the with initialization as long as they were before Start and End?

[gafter]

@ghord Once you do that, there isn't a lot of added value in the language construct over using the APIs directly. It is much less efficient than the proposal, as it would cause an allocation and copy for each property being modified.

[ghord]

@gafter You are right, it doesn't make much sense to cause unnecessary allocations in this case.

How about using @Eyas construct with slight modification:

Point With(int X = default(int), int Y = default(int), int Z = default(int)) =>
    new Point(X, Y, Z);

And allowing compiler to replace default values in optional parameter with values from properties of source object? This technique is already used in caller attributes after all.

[HaloFour]

@ghord Or a with operator?

public static Point operator with(int X, int Y) => new Point(X, Y);

Then the compilers could look for a convention of a static method called op_with and map the public properties/fields to the parameter names.

Point p2 = p1 with { Y = 2 };
// equivalent to
Point p2 = Point.op_with(X: p1.X, Y: 2);

[Eyas]

But in both these cases, how do you tell the compiler what values to substitute for the default values? There needs to be a new language construct there. Counting on the names being the same seems hacky. Counting on some special treatment of = default(X) also seems weird.

Perhaps a language construct such as:

Point With(int X = somekeyword(this.X), int Y = somekeyword(this.Y) ) =>
    new Point(X, Y);

Where somekeyword, whatever it is, has special compiler treatment, and lets the compiler know to substitute with p.X/etc. for any missing param..

[HaloFour]

@Eyas I mention having the compiler match the parameter names to the members of the source instance. Do you not think that is sufficient?

Whether it's a normal method, an operator (static method) or a constructor such a mechanism would need to exist to let the compiler know what the default value should be for any non-specified property. In my opinion the parameter names is probably sufficient, although an attribute that spells out the name of the member to override may be useful. I don't think a further language construct is really that necessary.

[Eyas]

@HaloFour I did miss that. Matching names seems a bit weird. Does C# do anything like this elsewhere? What happens if someone defines a with operator that takes a parameter that does not match a property name. Does it become a required parameter? Or is that a compile-time error?

[HaloFour]

@Eyas Literally typing that now. 😄

I think the bigger question about custom "withers" is how to deal with parameters that don't match to members (though whatever mechanism). Would that be allowed? Would the consumer be required to supply those parameters?

Also, if using a non-constructor, what if the return type is different? Could a "wither" perhaps with a non-matched parameter be used to construct a different type?

public class Point {
    ...
    public static Line operator with(int X, int Y, Point Destination) => new Line(new Point(X, Y), destination);
}
///
Point point = new Point(2, 2);
Line line = point with { Destination = new Point(4, 4) };

[RichiCoder1]

I'm in the camp that says With should be restricted to returning it's parent type, with no exceptions. I would agree with @gafter that if we let With take additional parameters, at that point is it really that much better then simply calling the Api directly? With should be a relatively cheap way to perform simple copy & transform operations. Any more than that and it risks getting confusing, convoluted, and, my biggest fear, abused.

[HaloFour]

@RichiCoder1

I'd probably agree. Even any interesting use cases where such flexibility could be useful are probably just a hair away from being a form of abuse.

I also like the idea that if a custom "wither" is an operator that the compiler can enforce that the return type is appropriate and that all of the properties can be matched. That way someone can catch that the "wither" isn't valid before it potentially breaks code that attempts to consume it.

public class Point {
    public readonly int X { get; }
    public readonly int Y { get; }
    public readonly int Z { get; }

    public Point(int x, int y, int z) {
        this.X = y; this.Y = y; this.Z = z;
    }

    // good
    public static Point operator with(int X, int Y, int Z) => new Point(X, Y, Z);

    // compiler error, wrong return type
    public static Location operator with(int X, int Y, int Z) => new Location(); 
    // compiler error, Foo is not a property of Point
    public static Point operator with(int X, int Y, int Z, int Foo) => new Point(X, Y, Z);

    // compiler error, Z is not a parameter?
    // or should withers with fewer parameters be permitted to facilitate versioning?
    public static Point operator with(int X, int Y) => new Point(X, Y, 0);
}

[RichiCoder1]

@HaloFour Agreed. Would love to see it just be an operator.

[alrz]

@gafter I'm suggesting that access modifier and sealed could be implicit for nested types of the ADT which are not base class for any other types.

public abstract sealed class A() {
  // public, inherited from A
  abstract case class B() {
    // public, inherited from B
    case class C() { 
      // public sealed, inherited from C
      case class D();
    }
  }
}

Nothing to do with primary constructors. Though, implicit base class and aggregated constructors would work with both ADTs and nested record types.

So now I can refer to Person, but I would have to say Person.Student for the derived type?

This issue would apply to ADTs as well, right? I don't know how would you address this for ADTs. One option is to use using static (and extend it to include all the nested classes?) which doesn't work for generic ADTs as expected. Another option would be generating flat classes which probably is the worst.

It takes otherwise orthogonal language features (parameter list on a type declaration, and nesting) and gives a different meaning to the use of them together.

I don't know if there is any acceptable way to address this, but working in current syntax is very cumbersome for large hierarchy of ADTs (like ASTs) or record types (in the context of DDD).

I would suggest an additional modifier like child, derived, or case itself on nested classes to be explicitly derived from the enclosing type. I think this, with aggregated constructors are a lot better than repeating all the properties and base type for each derived class.

PS: As an optimization, empty case classes could be implementated as a singleton so no new instance would be allocated for each usage.

[gafter]

public abstract sealed class A() {
  // public, inherited from A
  abstract case class B() {
    // public, inherited from B
    case class C() { 
      // public sealed, inherited from C
      case class D();
    }
  }
}

Ohhhkay... but what if I didn't want it to be derived? The normal way to not derive from something is to not place it in your base clause... but that's exactly what you have done and given the opposite meaning.

[alrz]

what if I didn't want it to be derived?

remove the case modifier?

[gafter]

So case as a modifier means inherit from the enclosing type and add extra parameters to the constructor and pass them through to the base constructor? That appears to have nothing whatsoever to do with any existing meaning case has in C# or English.

[alrz]

ok what does a parameter list has to do with generated is operator, With method and all that jazz? I'm saying that this would be a way to declare a record type hierarchy so the compiler would generate the class structure which is not explicit but it makes sense together. F# does a lot more of this by the way.

[gafter]

@alrz My current thinking is that there is no With method - we just call the constructor. And there is no generated operator is - we just use the properties. Only if those are explicitly declared do we pay any attention to them, and I don't think we actually need a With method. As with struct types, we need to define some meaning for Equals and GetHashCode and ToString, and we have a reasonable story for those, but there are no strange user-declared parameters for them that could possibly be accessed in user code.

[phrohdoh]

Why is this proposed?
I assume someone sees it as necessary, but I don't.

If you want to create one object that has values equivalent to others why not be explicit?

I don't see why C# should grow a new construct like this when
Person q = new Person(FirstName: "Neal", LastName: p.LastName);
is perfectly fine and valid.

There are numerous ways to get this result without introducing something new to the language.

[HaloFour]

@phrohdoh

If Person had a lot of properties that would become verbose boilerplate very quickly.

[gafter]

@phrohdoh The Roslyn syntax trees are auto-generated in part so we can add lots of "With" methods. Adding this feature to records means we would not need to.

[MgSam]

@phrohdoh Try doing that for a type that has 20 properties. And make sure you don't forget any, or else you'll have a super-subtle bug that you'll have to track down at runtime.

[alrz]

I think the point is to preserve immutability while you're altering properties' values, which, therefore, without a dedicated language construct, becomes cumbersome and error prone and eventually misses the point.

[kintar0]

The with-expression will come in handy especially when you are dealing with composed records. Consider the following records:

public class HumanName(string First, string Last)
public class HumanAddress(string Street, string City, string State)
public class Person(HumanName Name, HumanAddress Address)

var name = new HumanName( "John" , "Smith" )
var address = new HumanAddress( "A Street" , "Los Angeles" , "California" )
var john = new Person(name, address) 

Will it be possible to write

var movedJohn = john with { Name.Last = "Doe", Address.Street = "Another Street" }

to change the properties in deeper levels?

When working with immutable classes, changing properties that are not on the 'first level' were one of my main reasons to switch back to mutable classes. The number of With.... methods you need to provide grows pretty fast, especially when you think about adding them on every level.

[gafter]

@kintar0 As currently specified, you would have to

var movedJohn = john with {
    Name = john.Name with { Last = "Doe" },
    Address = john.Address with { Street = "Another Street" } }

[alrz]

Since all of the parameters are optional, calling the With method without args behaves like Clone,

var clone = obj.With(); // weird

PS: This issue is seriously outdated.

[bbarry]

What if there were simply pairs of optional parameters required on the wither:

public partial class Point
{
    public virtual Point With(
        [Wither("wX")] int X = 0, bool wX = false, 
        [Wither("wY")] int Y = 0, bool wY = false
        ) => new Point(xX ? X : this.X, wY ? Y : this.Y);
}

the syntax (C#Next):

var p2 = p1 with { X = 0 };

compiles the same as (C#6):

var p2 = p1.With(wX: true, X: 0);

This pattern could be done with extension methods:

public static class PointExtensions
{
    public static Point With(this Point @this,
        [Wither("wX")] int X = 0, bool wX = false, 
        [Wither("wY")] int Y = 0, bool wY = false
        ) => new Point(xX ? X : this.X, wY ? Y : this.Y);
}

It avoids decapitation (as much as method calls do today):

public partial class LimitedPoint : Point
{
    public override Point With(
        [Wither("wX")] int X = 0, bool wX = false, 
        [Wither("wY")] int Y = 0, bool wY = false
        ) => new LimitedPoint (xX ? X : this.X, wY ? Y : this.Y);
}

And it doesn't require any new syntax on the declaration side (though a syntax might be useful; it could be done separately). Nor does it require a new Optional<T> type.

[gafter]

We are not likely to do this. However, @MadsTorgersen has volunteered to champion "pattern-based with expressions"; see dotnet/csharplang#162 and the records proposal at https://github.com/dotnet/csharplang/blob/master/proposals/records.md .