support unboxed, uniquely typed closures

[thestinger]

It would need to be made obvious that these are generic functions/structs, because they won't be interchangeable.

fn foo<T: fn(int, int)>(x: T, y: int) { x(5, y) }

Every closure would have either a unique type (like C++ closures) or at least a type based on the environment. A stack closure would start off as a bare moveable fn(...) implementation able to be converted to boxed &fn(...), ~fn(...) or Rc<fn(...)> objects.

An unboxed closure would just be static dispatch without a function pointer. As with traits, you would choose between static/dynamic dispatch.

[nikomatsakis]

This is not necessarily related to #6308 -- closure types are not considered dynamically sized in the latest plans. It is certainly related to the idea of replacing closures with traits. Basically goes all the way, removing also & closures. The main obstacle here is the need to generalize bound lifetimes, which currently only occur in closure types, to something that can occur in any trait reference.

[pcwalton]

Not until Rust 2.0

[thestinger]

The ability to modify the environment with &fn is a backwards compatibility issue since a trait would require &mut fn as far as I can tell. I opened a sub-bug for the backwards compatibility hazard. If we did decide to do this, it would make sense to implement that relatively minor change for 1.0.

The distinction would be marginally useful in the current system because &fn would be copyable.

[thestinger]

I think this would remove the need for function bounds, since you would just take T: Send + fn(int, int) or do trait Foo: Clone + (fn(int, int) -> bool).

[glaebhoerl]

I really, really like this idea. (But I'm not a dev or anything.)

But a problem is that if &mut fn can be borrowed to &fn, &fn can be copied, &fn can be called, and fn can mutate (with DST you can't prevent the first two), then you can break soundness (in the usual way). The principled solution would be to separately track whether a closure can mutate from mutability of references to it (so fn and fn mut plus & and &mut), and adopt the same ownership/aliasing rules for calling mutating closures (through references) as for mutating mutable variables (through references). So

+----------------------------------------------------------+
|             | Can be called | Can be copied | Can mutate |
|-------------+---------------+---------------+------------|
| &fn         |      Yes      |      Yes      |     No     |
|-------------+---------------+---------------+------------|
| &fn mut     |      No       |      Yes      |     N/A    |
|-------------+---------------+---------------+------------|
| &mut fn     |      Yes      |      No       |     No     |
|-------------+---------------+---------------+------------|
| &mut fn mut |      Yes      |      No       |     Yes    |
+----------------------------------------------------------+

(Again these are the same rules as if you were to substitute &T and &mut T in place of fn and fn mut.)

Unfortunately this means that if you want a reference to a mutating closure that you can actually call, you have to write &mut fn mut which is somewhat onerous. I'd personally much rather sacrifice a little syntax for nice semantics than the other way around (we already require annotating the possibility of mutation everywhere else!), but again IANAD.

[thestinger]

A fn would only be mutable if you had it stored in a mutable location, and you wouldn't be able to get one out of &fn or &mut fn unless the environment (the closure) could be cloned. I think the same guarantees applying to other types would work for it, as long as the environment's lifetime was preserved.

You wouldn't be able to mutate through an &mut captured in the environment if you didn't have either &mut fn or an unboxed fn in a mutable location without any loans.

[glaebhoerl]

If you're referring to the "[if ...] and fn can mutate" part from above, the problem is not calling a bare fn and having it mutate, but calling one through a copyable reference and having it mutate. If &fn is callable, copyable, and can mutate, then you can write the code from nikomatsakis's blog post a few months ago.

You wouldn't be able to mutate through an &mut captured in the environment if you didn't have either &mut fn

But if you have &mut fn you can get (any number of) &fn. Would &fn be non-callable?

[thestinger]

You wouldn't be able to mutate the environment of &fn, it would follow the regular borrowed pointer rules. The current &fn would become &mut fn. A closure would really only represent the environment, with the unique type of the unboxed closure or the trait function pointer making it actually callable (just like other traits).

[huonw]

I think that @glehel's point is the current rules allow &mut T to be borrowed as &T, but this is the opposite: &fn can become &mut fn freely, but not vice versa.

[thestinger]

When I say:

The current &fn would become &mut fn.

I mean the current definition of &fn would be identical to the new definition of &mut fn.

You couldn't convert an &fn to &mut fn - it would have the exact same rules as a borrowed pointer, since they would be trait objects. The proposed &fn (same rules as normal borrowed pointers) doesn't exist in the type system today. The ability to call the function is semantically equivalent to the ability to call a method on any normal object.

[glaebhoerl]

(Re: your previous comment) I understand that (and like it). I'm assuming here that the DST proposal is also in effect. Are you?

The current &fn would become &mut fn.

Right, and there would be a "new" &fn (really just the composition of & and fn - an immutable trait object) which is copyable but can't mutate. So far so good. But what happens when you borrow &mut fn to &fn? Again, DST says that &mut T (and likewise &T) behaves the same for all T, so you can't prevent it.

To follow the trait analogy, restricting ourselves to single argument functions, consider the possible desugarings of fn:

trait Fn<Arg, Ret> {
    fn call(&self, Arg) -> Ret; // `self` is the environment
}

trait FnMut<Arg, Ret> {
    fn call(&mut self, Arg) -> Ret;
}

Which one would fn(Arg) -> Ret desugar to? If it's the first, then it can't mutate the environment. If it's the second, then &fn isn't callable (self is by &mut, but self is immutable). In my earlier post fn corresponds to Fn and fn mut corresponds to FnMut.

[thestinger]

The ability to call an object would require compiler support, and wouldn't de-sugar to a method call. The closure trait bound would be special-cased to support the call syntax, and only give you a mutable environment when the object is mutable.

I haven't really considered the ability to implement arbitrary callable objects as part of this - it's a magical capability like indexing on vectors or Send. There are no variadic functions so I don't think you could ever implement a normal trait.

[glaebhoerl]

Right the trait desugaring was only intended as an analogy (and why I restricted it to single-argument).

Possibly I'm being dense. Which part of the following would be illegal?

struct R<'self> {
    c: &'self fn(&R)
}

fn innocent_looking_victim() {
    let mut vec = ~[1, 2, 3];
    conspirator(|f| {
        if vec.len() < 100 {
            vec.push(4);
            for vec.each |i| {
                f.c(&f)
            }
        }
    })
}

// only this part is different
fn conspirator(f: &mut fn(&R)) {
    conspirator_impl(f);
}

fn conspirator_impl(f: &fn(&R)) {
    let r = R {c: f};
    f(&r)
}

[thestinger]

@glehel: ah, nevermind, I see what you've been saying now

[thestinger]

Another way to do this would be representing a stack closure as fn<T>(...) -> ... where the T is a tuple of the captures. For example, a stack closure might be fn<(int, &'a u8)>(int) -> bool and you could take one as a parameter on a generic function like fn foo<T>(x: fn<T>(int, int) -> bool).

[pnkfelix]

cc me

[thestinger]

I'm going to close this as obsoleted by the rust-lang/rfcs#77 proposal. If this or something along these lines is accepted, a new bug can be opened for implementing it without the now irrelevant discussion based on the Rust of nearly a year ago.