Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Syntax for precise capturing: impl Trait + use<..> #125836

Closed
traviscross opened this issue Jun 1, 2024 · 15 comments
Closed

Syntax for precise capturing: impl Trait + use<..> #125836

traviscross opened this issue Jun 1, 2024 · 15 comments
Labels
disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. F-precise_capturing `#![feature(precise_capturing)]` finished-final-comment-period The final comment period is finished for this PR / Issue. T-lang Relevant to the language team, which will review and decide on the PR/issue.

Comments

@traviscross
Copy link
Contributor

traviscross commented Jun 1, 2024

For precise capturing (#123432), we need to decide which syntax to adopt.

The original two, left as an open question in the RFC, were:

  1. impl use<..> Trait
    • This syntax is used throughout the RFC.
  2. use<..> impl Trait
    • This syntax is the worthy challenger.

(See the alternatives section in the RFC for a detailed comparative analysis of these options. In particular, so as to reduce duplication, please read that section carefully before commenting here.)

However, in the design meeting on 2024-06-05, as described below, we settled on placing use<..> within the list of bounds, e.g.:

fn foo<'a>() -> impl Sized + use<'a> {}

This issue is to track the resolution of the open question on syntax left in the RFC.

Tracking:

@traviscross traviscross added T-lang Relevant to the language team, which will review and decide on the PR/issue. F-precise_capturing `#![feature(precise_capturing)]` labels Jun 1, 2024
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Jun 1, 2024
@traviscross traviscross removed the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Jun 1, 2024
@traviscross
Copy link
Contributor Author

traviscross commented Jun 6, 2024

We discussed this in the lang design meeting today:

The conclusion of the team is that we'll make use<..> a bound. That is, we'll support impl use<..> + Trait, impl Trait + use<..>, etc.

For now, we will support at most one such bound in a list of bounds, and semantically we'll only support these bounds in the item bounds of RPIT-like impl Trait opaque types (i.e., in the places discussed in the RFC).

We were interested in later extending this so that it could be used in the bounds on associated types and with dyn. This syntactic choice is the most scalable in that way.

For RPITIT (i.e. within trait definitions), we defer to implementation work and to stabilization whether to support it or to leave this work to later, e.g. to the point at which we support use<..> in the bounds on associated types.

Note, if we were later to support multiple use<..> items in a list of bounds, that use<T> + use<U> would represent a union, and so use<T> + use<U> would not be equivalent to use<T, U>.

If the bounds of the opaque type reference generics that are not present in the use<..>, that's an error of course, and a machine-applicable fix seems possible and desirable here.

We found the idea of this being a bound surprisingly appealing, and people liked that this choice allows for putting + use<..> at the end of a list of bounds. Doing it this way also means we don't need to migrate the ty macro matcher fragment specifier.

Thanks to @joshtriplett for proposing this appealing option. We resolved a difficult choice between two good options by finding an even better one. That's Rust at its best.

bors added a commit to rust-lang-ci/rust that referenced this issue Jun 18, 2024
Rework `feature(precise_capturing)` to represent `use<...>` as a syntactical bound

Reworks `precise_capturing` for a recent lang-team consensus.

Specifically:

> The conclusion of the team is that we'll make use<..> a bound. That is, we'll support impl use<..> + Trait, impl Trait + use<..>, etc.

> For now, we will support at most one such bound in a list of bounds, and semantically we'll only support these bounds in the item bounds of RPIT-like impl Trait opaque types (i.e., in the places discussed in the RFC).

Lang decision in favor of this approach:

- rust-lang#125836 (comment)

Tracking:

- rust-lang#123432
@tmandry tmandry changed the title Decide for precise capturing: impl use<..> Trait vs use<..> impl Trait Syntax for precise capturing: impl Trait + use<..> Jun 19, 2024
@tmandry
Copy link
Member

tmandry commented Jun 19, 2024

Reopening to formally propose FCP on the above change. I think it is a significant enough delta from the RFC that we would be better off doing an FCP and documenting the rationale now, instead of at stabilization time.

This FCP resolves the unresolved question in the RFC about syntax.

@rfcbot fcp merge

@tmandry tmandry reopened this Jun 19, 2024
@rfcbot
Copy link

rfcbot commented Jun 19, 2024

Team member @tmandry has proposed to merge this. The next step is review by the rest of the tagged team members:

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

@rfcbot rfcbot added proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. labels Jun 19, 2024
@tmandry tmandry added the I-lang-nominated Nominated for discussion during a lang team meeting. label Jun 19, 2024
@pnkfelix
Copy link
Member

I don't know what our official policy is, but @traviscross , would you mind updating the description to reflect the new proposal from #125836 (comment) ?

(Or I guess even just making the line of the description be a link to that comment...)

@rfcbot rfcbot added final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. and removed proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. labels Jun 19, 2024
@rfcbot
Copy link

rfcbot commented Jun 19, 2024

🔔 This is now entering its final comment period, as per the review above. 🔔

@pnkfelix
Copy link
Member

Also, I do not necessarily agree with "Note, if we were later to support multiple use<..> items in a list of bounds, that use + use would represent a union, and so use + use would not be equivalent to use<T, U>." ; but I do not see us as committing to a union (vs intersection) path via our FCP here, so I'm willing to check my box and not make this a formal concern here.

@nikomatsakis
Copy link
Contributor

nikomatsakis commented Jun 19, 2024

@rfcbot reviewed

I concur with @pnkfelix and I specifically want us to leave room for different interpretations of use in the future. There are two that I think make sense, described below. However, thanks to the restrictions of "at most one use" and "must include all generics that appear elsewhere in the bounds", both of these interpretations are equivalent, and I think that's fine for the moment.

To explain the two options, let's start by defining a conceptual bound use_only<G..> that is implemented for all types that reference the generics in G... (but not generics outside that set). That is the right formal mechanism and what e.g. I expect a-mir-formality to think about. Lacking a use_only bound therefore implies any type could be used.

One option then is to desugar use to use_only. This has the counterintuitive implication that use<A> + use<B> is equivalent to use<> which seems surprising.

An alternative option is to have a more complex desugaring where use appears in the surface syntax but not in the internal representation. We would translate a "user-facing" set of bounds as follows. If there is a use<G..> at all, then we take the union G.. of all generics that appear in the bounds anywhere and add use_only<G..>. Otherwise we don't add anything at all. In this way use<A, B> is equivalent to use<A> + use<B>: both would desugar to a single use_only<A, B>.

I think the second rule is more intuitive but also brings up mild concerns. Specifically we try to avoid "discontinuities" like this, where having no uses is "different" than having one or more uses. These kind of rules can have composability problems, but in this instance it might be that it's more composable. Unclear.

Anyway, we don't have to decide this now, but I do want to leave room for it as a possible future direction (which we may never need).

@Dirbaio
Copy link
Contributor

Dirbaio commented Jun 19, 2024

I've got one concern with the + use<> syntax.

If we make use<> a bound, then it makes sense that it only restricts the hidden type further. That's what bounds do after all. Adding a bound can only make types no longer be accepted, it can't make a previously-not-accepted type start being accepted.

In Edition 2024, the default is to capture everything in scope. So, use<> is indeed only restricting. Without use<> the hidden type is allowed to capture anything, with use<> it's only allowed to capture the listed variables. Makes sense.

However, in Edition 2021 and lower, the default is to capture only some things. How does use<> work with this? There's a few options:

  • use<> overrides the default capture list. Therefore it can either remove or add variables to the capture list. Then, it's no longer true that use<> always restricts the hidden type, so it doesn't make sense to add a bound.
  • use<> only further restricts the default capture list. I find this a bit unfortunate because it means you'll still need the + Captures<'a> hack to add a lifetime capture.
  • use<> is entirely disallowed in Edition 2021 and lower. This is also unfortunate, and IIUC the aim is to add new features to all editions if possible, and it feels like in this case it should be.

Neither is particularly great


IMO making use<> a bound feels somewhat wrong, due to:

  • It doesn't make sense in Edition <2021 (see above)
  • Having multiple use<> doing intersection and not union is unintuitive.
  • Artificially disallowing use<> + use<> is also strange, no other bound in Rust behaves like that today.

I think the use<> impl Trait syntax makes much more sense. The way I see it, an opaque type is

  • a list of captures
  • a list of bounds

The list of captures is either automatically inferred (with different rules in <=2021 and >=2024), or explicitly specified with use<>.

Under this mental model, it makes sense for use<> to be a "modifier" applied to the impl keyword, and not a bound. So, use<> impl Trait syntax .The result makes sense for both <=2021 and >=2024 editions, and avoids the weirdness of two use<>s by only allowing one.

@traviscross
Copy link
Contributor Author

traviscross commented Jun 19, 2024

The intuition I'd suggest is that, if a bounds list contains a use<..> bound, that fully specifies the set of captures, and if it doesn't, then the compiler adds one in lowering that lists all of the generics captured under the rules for that edition.

That is, I'd suggest thinking about it in terms of the use<..> bound always being present, conceptually, and that there's a convenient elision rule.

@tmandry
Copy link
Member

tmandry commented Jun 20, 2024

One additional argument that was raised in the meeting, but never really written out as an example, was the use of use<> as a bound on GATs. This really cemented for me the value in thinking of it as a bound. As a quick sketch, the bound could go in the trait:

trait BufferedIterator<A: Arena> {
    type Item<'a>: use<'a>;
    fn next<'a>(&'a mut self, a: &'a mut A) -> Self::Item<'a>;
}

or in use sites:

fn foo(iter: impl BufferedIterator<MyArena, for<'a> Item<'a>: use<'a>>) { ... }

Obviously this is a pretty niche and advanced feature, but it seems to make sense conceptually.

@tmandry
Copy link
Member

tmandry commented Jun 20, 2024

However, in Edition 2021 and lower, the default is to capture only some things. How does use<> work with this? There's a few options:

  • use<> overrides the default capture list. Therefore it can either remove or add variables to the capture list. Then, it's no longer true that use<> always restricts the hidden type, so it doesn't make sense to add a bound.
  • use<> only further restricts the default capture list. I find this a bit unfortunate because it means you'll still need the + Captures<'a> hack to add a lifetime capture.
  • use<> is entirely disallowed in Edition 2021 and lower. This is also unfortunate, and IIUC the aim is to add new features to all editions if possible, and it feels like in this case it should be.

Neither is particularly great

I think this is possibly the strongest argument against use<> as a bound, but I'm personally okay with saying that use<> overrides the default capture list in all editions, therefore it makes slightly less sense to conceptualize it as a bound in earlier editions. This is based on the ideas of joint behavior across editions and that editions are meant to be adopted, so while we prefer to have language features work the same across all editions where possible, we're also okay with only delivering the best possible experience in later editions.

@traviscross traviscross removed the I-lang-nominated Nominated for discussion during a lang team meeting. label Jun 26, 2024
flip1995 pushed a commit to flip1995/rust-clippy that referenced this issue Jun 28, 2024
Rework `feature(precise_capturing)` to represent `use<...>` as a syntactical bound

Reworks `precise_capturing` for a recent lang-team consensus.

Specifically:

> The conclusion of the team is that we'll make use<..> a bound. That is, we'll support impl use<..> + Trait, impl Trait + use<..>, etc.

> For now, we will support at most one such bound in a list of bounds, and semantically we'll only support these bounds in the item bounds of RPIT-like impl Trait opaque types (i.e., in the places discussed in the RFC).

Lang decision in favor of this approach:

- rust-lang/rust#125836 (comment)

Tracking:

- rust-lang/rust#123432
@marziply
Copy link

For unions of use<>, as per the example given: use<T> + use<U>, could we not take inspiration from other languages by using the pipe operator? I feel use<T | U> makes more sense.

@rfcbot rfcbot added finished-final-comment-period The final comment period is finished for this PR / Issue. and removed final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. labels Jun 29, 2024
@rfcbot
Copy link

rfcbot commented Jun 29, 2024

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

@rfcbot rfcbot added the to-announce Announce this issue on triage meeting label Jun 29, 2024
@apiraino apiraino removed the to-announce Announce this issue on triage meeting label Jul 4, 2024
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 20, 2024
…=spastorino

Stabilize opaque type precise capturing (RFC 3617)

This PR partially stabilizes opaque type *precise capturing*, which was specified in [RFC 3617](rust-lang/rfcs#3617), and whose syntax was amended by FCP in [rust-lang#125836](rust-lang#125836).

This feature, as stabilized here, gives us a way to explicitly specify the generic lifetime parameters that an RPIT-like opaque type captures.  This solves the problem of overcapturing, for lifetime parameters in these opaque types, and will allow the Lifetime Capture Rules 2024 ([RFC 3498](rust-lang/rfcs#3498)) to be fully stabilized for RPIT in Rust 2024.

### What are we stabilizing?

This PR stabilizes the use of a `use<'a, T>` bound in return-position impl Trait opaque types.  Such a bound fully specifies the set of generic parameters captured by the RPIT opaque type, entirely overriding the implicit default behavior.  E.g.:

```rust
fn does_not_capture<'a, 'b>() -> impl Sized + use<'a> {}
//                               ~~~~~~~~~~~~~~~~~~~~
//                This RPIT opaque type does not capture `'b`.
```

The way we would suggest thinking of `impl Trait` types *without* an explicit `use<..>` bound is that the `use<..>` bound has been *elided*, and that the bound is filled in automatically by the compiler according to the edition-specific capture rules.

All non-`'static` lifetime parameters, named (i.e. non-APIT) type parameters, and const parameters in scope are valid to name, including an elided lifetime if such a lifetime would also be valid in an outlives bound, e.g.:

```rust
fn elided(x: &u8) -> impl Sized + use<'_> { x }
```

Lifetimes must be listed before type and const parameters, but otherwise the ordering is not relevant to the `use<..>` bound.  Captured parameters may not be duplicated.  For now, only one `use<..>` bound may appear in a bounds list.  It may appear anywhere within the bounds list.

### How does this differ from the RFC?

This stabilization differs from the RFC in one respect: the RFC originally specified `use<'a, T>` as syntactically part of the RPIT type itself, e.g.:

```rust
fn capture<'a>() -> impl use<'a> Sized {}
```

However, settling on the final syntax was left as an open question.  T-lang later decided via FCP in [rust-lang#125836](rust-lang#125836) to treat `use<..>` as a syntactic bound instead, e.g.:

```rust
fn capture<'a>() -> impl Sized + use<'a> {}
```

### What aren't we stabilizing?

The key goal of this PR is to stabilize the parts of *precise capturing* that are needed to enable the migration to Rust 2024.

There are some capabilities of *precise capturing* that the RFC specifies but that we're not stabilizing here, as these require further work on the type system.  We hope to lift these limitations later.

The limitations that are part of this PR were specified in the [RFC's stabilization strategy](https://rust-lang.github.io/rfcs/3617-precise-capturing.html#stabilization-strategy).

#### Not capturing type or const parameters

The RFC addresses the overcapturing of type and const parameters; that is, it allows for them to not be captured in opaque types.  We're not stabilizing that in this PR.  Since all in scope generic type and const parameters are implicitly captured in all editions, this is not needed for the migration to Rust 2024.

For now, when using `use<..>`, all in scope type and const parameters must be nameable (i.e., APIT cannot be used) and included as arguments.  For example, this is an error because `T` is in scope and not included as an argument:

```rust
fn test<T>() -> impl Sized + use<> {}
//~^ ERROR `impl Trait` must mention all type parameters in scope in `use<...>`
```

This is due to certain current limitations in the type system related to how generic parameters are represented as captured (i.e. bivariance) and how inference operates.

We hope to relax this in the future, and this stabilization is forward compatible with doing so.

#### Precise capturing for return-position impl Trait **in trait** (RPITIT)

The RFC specifies precise capturing for RPITIT.  We're not stabilizing that in this PR.  Since RPITIT already adheres to the Lifetime Capture Rules 2024, this isn't needed for the migration to Rust 2024.

The effect of this is that the anonymous associated types created by RPITITs must continue to capture all of the lifetime parameters in scope, e.g.:

```rust
trait Foo<'a> {
    fn test() -> impl Sized + use<Self>;
    //~^ ERROR `use<...>` precise capturing syntax is currently not allowed in return-position `impl Trait` in traits
}
```

To allow this involves a meaningful amount of type system work related to adding variance to GATs or reworking how generics are represented in RPITITs.  We plan to do this work separately from the stabilization.  See:

- rust-lang#124029

Supporting precise capturing for RPITIT will also require us to implement a new algorithm for detecting refining capture behavior.  This may involve looking through type parameters to detect cases where the impl Trait type in an implementation captures fewer lifetimes than the corresponding RPITIT in the trait definition, e.g.:

```rust
trait Foo {
    fn rpit() -> impl Sized + use<Self>;
}

impl<'a> Foo for &'a () {
    // This is "refining" due to not capturing `'a` which
    // is implied by the trait's `use<Self>`.
    fn rpit() -> impl Sized + use<>;

    // This is not "refining".
    fn rpit() -> impl Sized + use<'a>;
}
```

This stabilization is forward compatible with adding support for this later.

### The technical details

This bound is purely syntactical and does not lower to a [`Clause`](https://doc.rust-lang.org/1.79.0/nightly-rustc/rustc_middle/ty/type.ClauseKind.html) in the type system.  For the purposes of the type system (and for the types team's curiosity regarding this stabilization), we have no current need to represent this as a `ClauseKind`.

Since opaques already capture a variable set of lifetimes depending on edition and their syntactical position (e.g. RPIT vs RPITIT), a `use<..>` bound is just a way to explicitly rather than implicitly specify that set of lifetimes, and this only affects opaque type lowering from AST to HIR.

### FCP plan

While there's much discussion of the type system here, the feature in this PR is implemented internally as a transformation that happens before lowering to the type system layer.  We already support impl Trait types partially capturing the in scope lifetimes; we just currently only expose that implicitly.

So, in my (errs's) view as a types team member, there's nothing for types to weigh in on here with respect to the implementation being stabilized, and I'd suggest a lang-only proposed FCP (though we'll of course CC the team below).

### Authorship and acknowledgments

This stabilization report was coauthored by compiler-errors and TC.

TC would like to acknowledge the outstanding and speedy work that compiler-errors has done to make this feature happen.

compiler-errors thanks TC for authoring the RFC, for all of his involvement in this feature's development, and pushing the Rust 2024 edition forward.

### Open items

We're doing some things in parallel here.  In signaling the intention to stabilize, we want to uncover any latent issues so we can be sure they get addressed.  We want to give the maximum time for discussion here to happen by starting it while other remaining miscellaneous work proceeds.  That work includes:

- [x] Look into `syn` support.
  - dtolnay/syn#1677
  - dtolnay/syn#1707
- [x] Look into `rustfmt` support.
  - rust-lang#126754
- [x] Look into `rust-analyzer` support.
  - rust-lang/rust-analyzer#17598
  - rust-lang/rust-analyzer#17676
- [x] Look into `rustdoc` support.
  - rust-lang#127228
  - rust-lang#127632
  - rust-lang#127658
- [x] Suggest this feature to RfL (a known nightly user).
- [x] Add a chapter to the edition guide.
  - rust-lang/edition-guide#316
- [x] Update the Reference.
  - rust-lang/reference#1577

### (Selected) implementation history

* rust-lang/rfcs#3498
* rust-lang/rfcs#3617
* rust-lang#123468
* rust-lang#125836
* rust-lang#126049
* rust-lang#126753

Closes rust-lang#123432.

cc `@rust-lang/lang` `@rust-lang/types`

`@rustbot` labels +T-lang +I-lang-nominated +A-impl-trait +F-precise_capturing

Tracking:

- rust-lang#123432

----

For the compiler reviewer, I'll leave some inline comments about diagnostics fallout :^)

r? compiler
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 20, 2024
…=spastorino

Stabilize opaque type precise capturing (RFC 3617)

This PR partially stabilizes opaque type *precise capturing*, which was specified in [RFC 3617](rust-lang/rfcs#3617), and whose syntax was amended by FCP in [rust-lang#125836](rust-lang#125836).

This feature, as stabilized here, gives us a way to explicitly specify the generic lifetime parameters that an RPIT-like opaque type captures.  This solves the problem of overcapturing, for lifetime parameters in these opaque types, and will allow the Lifetime Capture Rules 2024 ([RFC 3498](rust-lang/rfcs#3498)) to be fully stabilized for RPIT in Rust 2024.

### What are we stabilizing?

This PR stabilizes the use of a `use<'a, T>` bound in return-position impl Trait opaque types.  Such a bound fully specifies the set of generic parameters captured by the RPIT opaque type, entirely overriding the implicit default behavior.  E.g.:

```rust
fn does_not_capture<'a, 'b>() -> impl Sized + use<'a> {}
//                               ~~~~~~~~~~~~~~~~~~~~
//                This RPIT opaque type does not capture `'b`.
```

The way we would suggest thinking of `impl Trait` types *without* an explicit `use<..>` bound is that the `use<..>` bound has been *elided*, and that the bound is filled in automatically by the compiler according to the edition-specific capture rules.

All non-`'static` lifetime parameters, named (i.e. non-APIT) type parameters, and const parameters in scope are valid to name, including an elided lifetime if such a lifetime would also be valid in an outlives bound, e.g.:

```rust
fn elided(x: &u8) -> impl Sized + use<'_> { x }
```

Lifetimes must be listed before type and const parameters, but otherwise the ordering is not relevant to the `use<..>` bound.  Captured parameters may not be duplicated.  For now, only one `use<..>` bound may appear in a bounds list.  It may appear anywhere within the bounds list.

### How does this differ from the RFC?

This stabilization differs from the RFC in one respect: the RFC originally specified `use<'a, T>` as syntactically part of the RPIT type itself, e.g.:

```rust
fn capture<'a>() -> impl use<'a> Sized {}
```

However, settling on the final syntax was left as an open question.  T-lang later decided via FCP in [rust-lang#125836](rust-lang#125836) to treat `use<..>` as a syntactic bound instead, e.g.:

```rust
fn capture<'a>() -> impl Sized + use<'a> {}
```

### What aren't we stabilizing?

The key goal of this PR is to stabilize the parts of *precise capturing* that are needed to enable the migration to Rust 2024.

There are some capabilities of *precise capturing* that the RFC specifies but that we're not stabilizing here, as these require further work on the type system.  We hope to lift these limitations later.

The limitations that are part of this PR were specified in the [RFC's stabilization strategy](https://rust-lang.github.io/rfcs/3617-precise-capturing.html#stabilization-strategy).

#### Not capturing type or const parameters

The RFC addresses the overcapturing of type and const parameters; that is, it allows for them to not be captured in opaque types.  We're not stabilizing that in this PR.  Since all in scope generic type and const parameters are implicitly captured in all editions, this is not needed for the migration to Rust 2024.

For now, when using `use<..>`, all in scope type and const parameters must be nameable (i.e., APIT cannot be used) and included as arguments.  For example, this is an error because `T` is in scope and not included as an argument:

```rust
fn test<T>() -> impl Sized + use<> {}
//~^ ERROR `impl Trait` must mention all type parameters in scope in `use<...>`
```

This is due to certain current limitations in the type system related to how generic parameters are represented as captured (i.e. bivariance) and how inference operates.

We hope to relax this in the future, and this stabilization is forward compatible with doing so.

#### Precise capturing for return-position impl Trait **in trait** (RPITIT)

The RFC specifies precise capturing for RPITIT.  We're not stabilizing that in this PR.  Since RPITIT already adheres to the Lifetime Capture Rules 2024, this isn't needed for the migration to Rust 2024.

The effect of this is that the anonymous associated types created by RPITITs must continue to capture all of the lifetime parameters in scope, e.g.:

```rust
trait Foo<'a> {
    fn test() -> impl Sized + use<Self>;
    //~^ ERROR `use<...>` precise capturing syntax is currently not allowed in return-position `impl Trait` in traits
}
```

To allow this involves a meaningful amount of type system work related to adding variance to GATs or reworking how generics are represented in RPITITs.  We plan to do this work separately from the stabilization.  See:

- rust-lang#124029

Supporting precise capturing for RPITIT will also require us to implement a new algorithm for detecting refining capture behavior.  This may involve looking through type parameters to detect cases where the impl Trait type in an implementation captures fewer lifetimes than the corresponding RPITIT in the trait definition, e.g.:

```rust
trait Foo {
    fn rpit() -> impl Sized + use<Self>;
}

impl<'a> Foo for &'a () {
    // This is "refining" due to not capturing `'a` which
    // is implied by the trait's `use<Self>`.
    fn rpit() -> impl Sized + use<>;

    // This is not "refining".
    fn rpit() -> impl Sized + use<'a>;
}
```

This stabilization is forward compatible with adding support for this later.

### The technical details

This bound is purely syntactical and does not lower to a [`Clause`](https://doc.rust-lang.org/1.79.0/nightly-rustc/rustc_middle/ty/type.ClauseKind.html) in the type system.  For the purposes of the type system (and for the types team's curiosity regarding this stabilization), we have no current need to represent this as a `ClauseKind`.

Since opaques already capture a variable set of lifetimes depending on edition and their syntactical position (e.g. RPIT vs RPITIT), a `use<..>` bound is just a way to explicitly rather than implicitly specify that set of lifetimes, and this only affects opaque type lowering from AST to HIR.

### FCP plan

While there's much discussion of the type system here, the feature in this PR is implemented internally as a transformation that happens before lowering to the type system layer.  We already support impl Trait types partially capturing the in scope lifetimes; we just currently only expose that implicitly.

So, in my (errs's) view as a types team member, there's nothing for types to weigh in on here with respect to the implementation being stabilized, and I'd suggest a lang-only proposed FCP (though we'll of course CC the team below).

### Authorship and acknowledgments

This stabilization report was coauthored by compiler-errors and TC.

TC would like to acknowledge the outstanding and speedy work that compiler-errors has done to make this feature happen.

compiler-errors thanks TC for authoring the RFC, for all of his involvement in this feature's development, and pushing the Rust 2024 edition forward.

### Open items

We're doing some things in parallel here.  In signaling the intention to stabilize, we want to uncover any latent issues so we can be sure they get addressed.  We want to give the maximum time for discussion here to happen by starting it while other remaining miscellaneous work proceeds.  That work includes:

- [x] Look into `syn` support.
  - dtolnay/syn#1677
  - dtolnay/syn#1707
- [x] Look into `rustfmt` support.
  - rust-lang#126754
- [x] Look into `rust-analyzer` support.
  - rust-lang/rust-analyzer#17598
  - rust-lang/rust-analyzer#17676
- [x] Look into `rustdoc` support.
  - rust-lang#127228
  - rust-lang#127632
  - rust-lang#127658
- [x] Suggest this feature to RfL (a known nightly user).
- [x] Add a chapter to the edition guide.
  - rust-lang/edition-guide#316
- [x] Update the Reference.
  - rust-lang/reference#1577

### (Selected) implementation history

* rust-lang/rfcs#3498
* rust-lang/rfcs#3617
* rust-lang#123468
* rust-lang#125836
* rust-lang#126049
* rust-lang#126753

Closes rust-lang#123432.

cc `@rust-lang/lang` `@rust-lang/types`

`@rustbot` labels +T-lang +I-lang-nominated +A-impl-trait +F-precise_capturing

Tracking:

- rust-lang#123432

----

For the compiler reviewer, I'll leave some inline comments about diagnostics fallout :^)

r? compiler
github-actions bot pushed a commit to rust-lang/miri that referenced this issue Aug 26, 2024
Stabilize opaque type precise capturing (RFC 3617)

This PR partially stabilizes opaque type *precise capturing*, which was specified in [RFC 3617](rust-lang/rfcs#3617), and whose syntax was amended by FCP in [#125836](rust-lang/rust#125836).

This feature, as stabilized here, gives us a way to explicitly specify the generic lifetime parameters that an RPIT-like opaque type captures.  This solves the problem of overcapturing, for lifetime parameters in these opaque types, and will allow the Lifetime Capture Rules 2024 ([RFC 3498](rust-lang/rfcs#3498)) to be fully stabilized for RPIT in Rust 2024.

### What are we stabilizing?

This PR stabilizes the use of a `use<'a, T>` bound in return-position impl Trait opaque types.  Such a bound fully specifies the set of generic parameters captured by the RPIT opaque type, entirely overriding the implicit default behavior.  E.g.:

```rust
fn does_not_capture<'a, 'b>() -> impl Sized + use<'a> {}
//                               ~~~~~~~~~~~~~~~~~~~~
//                This RPIT opaque type does not capture `'b`.
```

The way we would suggest thinking of `impl Trait` types *without* an explicit `use<..>` bound is that the `use<..>` bound has been *elided*, and that the bound is filled in automatically by the compiler according to the edition-specific capture rules.

All non-`'static` lifetime parameters, named (i.e. non-APIT) type parameters, and const parameters in scope are valid to name, including an elided lifetime if such a lifetime would also be valid in an outlives bound, e.g.:

```rust
fn elided(x: &u8) -> impl Sized + use<'_> { x }
```

Lifetimes must be listed before type and const parameters, but otherwise the ordering is not relevant to the `use<..>` bound.  Captured parameters may not be duplicated.  For now, only one `use<..>` bound may appear in a bounds list.  It may appear anywhere within the bounds list.

### How does this differ from the RFC?

This stabilization differs from the RFC in one respect: the RFC originally specified `use<'a, T>` as syntactically part of the RPIT type itself, e.g.:

```rust
fn capture<'a>() -> impl use<'a> Sized {}
```

However, settling on the final syntax was left as an open question.  T-lang later decided via FCP in [#125836](rust-lang/rust#125836) to treat `use<..>` as a syntactic bound instead, e.g.:

```rust
fn capture<'a>() -> impl Sized + use<'a> {}
```

### What aren't we stabilizing?

The key goal of this PR is to stabilize the parts of *precise capturing* that are needed to enable the migration to Rust 2024.

There are some capabilities of *precise capturing* that the RFC specifies but that we're not stabilizing here, as these require further work on the type system.  We hope to lift these limitations later.

The limitations that are part of this PR were specified in the [RFC's stabilization strategy](https://rust-lang.github.io/rfcs/3617-precise-capturing.html#stabilization-strategy).

#### Not capturing type or const parameters

The RFC addresses the overcapturing of type and const parameters; that is, it allows for them to not be captured in opaque types.  We're not stabilizing that in this PR.  Since all in scope generic type and const parameters are implicitly captured in all editions, this is not needed for the migration to Rust 2024.

For now, when using `use<..>`, all in scope type and const parameters must be nameable (i.e., APIT cannot be used) and included as arguments.  For example, this is an error because `T` is in scope and not included as an argument:

```rust
fn test<T>() -> impl Sized + use<> {}
//~^ ERROR `impl Trait` must mention all type parameters in scope in `use<...>`
```

This is due to certain current limitations in the type system related to how generic parameters are represented as captured (i.e. bivariance) and how inference operates.

We hope to relax this in the future, and this stabilization is forward compatible with doing so.

#### Precise capturing for return-position impl Trait **in trait** (RPITIT)

The RFC specifies precise capturing for RPITIT.  We're not stabilizing that in this PR.  Since RPITIT already adheres to the Lifetime Capture Rules 2024, this isn't needed for the migration to Rust 2024.

The effect of this is that the anonymous associated types created by RPITITs must continue to capture all of the lifetime parameters in scope, e.g.:

```rust
trait Foo<'a> {
    fn test() -> impl Sized + use<Self>;
    //~^ ERROR `use<...>` precise capturing syntax is currently not allowed in return-position `impl Trait` in traits
}
```

To allow this involves a meaningful amount of type system work related to adding variance to GATs or reworking how generics are represented in RPITITs.  We plan to do this work separately from the stabilization.  See:

- rust-lang/rust#124029

Supporting precise capturing for RPITIT will also require us to implement a new algorithm for detecting refining capture behavior.  This may involve looking through type parameters to detect cases where the impl Trait type in an implementation captures fewer lifetimes than the corresponding RPITIT in the trait definition, e.g.:

```rust
trait Foo {
    fn rpit() -> impl Sized + use<Self>;
}

impl<'a> Foo for &'a () {
    // This is "refining" due to not capturing `'a` which
    // is implied by the trait's `use<Self>`.
    fn rpit() -> impl Sized + use<>;

    // This is not "refining".
    fn rpit() -> impl Sized + use<'a>;
}
```

This stabilization is forward compatible with adding support for this later.

### The technical details

This bound is purely syntactical and does not lower to a [`Clause`](https://doc.rust-lang.org/1.79.0/nightly-rustc/rustc_middle/ty/type.ClauseKind.html) in the type system.  For the purposes of the type system (and for the types team's curiosity regarding this stabilization), we have no current need to represent this as a `ClauseKind`.

Since opaques already capture a variable set of lifetimes depending on edition and their syntactical position (e.g. RPIT vs RPITIT), a `use<..>` bound is just a way to explicitly rather than implicitly specify that set of lifetimes, and this only affects opaque type lowering from AST to HIR.

### FCP plan

While there's much discussion of the type system here, the feature in this PR is implemented internally as a transformation that happens before lowering to the type system layer.  We already support impl Trait types partially capturing the in scope lifetimes; we just currently only expose that implicitly.

So, in my (errs's) view as a types team member, there's nothing for types to weigh in on here with respect to the implementation being stabilized, and I'd suggest a lang-only proposed FCP (though we'll of course CC the team below).

### Authorship and acknowledgments

This stabilization report was coauthored by compiler-errors and TC.

TC would like to acknowledge the outstanding and speedy work that compiler-errors has done to make this feature happen.

compiler-errors thanks TC for authoring the RFC, for all of his involvement in this feature's development, and pushing the Rust 2024 edition forward.

### Open items

We're doing some things in parallel here.  In signaling the intention to stabilize, we want to uncover any latent issues so we can be sure they get addressed.  We want to give the maximum time for discussion here to happen by starting it while other remaining miscellaneous work proceeds.  That work includes:

- [x] Look into `syn` support.
  - dtolnay/syn#1677
  - dtolnay/syn#1707
- [x] Look into `rustfmt` support.
  - rust-lang/rust#126754
- [x] Look into `rust-analyzer` support.
  - rust-lang/rust-analyzer#17598
  - rust-lang/rust-analyzer#17676
- [x] Look into `rustdoc` support.
  - rust-lang/rust#127228
  - rust-lang/rust#127632
  - rust-lang/rust#127658
- [x] Suggest this feature to RfL (a known nightly user).
- [x] Add a chapter to the edition guide.
  - rust-lang/edition-guide#316
- [x] Update the Reference.
  - rust-lang/reference#1577

### (Selected) implementation history

* rust-lang/rfcs#3498
* rust-lang/rfcs#3617
* rust-lang/rust#123468
* rust-lang/rust#125836
* rust-lang/rust#126049
* rust-lang/rust#126753

Closes #123432.

cc `@rust-lang/lang` `@rust-lang/types`

`@rustbot` labels +T-lang +I-lang-nominated +A-impl-trait +F-precise_capturing

Tracking:

- rust-lang/rust#123432

----

For the compiler reviewer, I'll leave some inline comments about diagnostics fallout :^)

r? compiler
@QuineDot
Copy link

A flavor of bound that loosens restrictions and cannot be mentioned more than once is not unprecedented. ?Trait bounds do exactly that.

// Compiles (no warning)
fn example0<'a>() -> &'a (impl Debug + ?Sized) {
    ""
}

// Compiles (warning: relaxing a default bound only does something for `?Sized`)
fn example1<'a>() -> &'a (impl Debug + ?Display) {
    &0
}

// Fails: error[E0203]: type parameter has more than one relaxed default bound,
// only one is supported
fn example2<'a>() -> &'a (impl Debug + ?Display + ?Sized) {
    ""
}

Playground.

@bsodmike
Copy link

bsodmike commented Sep 8, 2024

I'm not sure if anyone has raised this, but maybe this error reporting could be improved to also say "consider adding + use<> rather than just complaining about an immutable borrow?

error[E0502]: cannot borrow `data` as mutable because it is also borrowed as immutable
 --> src/main.rs:6:5
  |
5 |     let mut i = indices(&data);
  |                         ----- immutable borrow occurs here
6 |     data.push(4);
  |     ^^^^^^^^^^^^ mutable borrow occurs here
7 |     i.next();
  |     - immutable borrow later used here

Instead,

error[E0502]: cannot borrow `data` as mutable because it is also borrowed as immutable
 --> src/main.rs:6:5
  |
5 |     let mut i = indices(&data);
  |                         ----- immutable borrow occurs here
6 |     data.push(4);
  |     ^^^^^^^^^^^^ mutable borrow occurs here
7 |     i.next();
  |     - immutable borrow later used here

10 | fn indices<T>(
11 |     slice: &[T],
12 | ) -> impl Iterator<Item = usize> + use<> {
  |                                     ----- `impl Trait` must mention all type parameters in scope in `use<...>`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. F-precise_capturing `#![feature(precise_capturing)]` finished-final-comment-period The final comment period is finished for this PR / Issue. T-lang Relevant to the language team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests