Give RPITITs variances, so they can (not) capture lifetimes #124029

compiler-errors · 2024-04-16T16:39:50Z

This is needed for edition 2021 and the precise_captures feature, so we can specify that we don't want to capture a lifetime implicitly for an RPITIT. I'll elaborate this further and try to motivate its soundness once perf comes back.

Tracking:

Tracking issue for precise_capturing syntax #123432

r? @ghost

rustbot · 2024-04-16T16:39:54Z

changes to the core type system

cc @compiler-errors, @lcnr

compiler-errors · 2024-04-16T16:39:57Z

@bors try @rust-timer queue

bors · 2024-04-16T16:41:08Z

⌛ Trying commit 8ba97b6 with merge 957db6d...

…<try> Give RPITITs variances, so they can (not) capture lifetimes This is needed for edition 2021 and the `precise_captures` feature, so we can specify that we *don't* want to capture a lifetime implicitly for an RPITIT. I'll elaborate this further and try to motivate its soundness once perf comes back. r? `@ghost`

bors · 2024-04-16T18:14:29Z

☀️ Try build successful - checks-actions
Build commit: 957db6d (957db6d1de288cd9a0bf37d3dd379b44a889dc33)

rust-timer · 2024-04-16T19:29:33Z

Finished benchmarking commit (957db6d): comparison URL.

Overall result: ❌✅ regressions and improvements - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.2%, 0.2%]	2
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.2%	[-1.2%, -1.2%]	1
All ❌✅ (primary)	0.2%	[0.2%, 0.2%]	2

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.1%	[2.1%, 2.1%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-3.2%	[-6.2%, -2.3%]	7
All ❌✅ (primary)	2.1%	[2.1%, 2.1%]	1

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 679.507s -> 678.77s (-0.11%)
Artifact size: 316.08 MiB -> 316.06 MiB (-0.01%)

wesleywiser · 2024-06-27T14:41:46Z

Hi @compiler-errors, is this ready to be reviewed? Should we assign a reviewer?

apiraino · 2024-07-02T09:50:13Z

Discussed on Zulip. Removing waiting on review flag

@rustbot label -S-waiting-on-review

lqd · 2024-07-02T13:31:06Z

Make it a draft and/or waiting on author, I dunno 😆

bors · 2024-07-08T09:23:22Z

☔ The latest upstream changes (presumably #127476) made this pull request unmergeable. Please resolve the merge conflicts.

…=spastorino Stabilize opaque type precise capturing (RFC 3617) This PR partially stabilizes opaque type *precise capturing*, which was specified in [RFC 3617](rust-lang/rfcs#3617), and whose syntax was amended by FCP in [rust-lang#125836](rust-lang#125836). This feature, as stabilized here, gives us a way to explicitly specify the generic lifetime parameters that an RPIT-like opaque type captures. This solves the problem of overcapturing, for lifetime parameters in these opaque types, and will allow the Lifetime Capture Rules 2024 ([RFC 3498](rust-lang/rfcs#3498)) to be fully stabilized for RPIT in Rust 2024. ### What are we stabilizing? This PR stabilizes the use of a `use<'a, T>` bound in return-position impl Trait opaque types. Such a bound fully specifies the set of generic parameters captured by the RPIT opaque type, entirely overriding the implicit default behavior. E.g.: ```rust fn does_not_capture<'a, 'b>() -> impl Sized + use<'a> {} // ~~~~~~~~~~~~~~~~~~~~ // This RPIT opaque type does not capture `'b`. ``` The way we would suggest thinking of `impl Trait` types *without* an explicit `use<..>` bound is that the `use<..>` bound has been *elided*, and that the bound is filled in automatically by the compiler according to the edition-specific capture rules. All non-`'static` lifetime parameters, named (i.e. non-APIT) type parameters, and const parameters in scope are valid to name, including an elided lifetime if such a lifetime would also be valid in an outlives bound, e.g.: ```rust fn elided(x: &u8) -> impl Sized + use<'_> { x } ``` Lifetimes must be listed before type and const parameters, but otherwise the ordering is not relevant to the `use<..>` bound. Captured parameters may not be duplicated. For now, only one `use<..>` bound may appear in a bounds list. It may appear anywhere within the bounds list. ### How does this differ from the RFC? This stabilization differs from the RFC in one respect: the RFC originally specified `use<'a, T>` as syntactically part of the RPIT type itself, e.g.: ```rust fn capture<'a>() -> impl use<'a> Sized {} ``` However, settling on the final syntax was left as an open question. T-lang later decided via FCP in [rust-lang#125836](rust-lang#125836) to treat `use<..>` as a syntactic bound instead, e.g.: ```rust fn capture<'a>() -> impl Sized + use<'a> {} ``` ### What aren't we stabilizing? The key goal of this PR is to stabilize the parts of *precise capturing* that are needed to enable the migration to Rust 2024. There are some capabilities of *precise capturing* that the RFC specifies but that we're not stabilizing here, as these require further work on the type system. We hope to lift these limitations later. The limitations that are part of this PR were specified in the [RFC's stabilization strategy](https://rust-lang.github.io/rfcs/3617-precise-capturing.html#stabilization-strategy). #### Not capturing type or const parameters The RFC addresses the overcapturing of type and const parameters; that is, it allows for them to not be captured in opaque types. We're not stabilizing that in this PR. Since all in scope generic type and const parameters are implicitly captured in all editions, this is not needed for the migration to Rust 2024. For now, when using `use<..>`, all in scope type and const parameters must be nameable (i.e., APIT cannot be used) and included as arguments. For example, this is an error because `T` is in scope and not included as an argument: ```rust fn test<T>() -> impl Sized + use<> {} //~^ ERROR `impl Trait` must mention all type parameters in scope in `use<...>` ``` This is due to certain current limitations in the type system related to how generic parameters are represented as captured (i.e. bivariance) and how inference operates. We hope to relax this in the future, and this stabilization is forward compatible with doing so. #### Precise capturing for return-position impl Trait **in trait** (RPITIT) The RFC specifies precise capturing for RPITIT. We're not stabilizing that in this PR. Since RPITIT already adheres to the Lifetime Capture Rules 2024, this isn't needed for the migration to Rust 2024. The effect of this is that the anonymous associated types created by RPITITs must continue to capture all of the lifetime parameters in scope, e.g.: ```rust trait Foo<'a> { fn test() -> impl Sized + use<Self>; //~^ ERROR `use<...>` precise capturing syntax is currently not allowed in return-position `impl Trait` in traits } ``` To allow this involves a meaningful amount of type system work related to adding variance to GATs or reworking how generics are represented in RPITITs. We plan to do this work separately from the stabilization. See: - rust-lang#124029 Supporting precise capturing for RPITIT will also require us to implement a new algorithm for detecting refining capture behavior. This may involve looking through type parameters to detect cases where the impl Trait type in an implementation captures fewer lifetimes than the corresponding RPITIT in the trait definition, e.g.: ```rust trait Foo { fn rpit() -> impl Sized + use<Self>; } impl<'a> Foo for &'a () { // This is "refining" due to not capturing `'a` which // is implied by the trait's `use<Self>`. fn rpit() -> impl Sized + use<>; // This is not "refining". fn rpit() -> impl Sized + use<'a>; } ``` This stabilization is forward compatible with adding support for this later. ### The technical details This bound is purely syntactical and does not lower to a [`Clause`](https://doc.rust-lang.org/1.79.0/nightly-rustc/rustc_middle/ty/type.ClauseKind.html) in the type system. For the purposes of the type system (and for the types team's curiosity regarding this stabilization), we have no current need to represent this as a `ClauseKind`. Since opaques already capture a variable set of lifetimes depending on edition and their syntactical position (e.g. RPIT vs RPITIT), a `use<..>` bound is just a way to explicitly rather than implicitly specify that set of lifetimes, and this only affects opaque type lowering from AST to HIR. ### FCP plan While there's much discussion of the type system here, the feature in this PR is implemented internally as a transformation that happens before lowering to the type system layer. We already support impl Trait types partially capturing the in scope lifetimes; we just currently only expose that implicitly. So, in my (errs's) view as a types team member, there's nothing for types to weigh in on here with respect to the implementation being stabilized, and I'd suggest a lang-only proposed FCP (though we'll of course CC the team below). ### Authorship and acknowledgments This stabilization report was coauthored by compiler-errors and TC. TC would like to acknowledge the outstanding and speedy work that compiler-errors has done to make this feature happen. compiler-errors thanks TC for authoring the RFC, for all of his involvement in this feature's development, and pushing the Rust 2024 edition forward. ### Open items We're doing some things in parallel here. In signaling the intention to stabilize, we want to uncover any latent issues so we can be sure they get addressed. We want to give the maximum time for discussion here to happen by starting it while other remaining miscellaneous work proceeds. That work includes: - [x] Look into `syn` support. - dtolnay/syn#1677 - dtolnay/syn#1707 - [x] Look into `rustfmt` support. - rust-lang#126754 - [x] Look into `rust-analyzer` support. - rust-lang/rust-analyzer#17598 - rust-lang/rust-analyzer#17676 - [x] Look into `rustdoc` support. - rust-lang#127228 - rust-lang#127632 - rust-lang#127658 - [x] Suggest this feature to RfL (a known nightly user). - [x] Add a chapter to the edition guide. - rust-lang/edition-guide#316 - [x] Update the Reference. - rust-lang/reference#1577 ### (Selected) implementation history * rust-lang/rfcs#3498 * rust-lang/rfcs#3617 * rust-lang#123468 * rust-lang#125836 * rust-lang#126049 * rust-lang#126753 Closes rust-lang#123432. cc `@rust-lang/lang` `@rust-lang/types` `@rustbot` labels +T-lang +I-lang-nominated +A-impl-trait +F-precise_capturing Tracking: - rust-lang#123432 ---- For the compiler reviewer, I'll leave some inline comments about diagnostics fallout :^) r? compiler

Stabilize opaque type precise capturing (RFC 3617) This PR partially stabilizes opaque type *precise capturing*, which was specified in [RFC 3617](rust-lang/rfcs#3617), and whose syntax was amended by FCP in [#125836](rust-lang/rust#125836). This feature, as stabilized here, gives us a way to explicitly specify the generic lifetime parameters that an RPIT-like opaque type captures. This solves the problem of overcapturing, for lifetime parameters in these opaque types, and will allow the Lifetime Capture Rules 2024 ([RFC 3498](rust-lang/rfcs#3498)) to be fully stabilized for RPIT in Rust 2024. ### What are we stabilizing? This PR stabilizes the use of a `use<'a, T>` bound in return-position impl Trait opaque types. Such a bound fully specifies the set of generic parameters captured by the RPIT opaque type, entirely overriding the implicit default behavior. E.g.: ```rust fn does_not_capture<'a, 'b>() -> impl Sized + use<'a> {} // ~~~~~~~~~~~~~~~~~~~~ // This RPIT opaque type does not capture `'b`. ``` The way we would suggest thinking of `impl Trait` types *without* an explicit `use<..>` bound is that the `use<..>` bound has been *elided*, and that the bound is filled in automatically by the compiler according to the edition-specific capture rules. All non-`'static` lifetime parameters, named (i.e. non-APIT) type parameters, and const parameters in scope are valid to name, including an elided lifetime if such a lifetime would also be valid in an outlives bound, e.g.: ```rust fn elided(x: &u8) -> impl Sized + use<'_> { x } ``` Lifetimes must be listed before type and const parameters, but otherwise the ordering is not relevant to the `use<..>` bound. Captured parameters may not be duplicated. For now, only one `use<..>` bound may appear in a bounds list. It may appear anywhere within the bounds list. ### How does this differ from the RFC? This stabilization differs from the RFC in one respect: the RFC originally specified `use<'a, T>` as syntactically part of the RPIT type itself, e.g.: ```rust fn capture<'a>() -> impl use<'a> Sized {} ``` However, settling on the final syntax was left as an open question. T-lang later decided via FCP in [#125836](rust-lang/rust#125836) to treat `use<..>` as a syntactic bound instead, e.g.: ```rust fn capture<'a>() -> impl Sized + use<'a> {} ``` ### What aren't we stabilizing? The key goal of this PR is to stabilize the parts of *precise capturing* that are needed to enable the migration to Rust 2024. There are some capabilities of *precise capturing* that the RFC specifies but that we're not stabilizing here, as these require further work on the type system. We hope to lift these limitations later. The limitations that are part of this PR were specified in the [RFC's stabilization strategy](https://rust-lang.github.io/rfcs/3617-precise-capturing.html#stabilization-strategy). #### Not capturing type or const parameters The RFC addresses the overcapturing of type and const parameters; that is, it allows for them to not be captured in opaque types. We're not stabilizing that in this PR. Since all in scope generic type and const parameters are implicitly captured in all editions, this is not needed for the migration to Rust 2024. For now, when using `use<..>`, all in scope type and const parameters must be nameable (i.e., APIT cannot be used) and included as arguments. For example, this is an error because `T` is in scope and not included as an argument: ```rust fn test<T>() -> impl Sized + use<> {} //~^ ERROR `impl Trait` must mention all type parameters in scope in `use<...>` ``` This is due to certain current limitations in the type system related to how generic parameters are represented as captured (i.e. bivariance) and how inference operates. We hope to relax this in the future, and this stabilization is forward compatible with doing so. #### Precise capturing for return-position impl Trait **in trait** (RPITIT) The RFC specifies precise capturing for RPITIT. We're not stabilizing that in this PR. Since RPITIT already adheres to the Lifetime Capture Rules 2024, this isn't needed for the migration to Rust 2024. The effect of this is that the anonymous associated types created by RPITITs must continue to capture all of the lifetime parameters in scope, e.g.: ```rust trait Foo<'a> { fn test() -> impl Sized + use<Self>; //~^ ERROR `use<...>` precise capturing syntax is currently not allowed in return-position `impl Trait` in traits } ``` To allow this involves a meaningful amount of type system work related to adding variance to GATs or reworking how generics are represented in RPITITs. We plan to do this work separately from the stabilization. See: - rust-lang/rust#124029 Supporting precise capturing for RPITIT will also require us to implement a new algorithm for detecting refining capture behavior. This may involve looking through type parameters to detect cases where the impl Trait type in an implementation captures fewer lifetimes than the corresponding RPITIT in the trait definition, e.g.: ```rust trait Foo { fn rpit() -> impl Sized + use<Self>; } impl<'a> Foo for &'a () { // This is "refining" due to not capturing `'a` which // is implied by the trait's `use<Self>`. fn rpit() -> impl Sized + use<>; // This is not "refining". fn rpit() -> impl Sized + use<'a>; } ``` This stabilization is forward compatible with adding support for this later. ### The technical details This bound is purely syntactical and does not lower to a [`Clause`](https://doc.rust-lang.org/1.79.0/nightly-rustc/rustc_middle/ty/type.ClauseKind.html) in the type system. For the purposes of the type system (and for the types team's curiosity regarding this stabilization), we have no current need to represent this as a `ClauseKind`. Since opaques already capture a variable set of lifetimes depending on edition and their syntactical position (e.g. RPIT vs RPITIT), a `use<..>` bound is just a way to explicitly rather than implicitly specify that set of lifetimes, and this only affects opaque type lowering from AST to HIR. ### FCP plan While there's much discussion of the type system here, the feature in this PR is implemented internally as a transformation that happens before lowering to the type system layer. We already support impl Trait types partially capturing the in scope lifetimes; we just currently only expose that implicitly. So, in my (errs's) view as a types team member, there's nothing for types to weigh in on here with respect to the implementation being stabilized, and I'd suggest a lang-only proposed FCP (though we'll of course CC the team below). ### Authorship and acknowledgments This stabilization report was coauthored by compiler-errors and TC. TC would like to acknowledge the outstanding and speedy work that compiler-errors has done to make this feature happen. compiler-errors thanks TC for authoring the RFC, for all of his involvement in this feature's development, and pushing the Rust 2024 edition forward. ### Open items We're doing some things in parallel here. In signaling the intention to stabilize, we want to uncover any latent issues so we can be sure they get addressed. We want to give the maximum time for discussion here to happen by starting it while other remaining miscellaneous work proceeds. That work includes: - [x] Look into `syn` support. - dtolnay/syn#1677 - dtolnay/syn#1707 - [x] Look into `rustfmt` support. - rust-lang/rust#126754 - [x] Look into `rust-analyzer` support. - rust-lang/rust-analyzer#17598 - rust-lang/rust-analyzer#17676 - [x] Look into `rustdoc` support. - rust-lang/rust#127228 - rust-lang/rust#127632 - rust-lang/rust#127658 - [x] Suggest this feature to RfL (a known nightly user). - [x] Add a chapter to the edition guide. - rust-lang/edition-guide#316 - [x] Update the Reference. - rust-lang/reference#1577 ### (Selected) implementation history * rust-lang/rfcs#3498 * rust-lang/rfcs#3617 * rust-lang/rust#123468 * rust-lang/rust#125836 * rust-lang/rust#126049 * rust-lang/rust#126753 Closes #123432. cc `@rust-lang/lang` `@rust-lang/types` `@rustbot` labels +T-lang +I-lang-nominated +A-impl-trait +F-precise_capturing Tracking: - rust-lang/rust#123432 ---- For the compiler reviewer, I'll leave some inline comments about diagnostics fallout :^) r? compiler

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Apr 16, 2024

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Apr 16, 2024

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Apr 16, 2024

compiler-errors added 3 commits June 28, 2024 11:00

Give RPITITs variance for precise capturing

b56a74f

Allow use<> syntax in RPITIT, fix test example

9b53b64

Tests, suppress refine lint when errors

7b0e962

compiler-errors force-pushed the rpitit-variances branch from 8ba97b6 to 7b0e962 Compare June 28, 2024 20:15

rustbot removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 2, 2024

traviscross mentioned this pull request Jul 2, 2024

Tracking issue for precise_capturing syntax #123432

Closed

14 tasks

traviscross added the F-precise_capturing `#![feature(precise_capturing)]` label Jul 2, 2024

bors added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Jul 8, 2024

compiler-errors mentioned this pull request Jul 13, 2024

Stabilize opaque type precise capturing (RFC 3617) #127672

Merged

7 tasks

Dylan-DPC marked this pull request as draft August 14, 2024 18:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Give RPITITs variances, so they can (not) capture lifetimes #124029

Give RPITITs variances, so they can (not) capture lifetimes #124029

compiler-errors commented Apr 16, 2024 •

edited by traviscross

Loading

rustbot commented Apr 16, 2024

compiler-errors commented Apr 16, 2024

This comment has been minimized.

bors commented Apr 16, 2024

bors commented Apr 16, 2024

This comment has been minimized.

rust-timer commented Apr 16, 2024

wesleywiser commented Jun 27, 2024

apiraino commented Jul 2, 2024

lqd commented Jul 2, 2024 •

edited

Loading

bors commented Jul 8, 2024

Give RPITITs variances, so they can (not) capture lifetimes #124029

Are you sure you want to change the base?

Give RPITITs variances, so they can (not) capture lifetimes #124029

Conversation

compiler-errors commented Apr 16, 2024 • edited by traviscross Loading

rustbot commented Apr 16, 2024

compiler-errors commented Apr 16, 2024

This comment has been minimized.

bors commented Apr 16, 2024

bors commented Apr 16, 2024

This comment has been minimized.

rust-timer commented Apr 16, 2024

Overall result: ❌✅ regressions and improvements - no action needed

Instruction count

Max RSS (memory usage)

Cycles

Binary size

wesleywiser commented Jun 27, 2024

apiraino commented Jul 2, 2024

lqd commented Jul 2, 2024 • edited Loading

bors commented Jul 8, 2024

compiler-errors commented Apr 16, 2024 •

edited by traviscross

Loading

lqd commented Jul 2, 2024 •

edited

Loading