From a45899605bd2be1a676cc758307cd725b6808660 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Tue, 4 Jun 2019 20:07:30 -0700 Subject: [PATCH 01/27] first draft of pointer to fields --- text/0000-ptr-to-field.md | 189 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 189 insertions(+) create mode 100644 text/0000-ptr-to-field.md diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md new file mode 100644 index 00000000000..95e34dcbf5b --- /dev/null +++ b/text/0000-ptr-to-field.md @@ -0,0 +1,189 @@ +- Feature Name: (fill me in with a unique ident, `my_awesome_feature`) +- Start Date: (fill me in with today's date, YYYY-MM-DD) +- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) + +# Summary +[summary]: #summary + +This feature could serve as the backbone for some pointer to field syntax, and even if no syntax is made, this feature serves as a safe generic way to talk about types and their fields. + +# Motivation +[motivation]: #motivation + +The motivation for this feature is to allow safe projection through smart pointers, for example `Pin<&mut T>` to `Pin<&mut Field>`. This is a much needed feature to make `Pin

` more usable in safe-contexts, without the need to use unsafe to map to a field. This also can allow projection through other smart pointers like `Rc`, `Arc`. This feature cannot be implemented as a library effectively because it depends on the layouts of types, so it requires integration with the Rust compiler until Rust gets a stable layout (which may never happen). + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +First the core trait, type, and functions that need to be added + +```rust +/// Contains metadata about how to get to the field from the parent using raw pointers +/// This is an opaque type that never needs to be stablized, and it only an implementation detail +struct MetaData { + ... +} + +/// The compiler should prevent user implementations for `Field`, +/// i.e. only the compiler is allowed to implement `Field` +/// This is to prevent people from creating fake "fields" +trait Field { + /// The type that the field is a part of + type Parent; + /// The type of the field + type Type; + + /// The metadata required to get to the field using raw pointers + const META: MetaData; +} + +trait Project { + /// The projected version of Self + type Projection; + + fn project(self, field: F) -> Self::Projection; +} + +impl *const T { + unsafe fn project_unchecked>(self, field: F) -> *const F::Type { + // make the field pointer, this code is allowed to assume that + // self points to a valid instance of T + ... + } +} + +impl *mut T { + unsafe fn project_unchecked>(self, field: F) -> *mut F::Type { + // make the field pointer, this code is allowed to assume that + // self points to a valid instance of T + ... + } +} +``` + +Now we need some syntax to refer to the fields of types. Some ideas for the syntax are + +* `Parent.field` +* `Parent::field` // bad as it conflicts with associated functions +* `Parent~field` // or any other sigil + +We will call these field types, because they will desugar to a unit type that correctly implements `Field`, like so + +```rust +struct Foo { + bar: Bar +} + +struct Foo.bar; + +impl Field for Foo.bar { ... } +``` + +These are the core parts of this proposal. Every other part of this proposal can be postponed or dropped without affecting this feature's core principles. + +Using these core parts we can build as a library projections through `Pin<&T>`, `Rc<_>` and more. We can then use this to safely project through smart pointers like so. + +```rust +let foo : Pin = Box::pin(immovable); +let foo : Pin<&mut Foo> = foo.as_mut(); +let field : Pin<&mut Field> = foo.project(Foo.field); +``` +But to do safe pin projections we will need to introduce a marker trait. +```rust +/// The only people who can implement `PinProjectable` are the creator of the parent type +/// This allows people to opt-in to allowing their fields to be pin projectable. +/// The guarantee is that once you create `Pin>`, all of the same guarantees that +/// apply to `Pin>` also apply to `Pin>` +/// For all `Parent: Unpin`, these can be auto implemented for all of their fields. +unsafe trait PinProjectable: Field {} +``` + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +The field types needs to interact with the privacy rules for fields. A field type has the same privacy as the field it is derived from. Anything else would be too restrictive or unsound. + +As example of how to implement `Project`, here is the implementation for `&T`. + +```rust +impl<'a, F: Field> Project for &'a F::Parent where F::Type: 'a { + type Projection = &'a F::Type; + + fn project(self, field: F) -> Self { + unsafe { + // This is safe because a reference is always valids + let ptr: *const F::Type = (self as *const F::Parent).project_unchecked(field); + + &*ptr + } + } +} +``` + + + +This is the technical portion of the RFC. Explain the design in sufficient detail that: + +- Its interaction with other features is clear. +- It is reasonably clear how the feature would be implemented. +- Corner cases are dissected by example. + +The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work. + +# Drawbacks +[drawbacks]: #drawbacks + +Why should we *not* do this? + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +- Why is this design the best in the space of possible designs? +- What other designs have been considered and what is the rationale for not choosing them? +- What is the impact of not doing this? + +# Prior art +[prior-art]: #prior-art + +Discuss prior art, both the good and the bad, in relation to this proposal. +A few examples of what this can include are: + +- For language, library, cargo, tools, and compiler proposals: Does this feature exist in other programming languages and what experience have their community had? +- For community proposals: Is this done by some other community and what were their experiences with it? +- For other teams: What lessons can we learn from what other communities have done here? +- Papers: Are there any published papers or great posts that discuss this? If you have some relevant papers to refer to, this can serve as a more detailed theoretical background. + +This section is intended to encourage you as an author to think about the lessons from other languages, provide readers of your RFC with a fuller picture. +If there is no prior art, that is fine - your ideas are interesting to us whether they are brand new or if it is an adaptation from other languages. + +Note that while precedent set by other languages is some motivation, it does not on its own motivate an RFC. +Please also take into consideration that rust sometimes intentionally diverges from common language features. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +- What parts of the design do you expect to resolve through the RFC process before this gets merged? +- What parts of the design do you expect to resolve through the implementation of this feature before stabilization? +- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? + +# Future possibilities +[future-possibilities]: #future-possibilities + +Think about what the natural extension and evolution of your proposal would +be and how it would affect the language and project as a whole in a holistic +way. Try to use this section as a tool to more fully consider all possible +interactions with the project and language in your proposal. +Also consider how the this all fits into the roadmap for the project +and of the relevant sub-team. + +This is also a good place to "dump ideas", if they are out of scope for the +RFC you are writing but otherwise related. + +If you have tried and cannot think of any future possibilities, +you may simply state that you cannot think of anything. + +Note that having something written down in the future-possibilities section +is not a reason to accept the current or a future RFC; such notes should be +in the section on motivation or rationale in this or subsequent RFCs. +The section merely provides additional information. From 7eb7ea4e91d9635a9bf2eb2e5a0b0423da331bc4 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Wed, 5 Jun 2019 15:26:18 -0700 Subject: [PATCH 02/27] added to the drawbacks and later sections --- text/0000-ptr-to-field.md | 47 ++++++++------------------------------- 1 file changed, 9 insertions(+), 38 deletions(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 95e34dcbf5b..9918dbe5467 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -121,8 +121,6 @@ impl<'a, F: Field> Project for &'a F::Parent where F::Type: 'a { } ``` - - This is the technical portion of the RFC. Explain the design in sufficient detail that: - Its interaction with other features is clear. @@ -135,55 +133,28 @@ The section should return to the examples given in the previous section, and exp [drawbacks]: #drawbacks Why should we *not* do this? +- This adds quite a bit of complexity and can increase compile times # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives -- Why is this design the best in the space of possible designs? -- What other designs have been considered and what is the rationale for not choosing them? -- What is the impact of not doing this? +- The `&[mut] raw T` could solve some of the problems, but only for raw pointers. It doesn't help with abstractions. +- Somehow expand on `Deref` to allow dereferencing to a smart pointer + - This would require Generic Associated Types at the very least, and maybe some other features like assocaited traits # Prior art [prior-art]: #prior-art -Discuss prior art, both the good and the bad, in relation to this proposal. -A few examples of what this can include are: - -- For language, library, cargo, tools, and compiler proposals: Does this feature exist in other programming languages and what experience have their community had? -- For community proposals: Is this done by some other community and what were their experiences with it? -- For other teams: What lessons can we learn from what other communities have done here? -- Papers: Are there any published papers or great posts that discuss this? If you have some relevant papers to refer to, this can serve as a more detailed theoretical background. - -This section is intended to encourage you as an author to think about the lessons from other languages, provide readers of your RFC with a fuller picture. -If there is no prior art, that is fine - your ideas are interesting to us whether they are brand new or if it is an adaptation from other languages. - -Note that while precedent set by other languages is some motivation, it does not on its own motivate an RFC. -Please also take into consideration that rust sometimes intentionally diverges from common language features. +- C++'s pointer to members `Parent::*field` +- Java's `class Field` + - Similar reflection capabilies in other languages # Unresolved questions [unresolved-questions]: #unresolved-questions -- What parts of the design do you expect to resolve through the RFC process before this gets merged? -- What parts of the design do you expect to resolve through the implementation of this feature before stabilization? -- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? +- Syntax for the type fields (not to be decided before accepting this RFC, but must be decided before stabilization) # Future possibilities [future-possibilities]: #future-possibilities -Think about what the natural extension and evolution of your proposal would -be and how it would affect the language and project as a whole in a holistic -way. Try to use this section as a tool to more fully consider all possible -interactions with the project and language in your proposal. -Also consider how the this all fits into the roadmap for the project -and of the relevant sub-team. - -This is also a good place to "dump ideas", if they are out of scope for the -RFC you are writing but otherwise related. - -If you have tried and cannot think of any future possibilities, -you may simply state that you cannot think of anything. - -Note that having something written down in the future-possibilities section -is not a reason to accept the current or a future RFC; such notes should be -in the section on motivation or rationale in this or subsequent RFCs. -The section merely provides additional information. +- Extend the `Project` trait to implement all smart pointers in the standard library From 7ada603500ae940a96ee4f2a77694f9cbc4a6d72 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Wed, 5 Jun 2019 15:54:43 -0700 Subject: [PATCH 03/27] expanded up reference level explanation --- text/0000-ptr-to-field.md | 54 ++++++++++++++++++++++++++++++++++----- 1 file changed, 48 insertions(+), 6 deletions(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 9918dbe5467..7c1bb79ec60 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -104,6 +104,8 @@ unsafe trait PinProjectable: Field {} The field types needs to interact with the privacy rules for fields. A field type has the same privacy as the field it is derived from. Anything else would be too restrictive or unsound. +The project trait will live inside the `std::project` module, and will not be added to `prelude`. + As example of how to implement `Project`, here is the implementation for `&T`. ```rust @@ -121,13 +123,46 @@ impl<'a, F: Field> Project for &'a F::Parent where F::Type: 'a { } ``` -This is the technical portion of the RFC. Explain the design in sufficient detail that: +The raw pointer implementations will be done via intrinsics or by depending on `F::MetaData`. If it is done by intrinsics, then `F::MetaData` can be removed. All other implementations of `Project` must boil down to some raw pointer projection. The raw pointer projections that we will provide include `project_unchecked` and a `Project` impl. The `project_unchecked` will assume that the input raw pointer is valid (i.e. points to a valid instance of `T` given a raw pointer `*[const|mut] T`) and optimize around that. The project impl will make no such guarantee, and if the pointer is not valid, then the behaviour is implementation defined, and may change between editions (but not other smaller version changes). + +For example of where `project_unchecked` would be UB. + +```rust +struct Foo { + bar: Bar +} + +let x : *const Foo = 2usize as *const Foo; +let y : *const Bar = x.project_unchecked(Foo.bar); // UB, x does not point to a valid instance of Foo +``` + +With `Project` trait + +```rust +use std::project::Project; + +let z : *const Bar = x.project(Foo.bar); // not UB, but z's value will be implementation defined +``` + +If the raw pointer is valid, then the result of both `project_unchecked` and `Project::project` is a raw pointer to the given field. + +The `Project` trait will be implemented for `*const T`, `*mut T`, `&T`, `&mut T`. Other smart pointers can get implementations later if they need them. We will also provide the following implementations to allow better documentation of intent + +```rust +impl<'a, T> Pin<&'a T> { + unsafe fn project_unchecked>(self, field: F) -> Pin<&'a F::Type> { + self.map_unchecked(|slf| slf.project(field)) + } +} -- Its interaction with other features is clear. -- It is reasonably clear how the feature would be implemented. -- Corner cases are dissected by example. +impl<'a, T> Pin<&'a mut T> { + unsafe fn project_unchecked>(self, field: F) -> Pin<&'a mut F::Type> { + self.map_unchecked_mut(|slf| slf.project(field)) + } +} +``` -The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work. +If `PinProjectable` is accepted, then `Project` trait will also be implemented for `Pin<&T>`, `Pin<&mut T>` and will be bound by `PinProjectable`. # Drawbacks [drawbacks]: #drawbacks @@ -152,7 +187,14 @@ Why should we *not* do this? # Unresolved questions [unresolved-questions]: #unresolved-questions -- Syntax for the type fields (not to be decided before accepting this RFC, but must be decided before stabilization) +- Behavior of `<*[const|mut] T as Project>::project` when the raw pointer is invalid (does not point to a valid T) + - Can this behaviour change across editions? How about smaller version changes? + - This issue blocks accepting this RFC + +- Syntax for the type fields + - not to be decided before accepting this RFC, but must be decided before stabilization +- Do we want a dedicated syntax to go with the Project trait? + - If yes, the actual syntax can be decided after accepting this RFC and before stabilization # Future possibilities [future-possibilities]: #future-possibilities From 6a9405f2b60da31ac963b083dc8e51c252de1e04 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Wed, 5 Jun 2019 16:06:31 -0700 Subject: [PATCH 04/27] updated formatting and added reference to InitPtr added PinProjectable to unresolved questions --- text/0000-ptr-to-field.md | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 7c1bb79ec60..138f21f9f7b 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -1,5 +1,5 @@ -- Feature Name: (fill me in with a unique ident, `my_awesome_feature`) -- Start Date: (fill me in with today's date, YYYY-MM-DD) +- Feature Name: ptr-to-field +- Start Date: 2019-06-05 - RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) - Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) @@ -16,12 +16,12 @@ The motivation for this feature is to allow safe projection through smart pointe # Guide-level explanation [guide-level-explanation]: #guide-level-explanation -First the core trait, type, and functions that need to be added +First the core traits, type, and functions that need to be added ```rust /// Contains metadata about how to get to the field from the parent using raw pointers /// This is an opaque type that never needs to be stablized, and it only an implementation detail -struct MetaData { +struct FieldDescriptor { ... } @@ -35,7 +35,7 @@ trait Field { type Type; /// The metadata required to get to the field using raw pointers - const META: MetaData; + const FIELD_DESCRIPTOR: FieldDescriptor; } trait Project { @@ -64,9 +64,9 @@ impl *mut T { Now we need some syntax to refer to the fields of types. Some ideas for the syntax are -* `Parent.field` +* `Parent.field` // my favorite, as it seems most natural * `Parent::field` // bad as it conflicts with associated functions -* `Parent~field` // or any other sigil +* `Parent~field` // or any other sigil We will call these field types, because they will desugar to a unit type that correctly implements `Field`, like so @@ -75,7 +75,7 @@ struct Foo { bar: Bar } -struct Foo.bar; +struct Foo.bar; // or some other name mangle that doesn't conflict with any other name impl Field for Foo.bar { ... } ``` @@ -89,7 +89,7 @@ let foo : Pin = Box::pin(immovable); let foo : Pin<&mut Foo> = foo.as_mut(); let field : Pin<&mut Field> = foo.project(Foo.field); ``` -But to do safe pin projections we will need to introduce a marker trait. +But to do safe pin projections we will need to introduce a marker trait. Adding this trait is up for debate. ```rust /// The only people who can implement `PinProjectable` are the creator of the parent type /// This allows people to opt-in to allowing their fields to be pin projectable. @@ -104,7 +104,8 @@ unsafe trait PinProjectable: Field {} The field types needs to interact with the privacy rules for fields. A field type has the same privacy as the field it is derived from. Anything else would be too restrictive or unsound. -The project trait will live inside the `std::project` module, and will not be added to `prelude`. +The `Project`, and `Field` traits and the `FieldDescriptor` type will live inside the `std::project` module, and will not be added to `prelude`. +The `PinProjectable` trait will be added to `std::marker` if it is accepted. As example of how to implement `Project`, here is the implementation for `&T`. @@ -123,7 +124,7 @@ impl<'a, F: Field> Project for &'a F::Parent where F::Type: 'a { } ``` -The raw pointer implementations will be done via intrinsics or by depending on `F::MetaData`. If it is done by intrinsics, then `F::MetaData` can be removed. All other implementations of `Project` must boil down to some raw pointer projection. The raw pointer projections that we will provide include `project_unchecked` and a `Project` impl. The `project_unchecked` will assume that the input raw pointer is valid (i.e. points to a valid instance of `T` given a raw pointer `*[const|mut] T`) and optimize around that. The project impl will make no such guarantee, and if the pointer is not valid, then the behaviour is implementation defined, and may change between editions (but not other smaller version changes). +The raw pointer implementations will be done via intrinsics or by depending on `F::FieldDescriptor`. If it is done by intrinsics, then `F::FieldDescriptor` can be removed. All other implementations of `Project` must boil down to some raw pointer projection. The raw pointer projections that we will provide include `project_unchecked` and a `Project` impl. The `project_unchecked` will assume that the input raw pointer is valid (i.e. points to a valid instance of `T` given a raw pointer `*[const|mut] T`) and optimize around that. The project impl will make no such guarantee, and if the pointer is not valid, then the behaviour is implementation defined, and may change between editions (but not other smaller version changes). For example of where `project_unchecked` would be UB. @@ -190,6 +191,9 @@ Why should we *not* do this? - Behavior of `<*[const|mut] T as Project>::project` when the raw pointer is invalid (does not point to a valid T) - Can this behaviour change across editions? How about smaller version changes? - This issue blocks accepting this RFC +- Are we going to accept `PinProjectable`? + - If not, we won't have a safe way to do pin-projections + - Do we want another way to do safe pin-projections? - Syntax for the type fields - not to be decided before accepting this RFC, but must be decided before stabilization @@ -200,3 +204,4 @@ Why should we *not* do this? [future-possibilities]: #future-possibilities - Extend the `Project` trait to implement all smart pointers in the standard library +- [`InitPtr`](https://internals.rust-lang.org/t/idea-pointer-to-field/10061/72), which encapsulates all of the safety requirements of `project_unchecked` into `InitPtr::new` and safely implements `Project` \ No newline at end of file From 9b98d977f564ac931222520224b22472f3d5847b Mon Sep 17 00:00:00 2001 From: Ozaren Date: Wed, 5 Jun 2019 16:14:25 -0700 Subject: [PATCH 05/27] updated parts about field types, updated drawbacks --- text/0000-ptr-to-field.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 138f21f9f7b..b85b93634d3 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -62,13 +62,15 @@ impl *mut T { } ``` -Now we need some syntax to refer to the fields of types. Some ideas for the syntax are +Now we need some syntax to refer to the fields types. Some ideas for the syntax are * `Parent.field` // my favorite, as it seems most natural * `Parent::field` // bad as it conflicts with associated functions * `Parent~field` // or any other sigil -We will call these field types, because they will desugar to a unit type that correctly implements `Field`, like so +But syntax discussions are defered until after this RFC has been accepted. This is to focues the RFC's discussion on if this addition is needed and close and lingering soundness holes. + +Field types will desugar to a unit type that correctly implements `Field`, like so ```rust struct Foo { @@ -124,7 +126,7 @@ impl<'a, F: Field> Project for &'a F::Parent where F::Type: 'a { } ``` -The raw pointer implementations will be done via intrinsics or by depending on `F::FieldDescriptor`. If it is done by intrinsics, then `F::FieldDescriptor` can be removed. All other implementations of `Project` must boil down to some raw pointer projection. The raw pointer projections that we will provide include `project_unchecked` and a `Project` impl. The `project_unchecked` will assume that the input raw pointer is valid (i.e. points to a valid instance of `T` given a raw pointer `*[const|mut] T`) and optimize around that. The project impl will make no such guarantee, and if the pointer is not valid, then the behaviour is implementation defined, and may change between editions (but not other smaller version changes). +The raw pointer implementations will be done via intrinsics or by depending on `F::FIELD_DESCRIPTOR`. If it is done by intrinsics, then `F::FIELD_DESCRIPTOR` can be removed. All other implementations of `Project` must boil down to some raw pointer projection. The raw pointer projections that we will provide include `project_unchecked` and a `Project` impl. The `project_unchecked` will assume that the input raw pointer is valid (i.e. points to a valid instance of `T` given a raw pointer `*[const|mut] T`) and optimize around that (i.e. it can use the `inbounds` assertion in LLVM). The project impl will make no such guarantee, and if the pointer is not valid, then the behaviour is implementation defined. For example of where `project_unchecked` would be UB. @@ -168,8 +170,7 @@ If `PinProjectable` is accepted, then `Project` trait will also be implemented f # Drawbacks [drawbacks]: #drawbacks -Why should we *not* do this? -- This adds quite a bit of complexity and can increase compile times +- This adds quite a bit of complexity to both the compiler and the standard library and could increase dramatically compile times # Rationale and alternatives [rationale-and-alternatives]: #rationale-and-alternatives From b81679892c6031e68ef959ebdece2a99dca54999 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Wed, 5 Jun 2019 16:20:28 -0700 Subject: [PATCH 06/27] added note about the `Copy` trait. --- text/0000-ptr-to-field.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index b85b93634d3..4fc3b1f992f 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -205,4 +205,5 @@ If `PinProjectable` is accepted, then `Project` trait will also be implemented f [future-possibilities]: #future-possibilities - Extend the `Project` trait to implement all smart pointers in the standard library -- [`InitPtr`](https://internals.rust-lang.org/t/idea-pointer-to-field/10061/72), which encapsulates all of the safety requirements of `project_unchecked` into `InitPtr::new` and safely implements `Project` \ No newline at end of file +- [`InitPtr`](https://internals.rust-lang.org/t/idea-pointer-to-field/10061/72), which encapsulates all of the safety requirements of `project_unchecked` into `InitPtr::new` and safely implements `Project` +- Distant Future, we could reformulate `Copy` based on the `Field` trait so that it enforces that all of the fields of a type must be `Copy` in order to be the type to be `Copy`, and thus reduce the amount of magic in the compiler. \ No newline at end of file From 0eb3d5cc32966fead2e6c40530489e3760f71033 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Thu, 6 Jun 2019 10:49:16 -0700 Subject: [PATCH 07/27] added unsafe to `Field` trait as it controls unsafe code, `Field` must be an unsafe trait --- text/0000-ptr-to-field.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 4fc3b1f992f..e5d403ceae0 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -28,7 +28,7 @@ struct FieldDescriptor { /// The compiler should prevent user implementations for `Field`, /// i.e. only the compiler is allowed to implement `Field` /// This is to prevent people from creating fake "fields" -trait Field { +unsafe trait Field { /// The type that the field is a part of type Parent; /// The type of the field From 9d826467e4c695393b031cc16365eae1d2599066 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Thu, 6 Jun 2019 10:51:16 -0700 Subject: [PATCH 08/27] added unresolved question: minimizing proposal --- text/0000-ptr-to-field.md | 1 + 1 file changed, 1 insertion(+) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index e5d403ceae0..6fc56509a6a 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -195,6 +195,7 @@ If `PinProjectable` is accepted, then `Project` trait will also be implemented f - Are we going to accept `PinProjectable`? - If not, we won't have a safe way to do pin-projections - Do we want another way to do safe pin-projections? +- Do we want to strip this proposal down to just the [api specified here](https://github.com/rust-lang/rfcs/pull/2708#issuecomment-499578814), where we only have the `Field` trait, `FieldDescriptor` type, and some associated functions on raw pointers. - Syntax for the type fields - not to be decided before accepting this RFC, but must be decided before stabilization From a1b545e1d303031ec7634d01a6d757713377bb20 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Thu, 6 Jun 2019 11:49:02 -0700 Subject: [PATCH 09/27] Added `?Sized` bounds --- text/0000-ptr-to-field.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 6fc56509a6a..9d05cf829ba 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -30,9 +30,9 @@ struct FieldDescriptor { /// This is to prevent people from creating fake "fields" unsafe trait Field { /// The type that the field is a part of - type Parent; + type Parent: ?Sized; /// The type of the field - type Type; + type Type: ?Sized; /// The metadata required to get to the field using raw pointers const FIELD_DESCRIPTOR: FieldDescriptor; From 8b5e4e2ec5ee85bb7f027ea4605241800880afa6 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Fri, 7 Jun 2019 08:30:57 -0700 Subject: [PATCH 10/27] added some details abot field types --- text/0000-ptr-to-field.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 9d05cf829ba..16a166c3da6 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -167,6 +167,26 @@ impl<'a, T> Pin<&'a mut T> { If `PinProjectable` is accepted, then `Project` trait will also be implemented for `Pin<&T>`, `Pin<&mut T>` and will be bound by `PinProjectable`. +Some notes about field types: + +You can make a field type for the following types of types +* tuples `(T, U, ...)` + * `<(T, U)>.0` + * `<(T, U)>::0` + * `(T, U)~0` +* tuple structs `Foo(T, U, ...)` + * `Foo.0` + * `Foo::0` + * `Foo~0` +* structs `struct Foo { field: Field, ... }` + * same syntax as tuple struct +* unions `union Foo { field: Field }` + * same syntax as tuple struct + * constructing a field type is `unsafe`, this is because accessing fields of `union`s is unsafe + +All fields types are treated as if they are declared in the same crate as their `Parent` type. +This will allow users to implement traits for field types, like the `PinProjectable` trait. + # Drawbacks [drawbacks]: #drawbacks From 14fc49f627593ffec39e988f3f9996219074ee1a Mon Sep 17 00:00:00 2001 From: Ozaren Date: Fri, 7 Jun 2019 12:13:20 -0700 Subject: [PATCH 11/27] reworked the RFC's presentation The new layout should make it easier for people who are unfamiar with the concept to understand it There are some more details about how some parts work, and why they work the way they do --- text/0000-ptr-to-field.md | 330 +++++++++++++++++++++++++++----------- 1 file changed, 235 insertions(+), 95 deletions(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 16a166c3da6..ad6419caaac 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -16,117 +16,219 @@ The motivation for this feature is to allow safe projection through smart pointe # Guide-level explanation [guide-level-explanation]: #guide-level-explanation -First the core traits, type, and functions that need to be added +This RFC introduces the idea of field types, and build projections on top of field types. + +## Terminology, types, and traits + +### Field Types + +A field type is a compiler generated type that represents a field. For example, ```rust -/// Contains metadata about how to get to the field from the parent using raw pointers -/// This is an opaque type that never needs to be stablized, and it only an implementation detail -struct FieldDescriptor { - ... +struct Person { + name: String, + age: u32, } +``` + +A field type for the `name` field of `Person` would be `Person.name`. + +Note about syntax: the syntax `Type.field` is going to be used as a placeholder syntax for the rest of this RFC, but this is *not* the final syntax for field types, that decision will be made after this RFC gets accepted but before it gets stabilized. + +Field types serve a few important purposes +* They are integrated with the new `Project` trait (explained later) +* Becaue they are types, they can implement traits + * This will allow conditional implementations of `Project`, which is important for `Pin

` (also explained later) -/// The compiler should prevent user implementations for `Field`, -/// i.e. only the compiler is allowed to implement `Field` -/// This is to prevent people from creating fake "fields" -unsafe trait Field { - /// The type that the field is a part of +Because they are types we can also generalize over field types like so... + +### `trait Field` and `type FieldDescriptor` + +In order to generalize over field types, we have the `trait Field` + +```rust +trait Field { + /// The type of the type that the field comes from type Parent: ?Sized; - /// The type of the field + + /// The type of the field itself type Type: ?Sized; - /// The metadata required to get to the field using raw pointers const FIELD_DESCRIPTOR: FieldDescriptor; } -trait Project { - /// The projected version of Self - type Projection; - - fn project(self, field: F) -> Self::Projection; +struct FieldDescriptor { + ... } +``` +`FieldDescriptor` is an opaque type that will store some metadata about how to convert from a `*const Field::Parent` to a `*const Field::Type`. There is no way to safely construct `FieldDescriptor` from user code on Stable Rust until Rust gets a defined stable type layout for `repr(Rust)` types. + +The `Field` trait will allow generalizing over field types, and thus allow other apis to be created, for example... + +### `*const T`/`*mut T` methods + +We will add the following methods to raw pointers +```rust +// details about why we need both and what they do exactly in Reference-level explanation impl *const T { - unsafe fn project_unchecked>(self, field: F) -> *const F::Type { - // make the field pointer, this code is allowed to assume that - // self points to a valid instance of T - ... - } + pub unsafe fn project_unchecked>(self, field: F) -> *const F::Type; + + pub fn wrapping_project>(self, field: F) -> *const F::Type; } impl *mut T { - unsafe fn project_unchecked>(self, field: F) -> *mut F::Type { - // make the field pointer, this code is allowed to assume that - // self points to a valid instance of T - ... - } + pub unsafe fn project_unchecked>(self, field: F) -> *mut F::Type; + + pub fn wrapping_project>(self, field: F) -> *mut F::Type; } ``` -Now we need some syntax to refer to the fields types. Some ideas for the syntax are +These will allowing projections through raw pointers without dereferencing the raw pointer. This is useful for building projections through other abstractions like smart pointers (`Rc`, `Pin<&T>`)! -* `Parent.field` // my favorite, as it seems most natural -* `Parent::field` // bad as it conflicts with associated functions -* `Parent~field` // or any other sigil +This is the extent of the core api of this RFC. -But syntax discussions are defered until after this RFC has been accepted. This is to focues the RFC's discussion on if this addition is needed and close and lingering soundness holes. - -Field types will desugar to a unit type that correctly implements `Field`, like so +Using this we can do something like this ```rust struct Foo { - bar: Bar + bar: Bar, + age: u32 } -struct Foo.bar; // or some other name mangle that doesn't conflict with any other name +struct Bar { + name: String, + id: i32 +} -impl Field for Foo.bar { ... } +let x : Foo = ...; +let y : *const Foo = &x; + +let y_bar_name: *const String = unsafe { y.project_unchecked(Foo.bar).project_unchecked(Foo.name) }; ``` -These are the core parts of this proposal. Every other part of this proposal can be postponed or dropped without affecting this feature's core principles. +In the end `y_bar_name` will contain a pointer to `x.bar.name`, all without dereferencing a single pointer! (Given that this is a verbose, we may want some syntax for this, but that is out of scope for this RFC) + +But, we can build on this foundation and create a more power abstraction, to generalize this project notion to smart pointers... -Using these core parts we can build as a library projections through `Pin<&T>`, `Rc<_>` and more. We can then use this to safely project through smart pointers like so. +### `trait Project` ```rust -let foo : Pin = Box::pin(immovable); -let foo : Pin<&mut Foo> = foo.as_mut(); -let field : Pin<&mut Field> = foo.project(Foo.field); +trait Project { + type Projection; + + fn project(self, field: F) -> Self::Projection; +} ``` -But to do safe pin projections we will need to introduce a marker trait. Adding this trait is up for debate. + +This trait takes a pointer/smart pointer/reference and gives back a projections that represents a field on the pointee. + +i.e. + +For raw pointers we could implement this as so + ```rust -/// The only people who can implement `PinProjectable` are the creator of the parent type -/// This allows people to opt-in to allowing their fields to be pin projectable. -/// The guarantee is that once you create `Pin>`, all of the same guarantees that -/// apply to `Pin>` also apply to `Pin>` -/// For all `Parent: Unpin`, these can be auto implemented for all of their fields. -unsafe trait PinProjectable: Field {} +impl> Project for *const T { + type Projection = F::Type; + + fn project(self, field: F) -> Self::Projection { + self.wrapping_project(field) + } +} ``` -# Reference-level explanation -[reference-level-explanation]: #reference-level-explanation +On it's own, this doesn't look like much, but once we have implementations for `&T`, we can also get an implementation for `Pin<&T>`! Meaning we would have safe projections through pins! (See [implementing pin projections](#implementing-pin-projections) for details about this) -The field types needs to interact with the privacy rules for fields. A field type has the same privacy as the field it is derived from. Anything else would be too restrictive or unsound. +But before we can get there we need to discuss... -The `Project`, and `Field` traits and the `FieldDescriptor` type will live inside the `std::project` module, and will not be added to `prelude`. -The `PinProjectable` trait will be added to `std::marker` if it is accepted. +### `trait PinProjectable` -As example of how to implement `Project`, here is the implementation for `&T`. +```rust +unsafe trait PinProjectable {} +``` +Due to some safety requirements that will be detailed in the reference-level explanation, we can't just freely hand out pin projections to every type (sad as it is). To enable pin projections to a field, a field type must implement `PinProjectable`. + +Like so, ```rust -impl<'a, F: Field> Project for &'a F::Parent where F::Type: 'a { - type Projection = &'a F::Type; +unsafe impl PinProjectable for Foo.field {} +``` - fn project(self, field: F) -> Self { - unsafe { - // This is safe because a reference is always valids - let ptr: *const F::Type = (self as *const F::Parent).project_unchecked(field); +## Examples of usage - &*ptr - } - } +Here is a toy example of how to use this api: +Given the implementation of `Project for Pin<&mut T>` +```rust +use std::project::Project; + +struct Foo { + bar: u32, + qaz: u32, + hal: u32, + ptr: *const u32, + pin: PhantomPinned +} + +// These type annotations are unnecessary, but I put them in for clarity + +let foo : Pin = Box::pin(...); + +let foo : Pin<&mut Foo> = foo.as_mut(); + +let bar : Pin<&mut u32> = foo.project(Foo.bar); + +*bar = 10; +... +``` + +In entirely safe code, we are able to set the value of a field inside a pinned type! + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +This section will go in detail about the semantics and implementation of the types, traits, and methods introduced by this RFC. + +## Field types + +Field types are just sugar for unit types (`struct Foo;`) that are declared right next to the `Parent` type. + +Like so, +```rust +struct Person { + pub name: String, + pub(super) age: u32, + id: u32, } + +pub struct Person.name; +pub(super) struct Person.age; +struct Person.id; ``` -The raw pointer implementations will be done via intrinsics or by depending on `F::FIELD_DESCRIPTOR`. If it is done by intrinsics, then `F::FIELD_DESCRIPTOR` can be removed. All other implementations of `Project` must boil down to some raw pointer projection. The raw pointer projections that we will provide include `project_unchecked` and a `Project` impl. The `project_unchecked` will assume that the input raw pointer is valid (i.e. points to a valid instance of `T` given a raw pointer `*[const|mut] T`) and optimize around that (i.e. it can use the `inbounds` assertion in LLVM). The project impl will make no such guarantee, and if the pointer is not valid, then the behaviour is implementation defined. +All field types will implement the following traits: `Debug`, `Clone`, `Copy`, `PartialEq`, `Eq`, `PartialOrd`, `Ord`, `Hash`, with the same implementation for unit types. + +They will also need to interact with the privacy rules for fields, and will have the same privacy as the field that they are derived from. + +You can make a field type for the following types of types (some example syntax is shown) +* tuples `(T, U, ...)` + * `<(T, U)>.0` +* tuple structs `Foo(T, U, ...)` + * `Foo.0` +* structs `struct Foo { field: Field, ... }` + * same syntax as tuple struct +* unions `union Foo { field: Field }` + * same syntax as tuple struct + * constructing a field type is `unsafe`, this is because accessing fields of `union`s is unsafe + +The compiler can decide whether to actual generate a field type, this is to help compile times (if you don't use field types, then the compiler shouldn't slow down too much because of this feature). + +## `*const T`/`*mut T` + +`project_unchecked` and `wrapping_project` will both live inside of `std::ptr`. + +We need both `project_unchecked` and `wrapping_project` because there are some important optimization available inside of LLVM related to aliasing and escape analysis. In particular the LLVM `inbounds` assertion tells LLVM that a pointer offset stays within the same allocation and if the pointer is invalid or the offset does not stay within the same allocation it is considered UB. This behaviour is exposed via `project_unchecked`. This can be used by implementations of the `Project` trait for smart pointers that are always valid, like `&T` to enable better codegen. `wrapping_project` on the other hand will not assert `inbounds`, and will just wrap around if the pointer offset is larger than `usize::max_value()`. This safe defined behaviour, even if it is almost always a bug, unlike `project_unchecked` which is UB on invalid pointers or offsets. + +This corresponds with `std::ptr::offset` and `std::ptr::wrapping_offset` in safety and behaviour. `project_unchecked` and `project` need to be implemented as intrinsics because there is no way to assert that the pointer metadata for fat pointers of `Field::Parent` and `Field::Type` will always match in general without some other compiler support. This is necessary to allow unsized types to be used transparently with this scheme. For example of where `project_unchecked` would be UB. @@ -139,53 +241,94 @@ let x : *const Foo = 2usize as *const Foo; let y : *const Bar = x.project_unchecked(Foo.bar); // UB, x does not point to a valid instance of Foo ``` -With `Project` trait +With `wrapping_project` trait ```rust -use std::project::Project; +let z : *const Bar = x.wrapping_project(Foo.bar); // not UB, but is still invalid +``` + +If the raw pointer is valid, then the result of both `project_unchecked` and `wrapping_project` is a raw pointer to the given field. + +## Type and Traits + +The `Project`, and `Field` traits and the `FieldDescriptor` type will live inside the `std::project` module, and will not be added to `prelude`. +The `PinProjectable` trait will be added to `std::marker`, and will also not be added to `prelude`. + +The `Field` trait will only be implemented by the compiler, and it compiler should make sure that no other implementations exist. This allows unsafe code to assume that the implmentors or the `Field` trait always refer to valid fields. The `FieldDescriptor` type may be unnecessary raw pointer projections are implemented via intrinsics, if so we can remove it entirely. + +The `Project` trait will be implemented for `*const T`, `*mut T`, `&T`, `&mut T`, `Pin<&T>`, `Pin<&mut T>`. Other smart pointers can get implementations later if they need them. We may also provide the following implementations to allow better documentation of intent + +## `PinProjectable` + +The safety of `PinProjectable` depends on a few things. One if a field type is marked `PinProjectable`, then the `Drop` on `Parent` may not move that field or otherwise invalidate it. i.e. You must treat that field as if a `Pin<&Field::Type>` has been made on it outside of your code. + +Because the following impls conflict, + +```rust +unsafe impl PinProjectable for F where F::Parent: Unpin {} + +struct Foo(std::marker::PhantomPinned); -let z : *const Bar = x.project(Foo.bar); // not UB, but z's value will be implementation defined +unsafe impl PinProjectable for Foo.0 {} ``` +[Proof](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=b0796a8b631e0fec1804318caef162c7) + +I think making `PhantomPinned` a lang-item that is known to always implment `!Unpin` would solve this. This way only those who implment `!Unpin` types need to worry about implementing `PinProjectable`. Another way to solve this would be to somehow make `PinProjectable` a lang-item that allows this one case of conflicting impls. But I am unsure of how to properly handle this, both way s that I showed seem unsatisfactory. The blanket impl is highly desirable, because it enables those who don't write `!Unpin` types to ignore safe pin projections, and still have them available. + +Because of this, is `PinProjectable` worth it? Or do we want to punt it to another RFC. + +## Implementing pin projections +[impl-pin-projections]: #implementing-pin-projections + +First we neeed an implmention of `Project` for `&T` before we can get `Pin<&T>` + +```rust +impl<'a, F: Field> Project for &'a F::Parent where F::Type: 'a { + type Projection = &'a F::Type; -If the raw pointer is valid, then the result of both `project_unchecked` and `Project::project` is a raw pointer to the given field. + fn project(self, field: F) -> Self { + unsafe { + // This is safe because a reference is always valids + // So offsetting the reference to a field is always fine + // The resulting pointer must always be valid + // because it came from a reference + let ptr: *const F::Type = (self as *const F::Parent).project_unchecked(field); + + &*ptr + } + } +} +``` -The `Project` trait will be implemented for `*const T`, `*mut T`, `&T`, `&mut T`. Other smart pointers can get implementations later if they need them. We will also provide the following implementations to allow better documentation of intent +We can introduce an unsafe projection interface to better document intent (this is optional) ```rust impl<'a, T> Pin<&'a T> { - unsafe fn project_unchecked>(self, field: F) -> Pin<&'a F::Type> { + /// This is unsafe because `Drop` code + pub unsafe fn project_unchecked>(self, field: F) -> Pin<&'a F::Type> { self.map_unchecked(|slf| slf.project(field)) } } impl<'a, T> Pin<&'a mut T> { - unsafe fn project_unchecked>(self, field: F) -> Pin<&'a mut F::Type> { + pub unsafe fn project_unchecked>(self, field: F) -> Pin<&'a mut F::Type> { self.map_unchecked_mut(|slf| slf.project(field)) } } ``` -If `PinProjectable` is accepted, then `Project` trait will also be implemented for `Pin<&T>`, `Pin<&mut T>` and will be bound by `PinProjectable`. - -Some notes about field types: +We can then implment safe pin projections like so -You can make a field type for the following types of types -* tuples `(T, U, ...)` - * `<(T, U)>.0` - * `<(T, U)>::0` - * `(T, U)~0` -* tuple structs `Foo(T, U, ...)` - * `Foo.0` - * `Foo::0` - * `Foo~0` -* structs `struct Foo { field: Field, ... }` - * same syntax as tuple struct -* unions `union Foo { field: Field }` - * same syntax as tuple struct - * constructing a field type is `unsafe`, this is because accessing fields of `union`s is unsafe +```rust +impl<'a, T> Project> for Pin<&'a T> +where F: PinProjectable { + type Projection = Pin<&'a F::Type>; -All fields types are treated as if they are declared in the same crate as their `Parent` type. -This will allow users to implement traits for field types, like the `PinProjectable` trait. + fn project(self, field: F) -> Self::Projection { + unsafe { self.project_unchecked(field) } + } +} +``` # Drawbacks [drawbacks]: #drawbacks @@ -209,13 +352,10 @@ This will allow users to implement traits for field types, like the `PinProjecta # Unresolved questions [unresolved-questions]: #unresolved-questions -- Behavior of `<*[const|mut] T as Project>::project` when the raw pointer is invalid (does not point to a valid T) - - Can this behaviour change across editions? How about smaller version changes? - - This issue blocks accepting this RFC +- Do we want to strip this proposal down to just the [api specified here](https://github.com/rust-lang/rfcs/pull/2708#issuecomment-499578814), where we only have the `Field` trait, `FieldDescriptor` type, and some associated functions on raw pointers. - Are we going to accept `PinProjectable`? - If not, we won't have a safe way to do pin-projections - Do we want another way to do safe pin-projections? -- Do we want to strip this proposal down to just the [api specified here](https://github.com/rust-lang/rfcs/pull/2708#issuecomment-499578814), where we only have the `Field` trait, `FieldDescriptor` type, and some associated functions on raw pointers. - Syntax for the type fields - not to be decided before accepting this RFC, but must be decided before stabilization From d3cac452be1206187ca4be63485c8e095ad7a103 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Fri, 7 Jun 2019 12:31:10 -0700 Subject: [PATCH 12/27] fixed the pin-projection example in guid-level fixed typo moved notes about syntax to unresolved questions --- text/0000-ptr-to-field.md | 40 ++++++++++++++++++++++++--------------- 1 file changed, 25 insertions(+), 15 deletions(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index ad6419caaac..61f9fea7bf1 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -121,11 +121,9 @@ trait Project { } ``` -This trait takes a pointer/smart pointer/reference and gives back a projections that represents a field on the pointee. +This trait takes a pointer/smart pointer/reference and gives back a projection that represents a field on the pointee. -i.e. - -For raw pointers we could implement this as so +i.e. For raw pointers we could implement this as so ```rust impl> Project for *const T { @@ -159,29 +157,38 @@ unsafe impl PinProjectable for Foo.field {} Here is a toy example of how to use this api: Given the implementation of `Project for Pin<&mut T>` ```rust -use std::project::Project; +/// Some other crate foo struct Foo { - bar: u32, + pub bar: u32, qaz: u32, hal: u32, ptr: *const u32, pin: PhantomPinned } -// These type annotations are unnecessary, but I put them in for clarity +unsafe impl PinProjectable for Foo.bar {} + +/// main.rs + +fn main() { + use std::project::Project; + use foo::Foo; -let foo : Pin = Box::pin(...); + // These type annotations are unnecessary, but I put them in for clarity -let foo : Pin<&mut Foo> = foo.as_mut(); + let foo : Pin = Box::pin(...); -let bar : Pin<&mut u32> = foo.project(Foo.bar); + let foo : Pin<&mut Foo> = foo.as_mut(); -*bar = 10; -... + let bar : Pin<&mut u32> = foo.project(Foo.bar); + + *bar = 10; +} ``` -In entirely safe code, we are able to set the value of a field inside a pinned type! +In entirely safe code (in `main.rs`), we are able to set the value of a field inside a pinned type! +We still need some `unsafe` to tell that it is indeed safe to project through the pin, but that falls on the owner of `Foo`. # Reference-level explanation [reference-level-explanation]: #reference-level-explanation @@ -215,9 +222,9 @@ You can make a field type for the following types of types (some example syntax * tuple structs `Foo(T, U, ...)` * `Foo.0` * structs `struct Foo { field: Field, ... }` - * same syntax as tuple struct + * `Foo.field` * unions `union Foo { field: Field }` - * same syntax as tuple struct + * same syntax as structs * constructing a field type is `unsafe`, this is because accessing fields of `union`s is unsafe The compiler can decide whether to actual generate a field type, this is to help compile times (if you don't use field types, then the compiler shouldn't slow down too much because of this feature). @@ -359,6 +366,9 @@ where F: PinProjectable { - Syntax for the type fields - not to be decided before accepting this RFC, but must be decided before stabilization + - Some other variations of the syntax are ... + - `Type::field` // This is bad because it conflicts with associated method, and there isn't a way to disambiguate them easily + - `Type~field` // This adds a new sigil to the language - Do we want a dedicated syntax to go with the Project trait? - If yes, the actual syntax can be decided after accepting this RFC and before stabilization From d00144280d86df7d6586a18cf0e86e9765a8deab Mon Sep 17 00:00:00 2001 From: Krishna Sannasi Date: Fri, 7 Jun 2019 13:16:35 -0700 Subject: [PATCH 13/27] fixed typo Thanks @skippy10110 for finding this --- text/0000-ptr-to-field.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 61f9fea7bf1..5e75acb36da 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -89,7 +89,6 @@ These will allowing projections through raw pointers without dereferencing the r This is the extent of the core api of this RFC. Using this we can do something like this - ```rust struct Foo { bar: Bar, @@ -104,7 +103,7 @@ struct Bar { let x : Foo = ...; let y : *const Foo = &x; -let y_bar_name: *const String = unsafe { y.project_unchecked(Foo.bar).project_unchecked(Foo.name) }; +let y_bar_name: *const String = unsafe { y.project_unchecked(Foo.bar).project_unchecked(Bar.name) }; ``` In the end `y_bar_name` will contain a pointer to `x.bar.name`, all without dereferencing a single pointer! (Given that this is a verbose, we may want some syntax for this, but that is out of scope for this RFC) @@ -377,4 +376,4 @@ where F: PinProjectable { - Extend the `Project` trait to implement all smart pointers in the standard library - [`InitPtr`](https://internals.rust-lang.org/t/idea-pointer-to-field/10061/72), which encapsulates all of the safety requirements of `project_unchecked` into `InitPtr::new` and safely implements `Project` -- Distant Future, we could reformulate `Copy` based on the `Field` trait so that it enforces that all of the fields of a type must be `Copy` in order to be the type to be `Copy`, and thus reduce the amount of magic in the compiler. \ No newline at end of file +- Distant Future, we could reformulate `Copy` based on the `Field` trait so that it enforces that all of the fields of a type must be `Copy` in order to be the type to be `Copy`, and thus reduce the amount of magic in the compiler. From d6716338ecfa6c3d540018ddae39ddc125f21901 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Mon, 24 Jun 2019 13:53:10 -0700 Subject: [PATCH 14/27] removed `trait Project` and `trait PinProjectable` They increased the scope of the RFC too much and were not critical for for this RFC to be useful, so they were cut from the RFC. --- text/0000-ptr-to-field.md | 237 +++++++++++++++++++------------------- 1 file changed, 119 insertions(+), 118 deletions(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 61f9fea7bf1..73d0202cbbe 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -61,6 +61,7 @@ struct FieldDescriptor { ... } ``` + `FieldDescriptor` is an opaque type that will store some metadata about how to convert from a `*const Field::Parent` to a `*const Field::Type`. There is no way to safely construct `FieldDescriptor` from user code on Stable Rust until Rust gets a defined stable type layout for `repr(Rust)` types. The `Field` trait will allow generalizing over field types, and thus allow other apis to be created, for example... @@ -68,6 +69,7 @@ The `Field` trait will allow generalizing over field types, and thus allow other ### `*const T`/`*mut T` methods We will add the following methods to raw pointers + ```rust // details about why we need both and what they do exactly in Reference-level explanation @@ -104,14 +106,123 @@ struct Bar { let x : Foo = ...; let y : *const Foo = &x; -let y_bar_name: *const String = unsafe { y.project_unchecked(Foo.bar).project_unchecked(Foo.name) }; +let y_bar_name: *const String = unsafe { y.project_unchecked(Foo.bar).project_unchecked(Bar.name) }; ``` In the end `y_bar_name` will contain a pointer to `x.bar.name`, all without dereferencing a single pointer! (Given that this is a verbose, we may want some syntax for this, but that is out of scope for this RFC) -But, we can build on this foundation and create a more power abstraction, to generalize this project notion to smart pointers... +But, we can build on this foundation and create a more power abstraction, to generalize this project notion to smart pointers. + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +This section will go in detail about the semantics and implementation of the types, traits, and methods introduced by this RFC. + +## Field types + +Field types are just sugar for unit types (`struct Foo;`) that are declared right next to the `Parent` type. + +Like so, +```rust +struct Person { + pub name: String, + pub(super) age: u32, + id: u32, +} + +pub struct Person.name; +pub(super) struct Person.age; +struct Person.id; +``` + +All field types will implement the following traits: `Debug`, `Clone`, `Copy`, `PartialEq`, `Eq`, `PartialOrd`, `Ord`, `Hash`, with the same implementation for unit types. + +They will also need to interact with the privacy rules for fields, and will have the same privacy as the field that they are derived from. + +You can make a field type for the following types of types (some example syntax is shown) +* tuples `(T, U, ...)` + * `<(T, U)>.0` +* tuple structs `Foo(T, U, ...)` + * `Foo.0` +* structs `struct Foo { field: Field, ... }` + * `Foo.field` +* unions `union Foo { field: Field }` + * same syntax as structs + * constructing a field type is `unsafe`, this is because accessing fields of `union`s is unsafe + +The compiler can decide whether to actual generate a field type, this is to help compile times (if you don't use field types, then the compiler shouldn't slow down too much because of this feature). + +## `*const T`/`*mut T` + +`project_unchecked` and `wrapping_project` will both live inside of `std::ptr`. + +We need both `project_unchecked` and `wrapping_project` because there are some important optimization available inside of LLVM related to aliasing and escape analysis. In particular the LLVM `inbounds` assertion tells LLVM that a pointer offset stays within the same allocation and if the pointer is invalid or the offset does not stay within the same allocation it is considered UB. This behaviour is exposed via `project_unchecked`. This can be used by implementations of the `Project` trait for smart pointers that are always valid, like `&T` to enable better codegen. `wrapping_project` on the other hand will not assert `inbounds`, and will just wrap around if the pointer offset is larger than `usize::max_value()`. This safe defined behaviour, even if it is almost always a bug, unlike `project_unchecked` which is UB on invalid pointers or offsets. + +This corresponds with `std::ptr::offset` and `std::ptr::wrapping_offset` in safety and behaviour. `project_unchecked` and `project` need to be implemented as intrinsics because there is no way to assert that the pointer metadata for fat pointers of `Field::Parent` and `Field::Type` will always match in general without some other compiler support. This is necessary to allow unsized types to be used transparently with this scheme. + +For example of where `project_unchecked` would be UB. + +```rust +struct Foo { + bar: Bar +} + +let x : *const Foo = 2usize as *const Foo; +let y : *const Bar = x.project_unchecked(Foo.bar); // UB, x does not point to a valid instance of Foo +``` + +With `wrapping_project` trait + +```rust +let z : *const Bar = x.wrapping_project(Foo.bar); // not UB, but is still invalid +``` + +If the raw pointer is valid, then the result of both `project_unchecked` and `wrapping_project` is a raw pointer to the given field. + +## Type and Traits + +The `Field` trait and the `FieldDescriptor` type will live inside the `std::project` module, and will not be added to `prelude`. + +The `Field` trait will only be implemented by the compiler, and it compiler should make sure that no other implementations exist. This allows unsafe code to assume that the implmentors or the `Field` trait always refer to valid fields. The `FieldDescriptor` type may be unnecessary raw pointer projections are implemented via intrinsics, if so we can remove it entirely. + +# Drawbacks +[drawbacks]: #drawbacks + +- This adds quite a bit of complexity to both the compiler and the standard library and could increase dramatically compile times + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +- The `&[mut] raw T` could solve some of the problems, but only for raw pointers. It doesn't help with abstractions. +- Somehow expand on `Deref` to allow dereferencing to a smart pointer + - This would require Generic Associated Types at the very least, and maybe some other features like assocaited traits + +# Prior art +[prior-art]: #prior-art + +- C++'s pointer to members `Parent::*field` +- Java's `class Field` + - Similar reflection capabilies in other languages + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +- Syntax for the type fields + - not to be decided before accepting this RFC, but must be decided before stabilization + - Some other variations of the syntax are ... + - `Type::field` // This is bad because it conflicts with associated method, and there isn't a way to disambiguate them easily + - `Type~field` // This adds a new sigil to the language + +# Future possibilities +[future-possibilities]: #future-possibilities + +- Extend the `Project` trait to implement all smart pointers in the standard library +- [`InitPtr`](https://internals.rust-lang.org/t/idea-pointer-to-field/10061/72), which encapsulates all of the safety requirements of `project_unchecked` into `InitPtr::new` and safely implements `Project` +- Distant Future, we could reformulate `Copy` based on the `Field` trait so that it enforces that all of the fields of a type must be `Copy` in order to be the type to be `Copy`, and thus reduce the amount of magic in the compiler. + +- a `Project` and `PinProjectable` traits -### `trait Project` +## `trait Project` ```rust trait Project { @@ -139,7 +250,7 @@ On it's own, this doesn't look like much, but once we have implementations for ` But before we can get there we need to discuss... -### `trait PinProjectable` +## `trait PinProjectable` ```rust unsafe trait PinProjectable {} @@ -190,82 +301,14 @@ fn main() { In entirely safe code (in `main.rs`), we are able to set the value of a field inside a pinned type! We still need some `unsafe` to tell that it is indeed safe to project through the pin, but that falls on the owner of `Foo`. -# Reference-level explanation -[reference-level-explanation]: #reference-level-explanation - -This section will go in detail about the semantics and implementation of the types, traits, and methods introduced by this RFC. - -## Field types - -Field types are just sugar for unit types (`struct Foo;`) that are declared right next to the `Parent` type. - -Like so, -```rust -struct Person { - pub name: String, - pub(super) age: u32, - id: u32, -} - -pub struct Person.name; -pub(super) struct Person.age; -struct Person.id; -``` - -All field types will implement the following traits: `Debug`, `Clone`, `Copy`, `PartialEq`, `Eq`, `PartialOrd`, `Ord`, `Hash`, with the same implementation for unit types. - -They will also need to interact with the privacy rules for fields, and will have the same privacy as the field that they are derived from. - -You can make a field type for the following types of types (some example syntax is shown) -* tuples `(T, U, ...)` - * `<(T, U)>.0` -* tuple structs `Foo(T, U, ...)` - * `Foo.0` -* structs `struct Foo { field: Field, ... }` - * `Foo.field` -* unions `union Foo { field: Field }` - * same syntax as structs - * constructing a field type is `unsafe`, this is because accessing fields of `union`s is unsafe - -The compiler can decide whether to actual generate a field type, this is to help compile times (if you don't use field types, then the compiler shouldn't slow down too much because of this feature). - -## `*const T`/`*mut T` - -`project_unchecked` and `wrapping_project` will both live inside of `std::ptr`. - -We need both `project_unchecked` and `wrapping_project` because there are some important optimization available inside of LLVM related to aliasing and escape analysis. In particular the LLVM `inbounds` assertion tells LLVM that a pointer offset stays within the same allocation and if the pointer is invalid or the offset does not stay within the same allocation it is considered UB. This behaviour is exposed via `project_unchecked`. This can be used by implementations of the `Project` trait for smart pointers that are always valid, like `&T` to enable better codegen. `wrapping_project` on the other hand will not assert `inbounds`, and will just wrap around if the pointer offset is larger than `usize::max_value()`. This safe defined behaviour, even if it is almost always a bug, unlike `project_unchecked` which is UB on invalid pointers or offsets. - -This corresponds with `std::ptr::offset` and `std::ptr::wrapping_offset` in safety and behaviour. `project_unchecked` and `project` need to be implemented as intrinsics because there is no way to assert that the pointer metadata for fat pointers of `Field::Parent` and `Field::Type` will always match in general without some other compiler support. This is necessary to allow unsized types to be used transparently with this scheme. - -For example of where `project_unchecked` would be UB. - -```rust -struct Foo { - bar: Bar -} - -let x : *const Foo = 2usize as *const Foo; -let y : *const Bar = x.project_unchecked(Foo.bar); // UB, x does not point to a valid instance of Foo -``` - -With `wrapping_project` trait - -```rust -let z : *const Bar = x.wrapping_project(Foo.bar); // not UB, but is still invalid -``` - -If the raw pointer is valid, then the result of both `project_unchecked` and `wrapping_project` is a raw pointer to the given field. +## Reference-level explanation (`Project` and `PinProjectable`) -## Type and Traits - -The `Project`, and `Field` traits and the `FieldDescriptor` type will live inside the `std::project` module, and will not be added to `prelude`. +The `Project` trait will live inside the `std::project` module, and will not be added to `prelude`. The `PinProjectable` trait will be added to `std::marker`, and will also not be added to `prelude`. -The `Field` trait will only be implemented by the compiler, and it compiler should make sure that no other implementations exist. This allows unsafe code to assume that the implmentors or the `Field` trait always refer to valid fields. The `FieldDescriptor` type may be unnecessary raw pointer projections are implemented via intrinsics, if so we can remove it entirely. - The `Project` trait will be implemented for `*const T`, `*mut T`, `&T`, `&mut T`, `Pin<&T>`, `Pin<&mut T>`. Other smart pointers can get implementations later if they need them. We may also provide the following implementations to allow better documentation of intent -## `PinProjectable` +### `PinProjectable` The safety of `PinProjectable` depends on a few things. One if a field type is marked `PinProjectable`, then the `Drop` on `Parent` may not move that field or otherwise invalidate it. i.e. You must treat that field as if a `Pin<&Field::Type>` has been made on it outside of your code. @@ -284,7 +327,7 @@ I think making `PhantomPinned` a lang-item that is known to always implment `!Un Because of this, is `PinProjectable` worth it? Or do we want to punt it to another RFC. -## Implementing pin projections +### Implementing pin projections [impl-pin-projections]: #implementing-pin-projections First we neeed an implmention of `Project` for `&T` before we can get `Pin<&T>` @@ -336,45 +379,3 @@ where F: PinProjectable { } } ``` - -# Drawbacks -[drawbacks]: #drawbacks - -- This adds quite a bit of complexity to both the compiler and the standard library and could increase dramatically compile times - -# Rationale and alternatives -[rationale-and-alternatives]: #rationale-and-alternatives - -- The `&[mut] raw T` could solve some of the problems, but only for raw pointers. It doesn't help with abstractions. -- Somehow expand on `Deref` to allow dereferencing to a smart pointer - - This would require Generic Associated Types at the very least, and maybe some other features like assocaited traits - -# Prior art -[prior-art]: #prior-art - -- C++'s pointer to members `Parent::*field` -- Java's `class Field` - - Similar reflection capabilies in other languages - -# Unresolved questions -[unresolved-questions]: #unresolved-questions - -- Do we want to strip this proposal down to just the [api specified here](https://github.com/rust-lang/rfcs/pull/2708#issuecomment-499578814), where we only have the `Field` trait, `FieldDescriptor` type, and some associated functions on raw pointers. -- Are we going to accept `PinProjectable`? - - If not, we won't have a safe way to do pin-projections - - Do we want another way to do safe pin-projections? - -- Syntax for the type fields - - not to be decided before accepting this RFC, but must be decided before stabilization - - Some other variations of the syntax are ... - - `Type::field` // This is bad because it conflicts with associated method, and there isn't a way to disambiguate them easily - - `Type~field` // This adds a new sigil to the language -- Do we want a dedicated syntax to go with the Project trait? - - If yes, the actual syntax can be decided after accepting this RFC and before stabilization - -# Future possibilities -[future-possibilities]: #future-possibilities - -- Extend the `Project` trait to implement all smart pointers in the standard library -- [`InitPtr`](https://internals.rust-lang.org/t/idea-pointer-to-field/10061/72), which encapsulates all of the safety requirements of `project_unchecked` into `InitPtr::new` and safely implements `Project` -- Distant Future, we could reformulate `Copy` based on the `Field` trait so that it enforces that all of the fields of a type must be `Copy` in order to be the type to be `Copy`, and thus reduce the amount of magic in the compiler. \ No newline at end of file From 08d9ec2277472228222a8d18193bf3d31eedcce0 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Mon, 24 Jun 2019 14:04:33 -0700 Subject: [PATCH 15/27] added inverse projections --- text/0000-ptr-to-field.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 73d0202cbbe..97f9bf964a5 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -77,12 +77,20 @@ impl *const T { pub unsafe fn project_unchecked>(self, field: F) -> *const F::Type; pub fn wrapping_project>(self, field: F) -> *const F::Type; + + pub unsafe fn inverse_project_unchecked>(self, field: F) -> *const F::Parent; + + pub fn inverse_wrapping_project>(self, field: F) -> *const F::Parent; } impl *mut T { pub unsafe fn project_unchecked>(self, field: F) -> *mut F::Type; pub fn wrapping_project>(self, field: F) -> *mut F::Type; + + pub unsafe fn inverse_project_unchecked>(self, field: F) -> *mut F::Parent; + + pub fn inverse_wrapping_project>(self, field: F) -> *mut F::Parent; } ``` @@ -158,7 +166,11 @@ The compiler can decide whether to actual generate a field type, this is to help We need both `project_unchecked` and `wrapping_project` because there are some important optimization available inside of LLVM related to aliasing and escape analysis. In particular the LLVM `inbounds` assertion tells LLVM that a pointer offset stays within the same allocation and if the pointer is invalid or the offset does not stay within the same allocation it is considered UB. This behaviour is exposed via `project_unchecked`. This can be used by implementations of the `Project` trait for smart pointers that are always valid, like `&T` to enable better codegen. `wrapping_project` on the other hand will not assert `inbounds`, and will just wrap around if the pointer offset is larger than `usize::max_value()`. This safe defined behaviour, even if it is almost always a bug, unlike `project_unchecked` which is UB on invalid pointers or offsets. -This corresponds with `std::ptr::offset` and `std::ptr::wrapping_offset` in safety and behaviour. `project_unchecked` and `project` need to be implemented as intrinsics because there is no way to assert that the pointer metadata for fat pointers of `Field::Parent` and `Field::Type` will always match in general without some other compiler support. This is necessary to allow unsized types to be used transparently with this scheme. +This corresponds with `std::ptr::add` and `std::ptr::wrapping_add` in safety and behaviour. `project_unchecked` and `project` need to be implemented as intrinsics because there is no way to assert that the pointer metadata for fat pointers of `Field::Parent` and `Field::Type` will always match in general without some other compiler support. This is necessary to allow unsized types to be used transparently with this scheme. + +`inverse_project_unchecked` and `inverse_wrapping_project` are just like their counterparts in safety. `inverse_project_unchecked` is UB to use on invalid pointers, where `inverse_wrapping_project` just wraps around on overflow. But there is one important safety check that must be performed before dereferencing the resulting pointer. The resulting pointer may not actually point to a valid `*const F::Parent` if `*const F::Type` does not live inside of a `F::Parent`, so one must take care to ensure that the parent pointer is indeed valid. This is different from `project_unchecked` and `wrapping_project` because there one only needs to validate the original pointer, not the resulting pointer. + +`inverse_project_unchecked` and `inverse_wrapping_project` correspond to `std::ptr::sub` and `std::ptr::wrapping_sub` in safety and behaviour. They also must be implemented as intrinsics for the same reasons as stated above. For example of where `project_unchecked` would be UB. From 00de06a7137fe744231df4a2b94f6a6cb032bdc4 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Mon, 24 Jun 2019 14:08:29 -0700 Subject: [PATCH 16/27] updated motivation --- text/0000-ptr-to-field.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 97f9bf964a5..bd17c167157 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -6,12 +6,14 @@ # Summary [summary]: #summary -This feature could serve as the backbone for some pointer to field syntax, and even if no syntax is made, this feature serves as a safe generic way to talk about types and their fields. +This feature could serve as the backbone for some pointer to field syntax, and even if no syntax is made, this feature serves as a safe generic way to talk about types and their fields, and projections through raw pointers. # Motivation [motivation]: #motivation -The motivation for this feature is to allow safe projection through smart pointers, for example `Pin<&mut T>` to `Pin<&mut Field>`. This is a much needed feature to make `Pin

` more usable in safe-contexts, without the need to use unsafe to map to a field. This also can allow projection through other smart pointers like `Rc`, `Arc`. This feature cannot be implemented as a library effectively because it depends on the layouts of types, so it requires integration with the Rust compiler until Rust gets a stable layout (which may never happen). +The motivation for this feature is to build a foundation for libraries to allow projections through smart pointers. For example, through `Pin<&mut T>` to `Pin<&mut Field>`. This is a much needed feature to make `Pin

` more usable in safe-contexts, without the need to use unsafe to map to a field. This also can allow projection through other smart pointers like `Rc`, `Arc`. This feature cannot be implemented as a library effectively because it depends on the layouts of types, so it requires integration with the Rust compiler until Rust gets a stable layout (which may never happen). + +This feature will also allow for safer patterns in `unsafe` code that deals with intrusive data strutures via `inverse_*` projections. It will also provide projections through raw pointers, which are currently not possible to do safely without a `#[repr(...)]` attribute (and even then it is easy to make a mistake). This will make `unsafe` code easier to audit and easier to write sound `unsafe` code. # Guide-level explanation [guide-level-explanation]: #guide-level-explanation From 43d559b0540993f869b6489e3d30c9eb0e0fac6f Mon Sep 17 00:00:00 2001 From: Ozaren Date: Mon, 24 Jun 2019 14:14:08 -0700 Subject: [PATCH 17/27] reason for removing `Project` and `PinProjectable` --- text/0000-ptr-to-field.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index bd17c167157..00eb977ae98 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -235,7 +235,8 @@ The `Field` trait will only be implemented by the compiler, and it compiler shou - Distant Future, we could reformulate `Copy` based on the `Field` trait so that it enforces that all of the fields of a type must be `Copy` in order to be the type to be `Copy`, and thus reduce the amount of magic in the compiler. - a `Project` and `PinProjectable` traits - + - These were in an earier version of this RFC, but were removed as they are not essential, and this RFC is already rather niche. So to limit the scope of this RFC, they were removed. If this RFC is accepted, then `Project` and `PinProjectable` could be implmented as library items outside of `std`. + ## `trait Project` ```rust From a83911cfcc11eae2a072009bc421235672a6b3bb Mon Sep 17 00:00:00 2001 From: Ozaren Date: Mon, 24 Jun 2019 14:24:31 -0700 Subject: [PATCH 18/27] moved from `std` to `core` --- text/0000-ptr-to-field.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 042b53f195b..a89c8a72845 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -163,15 +163,15 @@ The compiler can decide whether to actual generate a field type, this is to help ## `*const T`/`*mut T` -`project_unchecked` and `wrapping_project` will both live inside of `std::ptr`. +`project_unchecked` and `wrapping_project` will both live inside of `core::ptr`. We need both `project_unchecked` and `wrapping_project` because there are some important optimization available inside of LLVM related to aliasing and escape analysis. In particular the LLVM `inbounds` assertion tells LLVM that a pointer offset stays within the same allocation and if the pointer is invalid or the offset does not stay within the same allocation it is considered UB. This behaviour is exposed via `project_unchecked`. This can be used by implementations of the `Project` trait for smart pointers that are always valid, like `&T` to enable better codegen. `wrapping_project` on the other hand will not assert `inbounds`, and will just wrap around if the pointer offset is larger than `usize::max_value()`. This safe defined behaviour, even if it is almost always a bug, unlike `project_unchecked` which is UB on invalid pointers or offsets. -This corresponds with `std::ptr::add` and `std::ptr::wrapping_add` in safety and behaviour. `project_unchecked` and `project` need to be implemented as intrinsics because there is no way to assert that the pointer metadata for fat pointers of `Field::Parent` and `Field::Type` will always match in general without some other compiler support. This is necessary to allow unsized types to be used transparently with this scheme. +This corresponds with `core::ptr::add` and `core::ptr::wrapping_add` in safety and behaviour. `project_unchecked` and `project` need to be implemented as intrinsics because there is no way to assert that the pointer metadata for fat pointers of `Field::Parent` and `Field::Type` will always match in general without some other compiler support. This is necessary to allow unsized types to be used transparently with this scheme. `inverse_project_unchecked` and `inverse_wrapping_project` are just like their counterparts in safety. `inverse_project_unchecked` is UB to use on invalid pointers, where `inverse_wrapping_project` just wraps around on overflow. But there is one important safety check that must be performed before dereferencing the resulting pointer. The resulting pointer may not actually point to a valid `*const F::Parent` if `*const F::Type` does not live inside of a `F::Parent`, so one must take care to ensure that the parent pointer is indeed valid. This is different from `project_unchecked` and `wrapping_project` because there one only needs to validate the original pointer, not the resulting pointer. -`inverse_project_unchecked` and `inverse_wrapping_project` correspond to `std::ptr::sub` and `std::ptr::wrapping_sub` in safety and behaviour. They also must be implemented as intrinsics for the same reasons as stated above. +`inverse_project_unchecked` and `inverse_wrapping_project` correspond to `core::ptr::sub` and `core::ptr::wrapping_sub` in safety and behaviour. They also must be implemented as intrinsics for the same reasons as stated above. For example of where `project_unchecked` would be UB. @@ -194,7 +194,7 @@ If the raw pointer is valid, then the result of both `project_unchecked` and `wr ## Type and Traits -The `Field` trait and the `FieldDescriptor` type will live inside the `std::project` module, and will not be added to `prelude`. +The `Field` trait and the `FieldDescriptor` type will live inside the `core::project` module, and will not be added to `prelude`. The `Field` trait will only be implemented by the compiler, and it compiler should make sure that no other implementations exist. This allows unsafe code to assume that the implmentors or the `Field` trait always refer to valid fields. The `FieldDescriptor` type may be unnecessary raw pointer projections are implemented via intrinsics, if so we can remove it entirely. @@ -297,7 +297,7 @@ unsafe impl PinProjectable for Foo.bar {} /// main.rs fn main() { - use std::project::Project; + use core::project::Project; use foo::Foo; // These type annotations are unnecessary, but I put them in for clarity @@ -317,8 +317,8 @@ We still need some `unsafe` to tell that it is indeed safe to project through th ## Reference-level explanation (`Project` and `PinProjectable`) -The `Project` trait will live inside the `std::project` module, and will not be added to `prelude`. -The `PinProjectable` trait will be added to `std::marker`, and will also not be added to `prelude`. +The `Project` trait will live inside the `core::project` module, and will not be added to `prelude`. +The `PinProjectable` trait will be added to `core::marker`, and will also not be added to `prelude`. The `Project` trait will be implemented for `*const T`, `*mut T`, `&T`, `&mut T`, `Pin<&T>`, `Pin<&mut T>`. Other smart pointers can get implementations later if they need them. We may also provide the following implementations to allow better documentation of intent @@ -331,7 +331,7 @@ Because the following impls conflict, ```rust unsafe impl PinProjectable for F where F::Parent: Unpin {} -struct Foo(std::marker::PhantomPinned); +struct Foo(core::marker::PhantomPinned); unsafe impl PinProjectable for Foo.0 {} ``` From 6da5da24ef7f8d5d2693db52ebfd6a20ae392435 Mon Sep 17 00:00:00 2001 From: Krishna Sannasi Date: Mon, 24 Jun 2019 21:23:21 -0700 Subject: [PATCH 19/27] Update text/0000-ptr-to-field.md Co-Authored-By: Christopher Durham --- text/0000-ptr-to-field.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index a89c8a72845..dbd4dae371e 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -148,7 +148,7 @@ All field types will implement the following traits: `Debug`, `Clone`, `Copy`, ` They will also need to interact with the privacy rules for fields, and will have the same privacy as the field that they are derived from. -You can make a field type for the following types of types (some example syntax is shown) +You can make a field type for the following kinds of types (some example syntax is shown) * tuples `(T, U, ...)` * `<(T, U)>.0` * tuple structs `Foo(T, U, ...)` From 97e2a7f577926c0c8f2d0aff99e3f80325e34fe5 Mon Sep 17 00:00:00 2001 From: Krishna Sannasi Date: Mon, 24 Jun 2019 21:57:43 -0700 Subject: [PATCH 20/27] Update text/0000-ptr-to-field.md Co-Authored-By: Christopher Durham --- text/0000-ptr-to-field.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index dbd4dae371e..3f4a60fc53d 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -157,7 +157,7 @@ You can make a field type for the following kinds of types (some example syntax * `Foo.field` * unions `union Foo { field: Field }` * same syntax as structs - * constructing a field type is `unsafe`, this is because accessing fields of `union`s is unsafe + * accessing a field type is `unsafe`, because accessing fields of `union`s is unsafe The compiler can decide whether to actual generate a field type, this is to help compile times (if you don't use field types, then the compiler shouldn't slow down too much because of this feature). From 9d78b376ecb781198ee204d5e8f3c5e1847ca81d Mon Sep 17 00:00:00 2001 From: Ozaren Date: Mon, 24 Jun 2019 21:59:19 -0700 Subject: [PATCH 21/27] updated wording based off of @CAD97's suggestions updated `FiedDescriptor` type to make it fool-proof. added example of inverse projection UB and invalid pointers to complement projections. --- text/0000-ptr-to-field.md | 57 ++++++++++++++++++++++++++++----------- 1 file changed, 42 insertions(+), 15 deletions(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index a89c8a72845..3282609430e 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -37,12 +37,7 @@ A field type for the `name` field of `Person` would be `Person.name`. Note about syntax: the syntax `Type.field` is going to be used as a placeholder syntax for the rest of this RFC, but this is *not* the final syntax for field types, that decision will be made after this RFC gets accepted but before it gets stabilized. -Field types serve a few important purposes -* They are integrated with the new `Project` trait (explained later) -* Becaue they are types, they can implement traits - * This will allow conditional implementations of `Project`, which is important for `Pin

` (also explained later) - -Because they are types we can also generalize over field types like so... +Field types serve one important purpose, because they are types, they can implement traits, and thus be generalized over. This allows for powerful compile time guarantees that field types are being used correctly. ### `trait Field` and `type FieldDescriptor` @@ -56,12 +51,10 @@ trait Field { /// The type of the field itself type Type: ?Sized; - const FIELD_DESCRIPTOR: FieldDescriptor; + const FIELD_DESCRIPTOR: FieldDescriptor; } -struct FieldDescriptor { - ... -} +struct FieldDescriptor { .. } ``` `FieldDescriptor` is an opaque type that will store some metadata about how to convert from a `*const Field::Parent` to a `*const Field::Type`. There is no way to safely construct `FieldDescriptor` from user code on Stable Rust until Rust gets a defined stable type layout for `repr(Rust)` types. @@ -163,15 +156,19 @@ The compiler can decide whether to actual generate a field type, this is to help ## `*const T`/`*mut T` -`project_unchecked` and `wrapping_project` will both live inside of `core::ptr`. +`project_unchecked`, `wrapping_project`, `inverse_project_unchecked`, and `inverse_wrapping_project` will all be inherent methods on `*const T` and `*mut T`. They will be implemented on top of intrinsics (names of which don't matter), and those intrinsics will live in `core::intrinsics` (as all intrinsics do). + +raw pointer projections need to be implemented as intrinsics because there is no way to assert that the pointer metadata for fat pointers of `Field::Parent` and `Field::Type` will always match in general without some other compiler support. This is necessary to allow unsized types to be used transparently with this scheme. (See more details in next section). We need both `project_unchecked` and `wrapping_project` because there are some important optimization available inside of LLVM related to aliasing and escape analysis. In particular the LLVM `inbounds` assertion tells LLVM that a pointer offset stays within the same allocation and if the pointer is invalid or the offset does not stay within the same allocation it is considered UB. This behaviour is exposed via `project_unchecked`. This can be used by implementations of the `Project` trait for smart pointers that are always valid, like `&T` to enable better codegen. `wrapping_project` on the other hand will not assert `inbounds`, and will just wrap around if the pointer offset is larger than `usize::max_value()`. This safe defined behaviour, even if it is almost always a bug, unlike `project_unchecked` which is UB on invalid pointers or offsets. -This corresponds with `core::ptr::add` and `core::ptr::wrapping_add` in safety and behaviour. `project_unchecked` and `project` need to be implemented as intrinsics because there is no way to assert that the pointer metadata for fat pointers of `Field::Parent` and `Field::Type` will always match in general without some other compiler support. This is necessary to allow unsized types to be used transparently with this scheme. +This corresponds with `core::ptr::add` and `core::ptr::wrapping_add` in safety and behaviour. -`inverse_project_unchecked` and `inverse_wrapping_project` are just like their counterparts in safety. `inverse_project_unchecked` is UB to use on invalid pointers, where `inverse_wrapping_project` just wraps around on overflow. But there is one important safety check that must be performed before dereferencing the resulting pointer. The resulting pointer may not actually point to a valid `*const F::Parent` if `*const F::Type` does not live inside of a `F::Parent`, so one must take care to ensure that the parent pointer is indeed valid. This is different from `project_unchecked` and `wrapping_project` because there one only needs to validate the original pointer, not the resulting pointer. +`inverse_project_unchecked` and `inverse_wrapping_project` are have all of the same safety requirements as their counterparts, and some more. -`inverse_project_unchecked` and `inverse_wrapping_project` correspond to `core::ptr::sub` and `core::ptr::wrapping_sub` in safety and behaviour. They also must be implemented as intrinsics for the same reasons as stated above. +`inverse_project_unchecked` and `inverse_wrapping_project` produces a valid pointer without UB if and only if the initial pointer is both valid and points to a field in the parent type used to perform the inverse projection. `inverse_wrapping_project` will never cause UB, but may produce invalid pointers. `inverse_project_unchecked` will cause UB if the condition above is not met. This is different from `project_unchecked` and `wrapping_project` because they only need to validate the original pointer, not the resulting pointer. + +`inverse_project_unchecked` and `inverse_wrapping_project` correspond to `core::ptr::sub` and `core::ptr::wrapping_sub` in safety and behaviour. For example of where `project_unchecked` would be UB. @@ -184,14 +181,44 @@ let x : *const Foo = 2usize as *const Foo; let y : *const Bar = x.project_unchecked(Foo.bar); // UB, x does not point to a valid instance of Foo ``` -With `wrapping_project` trait +With `wrapping_project` ```rust let z : *const Bar = x.wrapping_project(Foo.bar); // not UB, but is still invalid ``` +For example of where `inverse_project_unchecked` would be UB. + +```rust +struct Foo { + bar: Bar +} + +let x : *const Foo = 2usize as *const Bar; +let y : *const Bar = x.inverse_project_unchecked(Foo.bar); // UB, x does not point to a valid instance of Bar + +let v : Bar = Bar(...); +let v : *const Bar = &v; +let y : *const Bar = v.inverse_project_unchecked(Foo.bar); // UB, v does not point to a field of Foo +``` + +With `inverse_wrapping_project` + +```rust + +let y : *const Bar = x.inverse_wrapping_project(Foo.bar); // not UB, but y is invalid + +let y : *const Bar = v.inverse_wrapping_project(Foo.bar); // not UB, but y is invalid +``` + If the raw pointer is valid, then the result of both `project_unchecked` and `wrapping_project` is a raw pointer to the given field. +## Pointer Metadata + +When projecting to a field the pointer metadata must be preserved. Currently types can only have 1 DST field, which means and we only have two forms of primitive DSTs, slices and trait objects. Slices's metadata is always a `usize` length, and trait objects always have a pointer to their v-table. When projecting to the DST. For either case the metadata should just be copied over to the resulting pointer if and only if the resulting pointer points to the DST field. + +If Rust ever gets multiple DST fields, pointer projections will need to adapt to handle that. If Rust gets Custom DSTs, then we must consider how that proposal affects pointer metadata. + ## Type and Traits The `Field` trait and the `FieldDescriptor` type will live inside the `core::project` module, and will not be added to `prelude`. From dece0b1ad35a0fd01eb5429e4c19c99fba8128b7 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Mon, 24 Jun 2019 22:10:18 -0700 Subject: [PATCH 22/27] editted note about pointer metadata --- text/0000-ptr-to-field.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index f1d631631bc..8a25cde98a5 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -215,7 +215,7 @@ If the raw pointer is valid, then the result of both `project_unchecked` and `wr ## Pointer Metadata -When projecting to a field the pointer metadata must be preserved. Currently types can only have 1 DST field, which means and we only have two forms of primitive DSTs, slices and trait objects. Slices's metadata is always a `usize` length, and trait objects always have a pointer to their v-table. When projecting to the DST. For either case the metadata should just be copied over to the resulting pointer if and only if the resulting pointer points to the DST field. +When projecting to a field the pointer metadata must be preserved. Currently types can only have 1 DST field, meaning that the metadata on the parent is exactly the same as the metadata on the child (only DST field). We only have two forms of primitive DSTs, slices and trait objects. These last two facts mean that every DST must eventually boil down to one of these. Slices's metadata is always a `usize` length, and trait objects always have a pointer to their v-table. So when projecting a raw pointer to a DST field in either case the metadata should just be copied over to the resulting pointer. This works because we only have slices and trait objects as primitive DSTs. For every other type of projection the metadata will be zero-sized, so there isn't anything to worry about. If Rust ever gets multiple DST fields, pointer projections will need to adapt to handle that. If Rust gets Custom DSTs, then we must consider how that proposal affects pointer metadata. @@ -252,6 +252,9 @@ The `Field` trait will only be implemented by the compiler, and it compiler shou - Some other variations of the syntax are ... - `Type::field` // This is bad because it conflicts with associated method, and there isn't a way to disambiguate them easily - `Type~field` // This adds a new sigil to the language +- How will pointer metadata be handled for more exotic Custom DSTs? + - see @CAD97's example [here](https://github.com/rust-lang/rfcs/pull/2708#discussion_r296933269) + > Note that this doesn't require equivalent metadata: a theoretical pointer with two metadatas could be split into two child fat pointers with one or the other metadata. # Future possibilities [future-possibilities]: #future-possibilities @@ -259,6 +262,7 @@ The `Field` trait will only be implemented by the compiler, and it compiler shou - Extend the `Project` trait to implement all smart pointers in the standard library - [`InitPtr`](https://internals.rust-lang.org/t/idea-pointer-to-field/10061/72), which encapsulates all of the safety requirements of `project_unchecked` into `InitPtr::new` and safely implements `Project` - Distant Future, we could reformulate `Copy` based on the `Field` trait so that it enforces that all of the fields of a type must be `Copy` in order to be the type to be `Copy`, and thus reduce the amount of magic in the compiler. +- Integration with Custom DSTs - a `Project` and `PinProjectable` traits - These were in an earier version of this RFC, but were removed as they are not essential, and this RFC is already rather niche. So to limit the scope of this RFC, they were removed. If this RFC is accepted, then `Project` and `PinProjectable` could be implmented as library items outside of `std`. From 2e77eefffc2918262a5fc0c6585095067390f3cb Mon Sep 17 00:00:00 2001 From: Krishna Sannasi Date: Tue, 25 Jun 2019 09:41:24 -0700 Subject: [PATCH 23/27] Update text/0000-ptr-to-field.md Co-Authored-By: Christopher Durham --- text/0000-ptr-to-field.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 8a25cde98a5..10c72810dba 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -160,7 +160,7 @@ The compiler can decide whether to actual generate a field type, this is to help raw pointer projections need to be implemented as intrinsics because there is no way to assert that the pointer metadata for fat pointers of `Field::Parent` and `Field::Type` will always match in general without some other compiler support. This is necessary to allow unsized types to be used transparently with this scheme. (See more details in next section). -We need both `project_unchecked` and `wrapping_project` because there are some important optimization available inside of LLVM related to aliasing and escape analysis. In particular the LLVM `inbounds` assertion tells LLVM that a pointer offset stays within the same allocation and if the pointer is invalid or the offset does not stay within the same allocation it is considered UB. This behaviour is exposed via `project_unchecked`. This can be used by implementations of the `Project` trait for smart pointers that are always valid, like `&T` to enable better codegen. `wrapping_project` on the other hand will not assert `inbounds`, and will just wrap around if the pointer offset is larger than `usize::max_value()`. This safe defined behaviour, even if it is almost always a bug, unlike `project_unchecked` which is UB on invalid pointers or offsets. +We need both `project_unchecked` and `wrapping_project` because there are some important optimization available inside of LLVM related to aliasing and escape analysis. In particular the LLVM `inbounds` assertion tells LLVM that a pointer offset stays within the same allocation and if the pointer is invalid or the offset does not stay within the same allocation it is considered UB. This behaviour is exposed via `project_unchecked`. This can be used when the pointer is known without a doubt to be valid, such as when derived from `&T`, to enable better codegen. `wrapping_project` on the other hand will not assert `inbounds`, and will just wrap around if the pointer offset is larger than `usize::max_value()`. This safe defined behaviour, even if it is almost always a bug, unlike `project_unchecked` which is UB on invalid pointers or offsets. This corresponds with `core::ptr::add` and `core::ptr::wrapping_add` in safety and behaviour. From e0e7e804415c6db3427b99cc5a38f8e1691696d3 Mon Sep 17 00:00:00 2001 From: Krishna Sannasi Date: Tue, 25 Jun 2019 09:49:47 -0700 Subject: [PATCH 24/27] Apply suggestions from code review From @CAD97's review Co-Authored-By: Christopher Durham --- text/0000-ptr-to-field.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 10c72810dba..215e77acf19 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -166,7 +166,7 @@ This corresponds with `core::ptr::add` and `core::ptr::wrapping_add` in safety a `inverse_project_unchecked` and `inverse_wrapping_project` are have all of the same safety requirements as their counterparts, and some more. -`inverse_project_unchecked` and `inverse_wrapping_project` produces a valid pointer without UB if and only if the initial pointer is both valid and points to a field in the parent type used to perform the inverse projection. `inverse_wrapping_project` will never cause UB, but may produce invalid pointers. `inverse_project_unchecked` will cause UB if the condition above is not met. This is different from `project_unchecked` and `wrapping_project` because they only need to validate the original pointer, not the resulting pointer. +`inverse_project_unchecked` and `inverse_wrapping_project` produces a valid pointer without UB if and only if the initial pointer is both valid and points to a field in the parent type used to perform the inverse projection. `inverse_wrapping_project` will never cause UB, but may produce invalid pointers. `inverse_project_unchecked` will cause UB if the condition above is not met. This is different from `project_unchecked` and `wrapping_project` because they only need to prove the validity of the original pointer, and not the context in which it resides. `inverse_project_unchecked` and `inverse_wrapping_project` correspond to `core::ptr::sub` and `core::ptr::wrapping_sub` in safety and behaviour. @@ -184,7 +184,7 @@ let y : *const Bar = x.project_unchecked(Foo.bar); // UB, x does not point to a With `wrapping_project` ```rust -let z : *const Bar = x.wrapping_project(Foo.bar); // not UB, but is still invalid +let z : *const Bar = x.wrapping_project(Foo.bar); // not UB, but still produces an invalid pointer ``` For example of where `inverse_project_unchecked` would be UB. @@ -259,7 +259,6 @@ The `Field` trait will only be implemented by the compiler, and it compiler shou # Future possibilities [future-possibilities]: #future-possibilities -- Extend the `Project` trait to implement all smart pointers in the standard library - [`InitPtr`](https://internals.rust-lang.org/t/idea-pointer-to-field/10061/72), which encapsulates all of the safety requirements of `project_unchecked` into `InitPtr::new` and safely implements `Project` - Distant Future, we could reformulate `Copy` based on the `Field` trait so that it enforces that all of the fields of a type must be `Copy` in order to be the type to be `Copy`, and thus reduce the amount of magic in the compiler. - Integration with Custom DSTs From da3afb66ed73efe2246bb70e23723925b360a9c7 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Tue, 25 Jun 2019 09:50:13 -0700 Subject: [PATCH 25/27] Some more changes from @CAD97's review --- text/0000-ptr-to-field.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 8a25cde98a5..5dbd766135a 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -91,8 +91,6 @@ impl *mut T { These will allowing projections through raw pointers without dereferencing the raw pointer. This is useful for building projections through other abstractions like smart pointers (`Rc`, `Pin<&T>`)! -This is the extent of the core api of this RFC. - Using this we can do something like this ```rust struct Foo { @@ -162,13 +160,13 @@ raw pointer projections need to be implemented as intrinsics because there is no We need both `project_unchecked` and `wrapping_project` because there are some important optimization available inside of LLVM related to aliasing and escape analysis. In particular the LLVM `inbounds` assertion tells LLVM that a pointer offset stays within the same allocation and if the pointer is invalid or the offset does not stay within the same allocation it is considered UB. This behaviour is exposed via `project_unchecked`. This can be used by implementations of the `Project` trait for smart pointers that are always valid, like `&T` to enable better codegen. `wrapping_project` on the other hand will not assert `inbounds`, and will just wrap around if the pointer offset is larger than `usize::max_value()`. This safe defined behaviour, even if it is almost always a bug, unlike `project_unchecked` which is UB on invalid pointers or offsets. -This corresponds with `core::ptr::add` and `core::ptr::wrapping_add` in safety and behaviour. +This corresponds with `<*const _>::add` and `<*const _>::wrapping_add` in safety and behaviour. `inverse_project_unchecked` and `inverse_wrapping_project` are have all of the same safety requirements as their counterparts, and some more. `inverse_project_unchecked` and `inverse_wrapping_project` produces a valid pointer without UB if and only if the initial pointer is both valid and points to a field in the parent type used to perform the inverse projection. `inverse_wrapping_project` will never cause UB, but may produce invalid pointers. `inverse_project_unchecked` will cause UB if the condition above is not met. This is different from `project_unchecked` and `wrapping_project` because they only need to validate the original pointer, not the resulting pointer. -`inverse_project_unchecked` and `inverse_wrapping_project` correspond to `core::ptr::sub` and `core::ptr::wrapping_sub` in safety and behaviour. +`inverse_project_unchecked` and `inverse_wrapping_project` correspond to `<*const _>::sub` and `<*const _>::wrapping_sub` in safety and behaviour. For example of where `project_unchecked` would be UB. From 21cf26b3fc2cc96a8b4997d43df0fecc354401ef Mon Sep 17 00:00:00 2001 From: Ozaren Date: Tue, 25 Jun 2019 09:51:12 -0700 Subject: [PATCH 26/27] added note about enum variants as types --- text/0000-ptr-to-field.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 5dbd766135a..ed45700e4e3 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -261,6 +261,8 @@ The `Field` trait will only be implemented by the compiler, and it compiler shou - [`InitPtr`](https://internals.rust-lang.org/t/idea-pointer-to-field/10061/72), which encapsulates all of the safety requirements of `project_unchecked` into `InitPtr::new` and safely implements `Project` - Distant Future, we could reformulate `Copy` based on the `Field` trait so that it enforces that all of the fields of a type must be `Copy` in order to be the type to be `Copy`, and thus reduce the amount of magic in the compiler. - Integration with Custom DSTs +- Integration with a possible Enum Variants as Types feature + - This will allow projecting to enum variants safely - a `Project` and `PinProjectable` traits - These were in an earier version of this RFC, but were removed as they are not essential, and this RFC is already rather niche. So to limit the scope of this RFC, they were removed. If this RFC is accepted, then `Project` and `PinProjectable` could be implmented as library items outside of `std`. From 00b4d4bfc1ac490eb60162e302a39bbb0ad786f6 Mon Sep 17 00:00:00 2001 From: Ozaren Date: Tue, 25 Jun 2019 10:00:58 -0700 Subject: [PATCH 27/27] fixed inverse projecting UB examples formatting added note about the future possiblity of the Project trait. --- text/0000-ptr-to-field.md | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/text/0000-ptr-to-field.md b/text/0000-ptr-to-field.md index 5f6361de408..be5e3b4ebbd 100644 --- a/text/0000-ptr-to-field.md +++ b/text/0000-ptr-to-field.md @@ -92,6 +92,7 @@ impl *mut T { These will allowing projections through raw pointers without dereferencing the raw pointer. This is useful for building projections through other abstractions like smart pointers (`Rc`, `Pin<&T>`)! Using this we can do something like this + ```rust struct Foo { bar: Bar, @@ -111,7 +112,7 @@ let y_bar_name: *const String = unsafe { y.project_unchecked(Foo.bar).project_un In the end `y_bar_name` will contain a pointer to `x.bar.name`, all without dereferencing a single pointer! (Given that this is a verbose, we may want some syntax for this, but that is out of scope for this RFC) -But, we can build on this foundation and create a more power abstraction, to generalize this project notion to smart pointers. +But, we can build on this foundation and create a more power abstraction, to generalize this project notion to smart pointers. See future possibilities section for some ideas about how this could work, via the `Project` trait. # Reference-level explanation [reference-level-explanation]: #reference-level-explanation @@ -123,6 +124,7 @@ This section will go in detail about the semantics and implementation of the typ Field types are just sugar for unit types (`struct Foo;`) that are declared right next to the `Parent` type. Like so, + ```rust struct Person { pub name: String, @@ -156,7 +158,7 @@ The compiler can decide whether to actual generate a field type, this is to help `project_unchecked`, `wrapping_project`, `inverse_project_unchecked`, and `inverse_wrapping_project` will all be inherent methods on `*const T` and `*mut T`. They will be implemented on top of intrinsics (names of which don't matter), and those intrinsics will live in `core::intrinsics` (as all intrinsics do). -raw pointer projections need to be implemented as intrinsics because there is no way to assert that the pointer metadata for fat pointers of `Field::Parent` and `Field::Type` will always match in general without some other compiler support. This is necessary to allow unsized types to be used transparently with this scheme. (See more details in next section). +Raw pointer projections need to be implemented as intrinsics because there is no way to assert that the pointer metadata for fat pointers of `Field::Parent` and `Field::Type` will always match in general without some other compiler support. This is necessary to allow unsized types to be used transparently with this scheme. (See more details in next section). We need both `project_unchecked` and `wrapping_project` because there are some important optimization available inside of LLVM related to aliasing and escape analysis. In particular the LLVM `inbounds` assertion tells LLVM that a pointer offset stays within the same allocation and if the pointer is invalid or the offset does not stay within the same allocation it is considered UB. This behaviour is exposed via `project_unchecked`. This can be used when the pointer is known without a doubt to be valid, such as when derived from `&T`, to enable better codegen. `wrapping_project` on the other hand will not assert `inbounds`, and will just wrap around if the pointer offset is larger than `usize::max_value()`. This safe defined behaviour, even if it is almost always a bug, unlike `project_unchecked` which is UB on invalid pointers or offsets. @@ -192,21 +194,21 @@ struct Foo { bar: Bar } -let x : *const Foo = 2usize as *const Bar; -let y : *const Bar = x.inverse_project_unchecked(Foo.bar); // UB, x does not point to a valid instance of Bar +let x : *const Bar = 2usize as *const Bar; +let y : *const Foo = x.inverse_project_unchecked(Foo.bar); // UB, x does not point to a valid instance of Bar let v : Bar = Bar(...); let v : *const Bar = &v; -let y : *const Bar = v.inverse_project_unchecked(Foo.bar); // UB, v does not point to a field of Foo +let y : *const Foo = v.inverse_project_unchecked(Foo.bar); // UB, v does not point to a field of Foo ``` With `inverse_wrapping_project` ```rust -let y : *const Bar = x.inverse_wrapping_project(Foo.bar); // not UB, but y is invalid +let y : *const Foo = x.inverse_wrapping_project(Foo.bar); // not UB, but y is invalid -let y : *const Bar = v.inverse_wrapping_project(Foo.bar); // not UB, but y is invalid +let y : *const Foo = v.inverse_wrapping_project(Foo.bar); // not UB, but y is invalid ``` If the raw pointer is valid, then the result of both `project_unchecked` and `wrapping_project` is a raw pointer to the given field. @@ -232,6 +234,7 @@ The `Field` trait will only be implemented by the compiler, and it compiler shou [rationale-and-alternatives]: #rationale-and-alternatives - The `&[mut] raw T` could solve some of the problems, but only for raw pointers. It doesn't help with abstractions. + - Somehow expand on `Deref` to allow dereferencing to a smart pointer - This would require Generic Associated Types at the very least, and maybe some other features like assocaited traits @@ -239,6 +242,7 @@ The `Field` trait will only be implemented by the compiler, and it compiler shou [prior-art]: #prior-art - C++'s pointer to members `Parent::*field` + - Java's `class Field` - Similar reflection capabilies in other languages @@ -250,6 +254,7 @@ The `Field` trait will only be implemented by the compiler, and it compiler shou - Some other variations of the syntax are ... - `Type::field` // This is bad because it conflicts with associated method, and there isn't a way to disambiguate them easily - `Type~field` // This adds a new sigil to the language + - How will pointer metadata be handled for more exotic Custom DSTs? - see @CAD97's example [here](https://github.com/rust-lang/rfcs/pull/2708#discussion_r296933269) > Note that this doesn't require equivalent metadata: a theoretical pointer with two metadatas could be split into two child fat pointers with one or the other metadata. @@ -257,9 +262,12 @@ The `Field` trait will only be implemented by the compiler, and it compiler shou # Future possibilities [future-possibilities]: #future-possibilities -- [`InitPtr`](https://internals.rust-lang.org/t/idea-pointer-to-field/10061/72), which encapsulates all of the safety requirements of `project_unchecked` into `InitPtr::new` and safely implements `Project` +- [`InitPtr`](https://internals.rust-lang.org/t/idea-pointer-to-field/10061/72), which encapsulates all of the safety requirements of `project_unchecked` into `InitPtr::new` and safely implements `Project` (see below for `Project`) + - Distant Future, we could reformulate `Copy` based on the `Field` trait so that it enforces that all of the fields of a type must be `Copy` in order to be the type to be `Copy`, and thus reduce the amount of magic in the compiler. + - Integration with Custom DSTs + - Integration with a possible Enum Variants as Types feature - This will allow projecting to enum variants safely @@ -303,6 +311,7 @@ unsafe trait PinProjectable {} Due to some safety requirements that will be detailed in the reference-level explanation, we can't just freely hand out pin projections to every type (sad as it is). To enable pin projections to a field, a field type must implement `PinProjectable`. Like so, + ```rust unsafe impl PinProjectable for Foo.field {} ``` @@ -311,6 +320,7 @@ unsafe impl PinProjectable for Foo.field {} Here is a toy example of how to use this api: Given the implementation of `Project for Pin<&mut T>` + ```rust /// Some other crate foo @@ -365,6 +375,7 @@ struct Foo(core::marker::PhantomPinned); unsafe impl PinProjectable for Foo.0 {} ``` + [Proof](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=b0796a8b631e0fec1804318caef162c7) I think making `PhantomPinned` a lang-item that is known to always implment `!Unpin` would solve this. This way only those who implment `!Unpin` types need to worry about implementing `PinProjectable`. Another way to solve this would be to somehow make `PinProjectable` a lang-item that allows this one case of conflicting impls. But I am unsure of how to properly handle this, both way s that I showed seem unsatisfactory. The blanket impl is highly desirable, because it enables those who don't write `!Unpin` types to ignore safe pin projections, and still have them available.