From 634b4cf122851289a7451f17d659eb3d41ee1545 Mon Sep 17 00:00:00 2001 From: Vladimir Sadov Date: Thu, 20 Apr 2017 18:05:10 -0700 Subject: [PATCH 1/2] Updated draft for the safety of ref-like types such as `Span` --- proposals/span-safety.md | 188 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 188 insertions(+) create mode 100644 proposals/span-safety.md diff --git a/proposals/span-safety.md b/proposals/span-safety.md new file mode 100644 index 0000000000..39a13aded7 --- /dev/null +++ b/proposals/span-safety.md @@ -0,0 +1,188 @@ +# Compile time enforcement of safety for ref-like types # + +The main reason for the additional safety rules when dealing with types like `Span` and `ReadonlySpan` is that such types must be confined to the execution stack. + +There are two reasons why `Span` and similar types must be a stack-only types. + +1. `Span` is semantically a struct containing a reference and a range - `(ref T data, int length)`. Regardless of actual implementation, writes to such struct would not be atomic. Concurrent "tearing" of such struct would lead to the possibility of `length` not matching the `data`, causing out-of-range accesses and type-safety violations, which ultimately could result in GC heap corruption in seemingly "safe" code. +2. Some implementations of `Span` literally contain a managed pointer in one of its fields. Managed pointers are not supported as fields of heap objects and code that manages to put a managed pointer on the GC heap typically crashes at JIT time. + +All the above problems would be alleviated if instances of `Span` are constrained to exist only on the execution stack. + +An additional problem arises due to composition. It would be generally desirable to build more complex data types that would embed `Span` and `ReadonlySpan` instances. Such composite types would have to be structs and would share all the hazards and requirements of `Span`. As a result the safety rules described here should be viewed as applicable to the whole range of **_ref-like types_** + +## `ref-like` types must be stack-only. ## + +C# compiler already has a concept of types with stack-only requirements that covers special platform types such as `TypedReference`. In some cases those types are referred as ref-like types as well. We should probably use some different term to refer to `TypedReference` and similar types - like `restricted types` to avoid confusion. + +In this document the "ref-like" types include only `Span` and related types. And the ref-like variables mean variables of "ref-like" types. + +In order to force ref-like variables to be stack only, we need the following restrictions: + +- ref-like type cannot be a type of an array element +- ref-like type cannot be used as a generic type argument +- ref-like variable cannot be boxed +- ref-like type cannot be a static field +- ref-like type cannot be an instance field of ordinary not ref-like type +- indirect restrictions, such as disallowed use of ref-like typed parameters and locals in async methods, which are really a result of disallowing ref-like typed fields. + +Note: + +- it is ok to return ref-like values from methods/lambdas, including by reference. +- it is ok to pass ref-like values to methods/lambdas, including by reference. +- it is ok for a ref-like type be a field of another struct as long as the struct itself is ref-like. +- it is ok for an intermediate span values to be used in lambdas/async methods. + + +*Possible incremental relaxations* +- It may be possible to allow span locals be defined and used in a block inside an async method as long as the block does not contain `await` expressions. + + +## Stack-referring spans + +An additional desirable enhancement for `Span` is to allow referring to data in the local stack frame - individual local variables or `stackalloc`-ed arrays. + +The tricky part here is that a scenario when an instance of a `Span` outlives the referred data would lead to undefined behavior, including type-safety violations and heap corruptions. Therefore, in order to allow stack-referring spans in safe code, we need additional rules. + +The design goal here is to make rules for stack-referring spans "pay-for-play". +**A person who does not intend to trade in stack-referring spans should be able to stop reading right here.** + +NOTE: nothing will prevent a possibility to use Spans that wrap pointers to stack-allocated arrays in _unsafe_ code as long as it is possible to wrap a pointer in a span. As per usual rules of _unsafe_, it would be responsibility of the user to not cause GC holes. + + +## Stack-referring spans in safe code. + +There are two main scenarios where stack-referring spans could come into being. +- Stackalloc spans. + To enable safe wrapping of stack allocated memory in a span, we will need a special syntax that would not allow intermediate pointer to escape. + A preliminary solution is to use target-typing. + +```cs + //This usage of stackalloc does not require unsafe + Span sp = stackalloc int[100]; +``` + +- Spans wrapping individual locals or parameters. [Suggest disallowing] + +```cs + int x = 42; + + //This is currently not possible + Span sp = new Span(ref x); +``` + + This is currently not possible in Span API. I suggest to never allow. + + While in theory possible, one-element spans are not as practically appealing as stackalloc ones. + + One problem is that in addition to being tied to the stack frame, local variables have scopes, so we would need extra rules to prevent mixing locals of different life-times when passing by reference or assigning to other locals. + + The main problem is the "pay-for-play" rule. If we have to assume that any span value returned from a member that takes `ref/out/in` parameters, including `this` in struct members might be stack-referring. People that do not use stack-referring spans will be affected by this. + +```cs + int x = 42; + + //Do I really have a stack-referring span here? + Span sp = SomeMethod(ref x); + +``` + +## Safety rules for stack-referring spans. +Note: all these rules equally apply to ref-like types in general. I just use span for brevity. + +- stack-referring span values cannot be returned from a method. + +- local variable initialized using a stack-referring expression is considered a stack-referring local. + +- stack-referring local is conservatively considered as always returning a stack-referring value. + +- stack-referring values cannot be assigned to any variable with exception of stack-referring locals. + +- "What comes in is what comes out" rule for span-returning expressions: + If any operand of an expression is a stack-referring value, the result is a stack-referring value. + This applies equally to ternary expressions, indexers, method calls, and so on. + Receiver of an invocation/property/indexer is considered an operand here. + + In particular: _Slice of a stack-referring value is a stack-referring value._ + +- "No Mixing with refs to ordinary" rule: + Stack-referring spans can be passed as an argument as long as no other argument is an ordinary span variable (not a stack-referring local), and: + a) is passed via a writeable reference `ref/out` or + b) is a receiver, and the containing span-like struct is not a readonly struct. + + +== +- [_only if allow referring to regular locals_] stack-referring value cannot be passed via a writeable reference. +- [_only if allow referring to regular locals_] stack-referring value cannot be assigned to a local except at initialization. + + +Examples: + +```c# + +SpanLikeType M1(ref SpanLikeType x, Span y) +{ + // this is all valid, unconcerned with stack-referring stuff + var local = new SpanLikeType(y); + x = local; + return x; +} + +void Test1(ref SpanLikeType param1, Span param2) +{ + Span stackReferring1 = stackalloc byte[10]; + var stackReferring2 = new SpanLikeType(stackReferring1); + + // this is allowed + stackReferring2 = M1(ref stackReferring2, stackReferring1); + + // this is NOT allowed + stackReferring2 = M1(ref param1, stackReferring1); + + // this is NOT allowed + param1 = M1(ref stackReferring2, stackReferring1); + + // this is NOT allowed + param2 = stackReferring1.Slice(10); + + // this is allowed + param1 = new SpanLikeType(param2); + + // this is allowed + stackReferring2 = param1; +} + +ref SpanLikeType M2(ref SpanLikeType x) +{ + return ref x; +} + +ref SpanLikeType Test2(ref SpanLikeType param1, Span param2) +{ + Span stackReferring1 = stackalloc byte[10]; + var stackReferring2 = new SpanLikeType(stackReferring1); + + ref var stackReferring3 = M2(ref stackReferring2); + + // this is allowed + stackReferring3 = M1(ref stackReferring2, stackReferring1); + + // this is allowed + M2(ref stackReferring3) = stackReferring2; + + // this is NOT allowed + M1(ref param1) = stackReferring2; + + // this is NOT allowed + param1 = stackReferring3; + + // this is NOT allowed + return ref stackReferring3; + + // this is allowed + return ref param1; +} + +``` + + From 3ab16a77b9fefe3805fa81044ba72d6c254e00f9 Mon Sep 17 00:00:00 2001 From: Vladimir Sadov Date: Mon, 1 May 2017 17:48:03 -0700 Subject: [PATCH 2/2] Added more info on ref-like structs in source and metadata. --- proposals/span-safety.md | 52 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 51 insertions(+), 1 deletion(-) diff --git a/proposals/span-safety.md b/proposals/span-safety.md index 39a13aded7..07e0df58c6 100644 --- a/proposals/span-safety.md +++ b/proposals/span-safety.md @@ -37,6 +37,57 @@ Note: *Possible incremental relaxations* - It may be possible to allow span locals be defined and used in a block inside an async method as long as the block does not contain `await` expressions. +## Generalized `ref-like` types in source code. + +`ref-like` structs will have to be explicitly marked in the source code using `ref` modifier: + +```c# +ref struct TwoSpans +{ + // can have ref-like instance fields + public Span first; + public Span second; +} + +// error: arrays of ref-like types are not allowed. +TwoSpans[] arr = null; + +``` + +Designating a struct as ref-like will allow the struct to have ref-like instance fields and will also make all the requirements of ref-like types applicable to the struct. + +An alternative proposal of "inferring" the ref-like property from the fact that a struct contains ref-like fields was also considered and rejected for the following reasons: + +1) Implicit cascading of ref-like through containing structs appears to be dangerous. +Since the implicit ref-like would work recursively, it would be too easy to turn multiple structs, possibly not in the current project, to become ref-like with one change. + +1) Span and ReadOnlySpan themselves do not contain ref-like fields. These types will have to be special-cased, which will be an issue if another trivial ref-like type needs to be added in the future. + +## Metadata representation or ref-like structs. + +Ref-like structs will be marked with **System.Runtime.CompilerServices.IsRefLikeAttribute** attribute. + +The attribute will be added to common base libraries such as `mscorlib`. In a case if the attribute is not available, compiler will generate an internal one similarly to other embedded-on-demand attributes such as `IsReadOnlyAttribute`. + +An additional measure will be taken to prevent the use of ref-like structs in compilers not familiar with the safety rules (this includes C# compilers prior to the one in which this feature is implemented). + +Having no other good alternatives that work in old compilers without servicing, an `Obsolete` attribute with a known string will be added to all ref-like structs. Compilers that know how to use ref-like types will ignore this particular form of `Obsolete`. + +A typical matadata representation: + +```C# + [IsRefLike] + [Obsolete("Types with embedded references are not supported in this version of your compiler.")] + public struct TwoSpans + { + . . . . + } +``` + +NOTE: it is not the goal to make it so that any use of ref-like types on old compilers fails 100%. That is hard to achieve and is not strictly necessary. For example there would always be away to get around the `Obsolete` using dynamic code or, for example, creating an array of ref-like types through reflection. + +In particular, if user wants to actually put an `Obsolete` attribute on a ref-like type, we will have no choice other than not emitting the predefined one since `Obsolete` attribute cannot be applied more than once.. +(TODO: We should consider giving a warning in such scenario, informing user of the danger.) ## Stack-referring spans @@ -49,7 +100,6 @@ The design goal here is to make rules for stack-referring spans "pay-for-play". NOTE: nothing will prevent a possibility to use Spans that wrap pointers to stack-allocated arrays in _unsafe_ code as long as it is possible to wrap a pointer in a span. As per usual rules of _unsafe_, it would be responsibility of the user to not cause GC holes. - ## Stack-referring spans in safe code. There are two main scenarios where stack-referring spans could come into being.