Skip to content

Commit

Permalink
Hook up DllImportGenerator to the libraries build (excluding solution…
Browse files Browse the repository at this point in the history
… file regeneration).

Move System.Runtime.InteropServices unit tests to a subdirectory.
  • Loading branch information
jkoritzinsky committed Sep 23, 2021
1 parent fc18f4d commit 4cfb51f
Show file tree
Hide file tree
Showing 211 changed files with 212 additions and 377 deletions.
4 changes: 2 additions & 2 deletions docs/design/libraries/DllImportGenerator/StructMarshalling.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ These types pose an interesting problem for a number of reasons listed below. Wi
- In the ref assemblies generated by dotnet/runtime, we save space and prevent users from relying on private implementation details of structures by emitting limited information about their fields. Structures that have at least one non-object field are given a private `int` field, and structures that have at least one field that transitively contains an object are given one private `object`-typed field. As a result, we do not have full type information at code-generation time for any structures defined in the BCL when compiling a library that uses the ref assemblies.
- Private reflection
- Even when we do have information about all of the fields, we can't emit code that references them if they are private, so we would have to emit unsafe code and calculate offsets manually to support marshaling them.

## Opt-in Interop

We've been working around another problem for a while in the runtime-integrated interop design: The user can use any type that is non-auto layout and has fields that can be marshaled in interop. This has lead to various issues where types that were not intended for interop usage became usable and then we couldn't update their behavior to be special-cased since users may have been relying on the generic behavior (`Span<T>`, `Vector<T>` come to mind).
Expand Down Expand Up @@ -213,7 +213,7 @@ Because the Roslyn compiler needs to be able to validate that there are not recu
To enable blittable generics support in this struct marshalling model, we extend `[BlittableType]` as follows:

- In a generic type definition, we consider all type parameters that can be value types as blittable for the purposes of validating that `[BlittableType]` is only applied to blittable types.
- When the source generator discovers a generic type marked with `[BlittableType]` it will look through the fields on the type and validate that they are blittable.
- When the source generator discovers a generic type marked with `[BlittableType]` it will look through the fields on the type and validate that they are blittable.

Since all fields typed with non-parameterized types are validated to be blittable at type definition time, we know that they are all blittable at type usage time. So, we only need to validate that the generic fields are instantiated with blittable types.

Expand Down
5 changes: 3 additions & 2 deletions eng/Versions.props
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@
</PropertyGroup>
<PropertyGroup>
<!-- Code analysis dependencies -->
<MicrosoftCodeAnalysisAnalyzersVersion>3.3.2</MicrosoftCodeAnalysisAnalyzersVersion>
<MicrosoftCodeAnalysisCSharpCodeStyleVersion>3.10.0</MicrosoftCodeAnalysisCSharpCodeStyleVersion>
<MicrosoftCodeAnalysisCSharpVersion>3.10.0</MicrosoftCodeAnalysisCSharpVersion>
<MicrosoftCodeAnalysisNetAnalyzersVersion>7.0.0-preview1.21460.1</MicrosoftCodeAnalysisNetAnalyzersVersion>
Expand Down Expand Up @@ -141,6 +142,7 @@
<XunitPerformanceApiPackageVersion>1.0.0-beta-build0015</XunitPerformanceApiPackageVersion>
<MicrosoftDiagnosticsToolsRuntimeClientVersion>1.0.4-preview6.19326.1</MicrosoftDiagnosticsToolsRuntimeClientVersion>
<MicrosoftDiagnosticsNETCoreClientVersion>0.2.61701</MicrosoftDiagnosticsNETCoreClientVersion>
<DNNEVersion>1.0.23</DNNEVersion>
<!--
These are used as reference assemblies only, so they must not take a ProdCon/source-build
version. Insert "RefOnly" to avoid assignment via PVP.
Expand All @@ -165,6 +167,7 @@
<MoqVersion>4.12.0</MoqVersion>
<FsCheckVersion>2.14.3</FsCheckVersion>
<SdkVersionForWorkloadTesting>6.0.100-rc.2.21425.12</SdkVersionForWorkloadTesting>
<CompilerPlatformTestingVersion>1.1.1-beta1.21467.5</CompilerPlatformTestingVersion>
<!-- Docs -->
<MicrosoftPrivateIntellisenseVersion>5.0.0-preview-20201009.2</MicrosoftPrivateIntellisenseVersion>
<!-- ILLink -->
Expand All @@ -190,7 +193,5 @@
<SwixPackageVersion>1.1.87-gba258badda</SwixPackageVersion>
<WixPackageVersion>3.14.0-dotnet</WixPackageVersion>
<MonoWorkloadManifestVersion>6.0.0-preview.5.21275.7</MonoWorkloadManifestVersion>
<!-- Experimental -->
<MicrosoftInteropDllImportGeneratorVersion>1.0.0-alpha.21464.1</MicrosoftInteropDllImportGeneratorVersion>
</PropertyGroup>
</Project>
86 changes: 59 additions & 27 deletions eng/generators.targets
Original file line number Diff line number Diff line change
@@ -1,45 +1,77 @@
<Project InitialTargets="ConfigureGenerators">
<Project>

<PropertyGroup>
<EnableDllImportGenerator Condition="'$(EnableDllImportGenerator)' == ''
and '$(MSBuildProjectName)' == 'System.Private.CoreLib'">true</EnableDllImportGenerator>
<SetDllImportGeneratorConstants>false</SetDllImportGeneratorConstants>
<IncludeDllImportGeneratorSources Condition="'$(IncludeDllImportGeneratorSources)' == ''">true</IncludeDllImportGeneratorSources>
</PropertyGroup>
<ItemGroup>
<EnabledGenerators Include="DllImportGenerator" Condition="'$(EnableDllImportGenerator)' == 'true'" />
<!-- If the current project is not System.Private.CoreLib, we enable the DllImportGenerator source generator
when the project is a C# source project that either:
- references System.Private.CoreLib, or
- references the following assemblies:
- System.Runtime.InteropServices
- System.Runtime.CompilerServices.Unsafe
- System.Memory -->
<EnabledGenerators Include="DllImportGenerator"
Condition="'$(EnableDllImportGenerator)' == ''
and '$(IsFrameworkSupportFacade)' != 'true'
and '$(IsNetCoreAppSrc)' == 'true'
and '$(MSBuildProjectExtension)' == '.csproj'
and (
('@(Reference)' != ''
and @(Reference->AnyHaveMetadataValue('Identity', 'System.Runtime.InteropServices'))
and @(Reference->AnyHaveMetadataValue('Identity', 'System.Runtime.CompilerServices.Unsafe'))
and @(Reference->AnyHaveMetadataValue('Identity', 'System.Memory')))
or ('@(ProjectReference)' != ''
and @(ProjectReference->AnyHaveMetadataValue('Identity', '$(CoreLibProject)'))))" />
</ItemGroup>

<!-- Use this complex ItemGroup-based filtering to add the ProjectReference to make sure dotnet/runtime stays compatible with NuGet Static Graph Restore. -->
<ItemGroup Condition="'@(EnabledGenerators)' != ''
and @(EnabledGenerators->AnyHaveMetadataValue('Identity', 'DllImportGenerator'))
and '$(IncludeDllImportGeneratorSources)' == 'true'">
<ProjectReference Include="$(LibrariesProjectRoot)System.Runtime.InteropServices/gen/DllImportGenerator/DllImportGenerator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
<Compile Include="$(LibrariesProjectRoot)Common/src/System/Runtime/InteropServices/GeneratedDllImportAttribute.cs" />
<Compile Include="$(LibrariesProjectRoot)Common/src/System/Runtime/InteropServices/GeneratedMarshallingAttribute.cs" />
<Compile Include="$(LibrariesProjectRoot)Common/src/System/Runtime/InteropServices/ArrayMarshaller.cs" />
</ItemGroup>

<Target Name="ConfigureGenerators"
DependsOnTargets="ConfigureDllImportGenerator" />
DependsOnTargets="ConfigureDllImportGenerator"
BeforeTargets="CoreCompile" />

<!-- Microsoft.Interop.DllImportGenerator -->
<Target Name="ConfigureDllImportGenerator" DependsOnTargets="EnableDllImportGeneratorForNetCoreApp">
<PropertyGroup Condition="'$(EnableDllImportGenerator)' == 'true'">
<Target Name="ConfigureDllImportGenerator"
Condition="'@(EnabledGenerators)' != '' and @(EnabledGenerators->AnyHaveMetadataValue('Identity', 'DllImportGenerator'))"
DependsOnTargets="ResolveReferences"
BeforeTargets="GenerateMSBuildEditorConfigFileShouldRun">
<PropertyGroup>
<DllImportGenerator_UseMarshalType>true</DllImportGenerator_UseMarshalType>
</PropertyGroup>
<!-- Projects that directly reference System.Private.CoreLib are typically low level enough that they don't reference
System.Runtime.CompilerServices.Unsafe, so we use the Unsafe type defined in CoreLib (Internal.Runtime.CompilerServices.Unsafe) for these projects. -->
<PropertyGroup Condition="'$(MSBuildProjectName)' == 'System.Private.CoreLib' or
('@(ProjectReference)' != '' and @(ProjectReference->AnyHaveMetadataValue('Identity', '$(CoreLibProject)')))">
<DllImportGenerator_UseInternalUnsafeType>true</DllImportGenerator_UseInternalUnsafeType>
<DefineConstants>$(DefineConstants);DLLIMPORTGENERATOR_INTERNALUNSAFE</DefineConstants>
</PropertyGroup>
<ItemGroup Condition="'$(EnableDllImportGenerator)' == 'true' and $([MSBuild]::IsTargetFrameworkCompatible('$(TargetFramework)', 'net6.0'))">
<PackageReference Include="Microsoft.Interop.DllImportGenerator" Version="$(MicrosoftInteropDllImportGeneratorVersion)" PrivateAssets="all" />

<!-- For testing purposes, compile sources files with attributes directly into the project.
This is mimicking the case where the source generator always generates the attributes. -->
<Compile Include="$(LibrariesProjectRoot)Common/src/System/Runtime/InteropServices/GeneratedDllImportAttribute.cs" />
<Compile Include="$(LibrariesProjectRoot)Common/src/System/Runtime/InteropServices/GeneratedMarshallingAttribute.cs" />
<Compile Include="$(LibrariesProjectRoot)Common/src/System/Runtime/InteropServices/ArrayMarshaller.cs" />
</ItemGroup>
</Target>
<MSBuild Projects="$(LibrariesProjectRoot)System.Runtime.InteropServices/gen/DllImportGenerator/DllImportGenerator.csproj">
<Output TaskParameter="TargetOutputs" PropertyName="DllImportGeneratorOutputPath" />
</MSBuild>

<Target Name="EnableDllImportGeneratorForNetCoreApp"
Condition="'$(EnableDllImportGenerator)' == ''
and '$(IsFrameworkSupportFacade)' != 'true'
and ('$(IsNetCoreAppSrc)' == 'true' or '$(MSBuildProjectName)' == 'System.Private.CoreLib')
and '$(MSBuildProjectExtension)' == '.csproj'">
<!-- We add the copy of Microsoft.Interop.SourceGeneration.dll that lives next to Microsoft.Interop.DllImportGenerator.dll
to work around https://github.com/dotnet/roslyn/issues/56442 -->
<ItemGroup>
<Analyzer Include="$([MSBuild]::NormalizePath('$([System.IO.Path]::GetDirectoryName('$(DllImportGeneratorOutputPath)'))', 'Microsoft.Interop.SourceGeneration.dll'))" />
</ItemGroup>
<PropertyGroup>
<!-- Enable DllImportGenerator by default for System.Private.CoreLib -->
<EnableDllImportGenerator Condition="'$(MSBuildProjectName)' == 'System.Private.CoreLib'">true</EnableDllImportGenerator>

<!-- Enable DllImportGenerator by default for NETCoreApp libraries that reference either System.Runtime.InteropServices or System.Private.CoreLib -->
<EnableDllImportGenerator Condition="'@(Reference)' != ''
and @(Reference->AnyHaveMetadataValue('Identity', 'System.Runtime.InteropServices'))
and @(Reference->AnyHaveMetadataValue('Identity', 'System.Runtime.CompilerServices.Unsafe'))
and @(Reference->AnyHaveMetadataValue('Identity', 'System.Memory'))">true</EnableDllImportGenerator>
<EnableDllImportGenerator Condition="'@(ProjectReference)' != '' and @(ProjectReference->AnyHaveMetadataValue('Identity', '$(CoreLibProject)'))">true</EnableDllImportGenerator>
<DefineConstants>$(DefineConstants);DLLIMPORTGENERATOR_ENABLED</DefineConstants>
</PropertyGroup>
</Target>

<Import Project="$(LibrariesProjectRoot)System.Runtime.InteropServices/gen/DllImportGenerator/Microsoft.Interop.DllImportGenerator.props" />
</Project>
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ internal static partial class Kernel32

#if DLLIMPORTGENERATOR_ENABLED
[GeneratedDllImport(Libraries.Kernel32, SetLastError = true)]
internal static partial bool SetConsoleCtrlHandler(delegate* unmanaged<int, BOOL> handler, bool addOrRemove);
internal static unsafe partial bool SetConsoleCtrlHandler(delegate* unmanaged<int, BOOL> handler, bool Add);
#else
[DllImport(Libraries.Kernel32, SetLastError = true)]
internal static extern unsafe bool SetConsoleCtrlHandler(delegate* unmanaged<int, BOOL> HandlerRoutine, bool Add);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,12 @@

namespace System.Runtime.InteropServices.GeneratedMarshalling
{
internal unsafe ref struct ArrayMarshaller<T>
#if DLLIMPORT_GENERATOR_TEST
public
#else
internal
#endif
unsafe ref struct ArrayMarshaller<T>
{
private T[]? _managedArray;
private readonly int _sizeOfNativeElement;
Expand Down Expand Up @@ -118,7 +123,12 @@ public void FreeNative()
}
}

internal unsafe ref struct PtrArrayMarshaller<T> where T : unmanaged
#if DLLIMPORT_GENERATOR_TEST
public
#else
internal
#endif
unsafe ref struct PtrArrayMarshaller<T> where T : unmanaged
{
private T*[]? _managedArray;
private readonly int _sizeOfNativeElement;
Expand Down Expand Up @@ -222,4 +232,4 @@ public void FreeNative()
}
}
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,12 @@ namespace System.Runtime.InteropServices
/// Indicates that method will be generated at compile time and invoke into an unmanaged library entry point
/// </summary>
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]
internal sealed class GeneratedDllImportAttribute : Attribute
#if DLLIMPORT_GENERATOR_TEST
public
#else
internal
#endif
sealed class GeneratedDllImportAttribute : Attribute
{
public bool BestFitMapping { get; set; }
public CallingConvention CallingConvention { get; set; }
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,32 @@
namespace System.Runtime.InteropServices
{
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Struct)]
internal class GeneratedMarshallingAttribute : Attribute
#if DLLIMPORT_GENERATOR_TEST
public
#else
internal
#endif
sealed class GeneratedMarshallingAttribute : Attribute
{
}

[AttributeUsage(AttributeTargets.Struct)]
internal class BlittableTypeAttribute : Attribute
#if DLLIMPORT_GENERATOR_TEST
public
#else
internal
#endif
sealed class BlittableTypeAttribute : Attribute
{
}

[AttributeUsage(AttributeTargets.Struct | AttributeTargets.Class)]
internal class NativeMarshallingAttribute : Attribute
#if DLLIMPORT_GENERATOR_TEST
public
#else
internal
#endif
sealed class NativeMarshallingAttribute : Attribute
{
public NativeMarshallingAttribute(Type nativeType)
{
Expand All @@ -28,14 +43,46 @@ public NativeMarshallingAttribute(Type nativeType)
public Type NativeType { get; }
}

[AttributeUsage(AttributeTargets.Parameter | AttributeTargets.ReturnValue | AttributeTargets.Field)]
internal class MarshalUsingAttribute : Attribute
[AttributeUsage(AttributeTargets.Parameter | AttributeTargets.ReturnValue | AttributeTargets.Field, AllowMultiple = true)]
#if DLLIMPORT_GENERATOR_TEST
public
#else
internal
#endif
sealed class MarshalUsingAttribute : Attribute
{
public MarshalUsingAttribute()
{
CountElementName = string.Empty;
}

public MarshalUsingAttribute(Type nativeType)
:this()
{
NativeType = nativeType;
}

public Type NativeType { get; }
public Type? NativeType { get; }

public string CountElementName { get; set; }

public int ConstantElementCount { get; set; }

public int ElementIndirectionLevel { get; set; }

public const string ReturnsCountValue = "return-value";
}

[AttributeUsage(AttributeTargets.Struct | AttributeTargets.Class)]
#if DLLIMPORT_GENERATOR_TEST
public
#else
internal
#endif
sealed class GenericContiguousCollectionMarshallerAttribute : Attribute
{
public GenericContiguousCollectionMarshallerAttribute()
{
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
<Project>
<Import Project="..\Directory.Build.props" />
<PropertyGroup>
<IsShipping>false</IsShipping>
<IsPackable>false</IsPackable>
<!-- We manually enable DllImportGenerator for projects in this folder as part of testing. -->
<EnableDllImportGenerator>false</EnableDllImportGenerator>
<EnableDefaultItems>true</EnableDefaultItems>
<CLSCompliant>false</CLSCompliant>
<ILLinkTrimAssembly>false</ILLinkTrimAssembly>
</PropertyGroup>
</Project>
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
namespace Microsoft.Interop
{
[Generator]
public class DllImportGenerator : IIncrementalGenerator
public sealed class DllImportGenerator : IIncrementalGenerator
{
private const string GeneratedDllImport = nameof(GeneratedDllImport);
private const string GeneratedDllImportAttribute = nameof(GeneratedDllImportAttribute);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.CodeAnalysis.CSharp.Workspaces" Version="$(CompilerPlatformVersion)" PrivateAssets="all" />
<PackageReference Include="Microsoft.CodeAnalysis.CSharp.Workspaces" Version="$(MicrosoftCodeAnalysisVersion)" PrivateAssets="all" />
<PackageReference Include="Microsoft.CodeAnalysis.Analyzers" Version="$(MicrosoftCodeAnalysisAnalyzersVersion)" PrivateAssets="all" />
</ItemGroup>

Expand All @@ -40,7 +40,7 @@
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\Microsoft.Interop.SourceGeneration\Microsoft.Interop.SourceGeneration.csproj" />
<ProjectReference Include="..\Microsoft.Interop.SourceGeneration\Microsoft.Interop.SourceGeneration.csproj" Private="true" />
</ItemGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
-->
<CompilerVisibleProperty Include="DllImportGenerator_UseInternalUnsafeType" />
</ItemGroup>
<PropertyGroup>
<PropertyGroup Condition="'$(SetDllImportGeneratorConstants)' != 'false'">
<DefineConstants>$(DefineConstants);DLLIMPORTGENERATOR_ENABLED</DefineConstants>
</PropertyGroup>
</Project>
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.CodeAnalysis" Version="$(CompilerPlatformVersion)" PrivateAssets="all" />
<PackageReference Include="Microsoft.CodeAnalysis" Version="$(MicrosoftCodeAnalysisVersion)" PrivateAssets="all" />
<PackageReference Include="Microsoft.CodeAnalysis.Analyzers" Version="$(MicrosoftCodeAnalysisAnalyzersVersion)" PrivateAssets="all" />
</ItemGroup>

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,17 @@

<PropertyGroup>
<AssemblyName>Microsoft.Interop.Ancillary</AssemblyName>
<TargetFramework>net6.0</TargetFramework>
<TargetFramework>$(NetCoreAppCurrent)</TargetFramework>
<RootNamespace>System.Runtime.InteropServices</RootNamespace>
<Nullable>enable</Nullable>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<Description>APIs required for usage of the DllImportGenerator and related tools.</Description>
<DefineConstants>$(DefineConstants);DLLIMPORT_GENERATOR_TEST</DefineConstants>
</PropertyGroup>

<ItemGroup>
<Compile Include="$(LibrariesProjectRoot)Common/src/System/Runtime/InteropServices/GeneratedDllImportAttribute.cs" Link="GeneratedDllImportAttribute.cs" />
<Compile Include="$(LibrariesProjectRoot)Common/src/System/Runtime/InteropServices/GeneratedMarshallingAttribute.cs" Link="GeneratedMarshallingAttribute.cs" />
<Compile Include="$(LibrariesProjectRoot)Common/src/System/Runtime/InteropServices/ArrayMarshaller.cs" Link="ArrayMarshaller.cs" />
</ItemGroup>
</Project>
Loading

0 comments on commit 4cfb51f

Please sign in to comment.