Feat: extract reflection behind a provider #3801

nohwnd · 2024-09-10T11:10:24Z

Moves reflection calls so we can plugin a sepearate IReflectionOperations and IFileOperations provider implementations that tie the existing engine to a source generated map of data. Allowing the current engine to be used together with native aot.

Some parts are disabled when running in native, such as:

Deployment
Reflection only load
DynamicDataOperations is introduced to encapsulate the logic that was hardcoded in TestFramework project in attribute.
IReflectionOperations2 internal interface is introduced to encapsulate the remaining reflection functionality that was not isolated by the public IReflectionOperations interface.

Public types in Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration were added so we can interface with a source generator. These types need to be public, but are not meant for public consumption.

…or callback

nohwnd · 2024-09-10T11:12:35Z

eng/Versions.props

@@ -1,9 +1,9 @@
 <Project>
  <PropertyGroup Label="Version settings">
    <!-- MSTest version -->
-    <VersionPrefix>3.7.0</VersionPrefix>
+    <VersionPrefix>3.8.0</VersionPrefix>


this will go away before merge, but I need it for local development, and I expect some changes will be needed.

nohwnd · 2024-09-11T12:45:48Z

src/Adapter/MSTest.TestAdapter/MSTest.TestAdapter.csproj

@@ -143,4 +147,12 @@
  <!-- SDK bottom import -->
  <Import Project="Sdk.targets" Sdk="MSBuild.Sdk.Extras" Condition=" '$(OS)' == 'Windows_NT' " />
  <Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" Condition=" '$(OS)' != 'Windows_NT' " />
+  <ItemGroup>


dunno what this is about, this project is somehow busted, every time I add files the get removed from compilation. Removing this.

It's because of the line 109. @MarcoRossignoli could we workaround that and still use the MSTestVersion.cs?

if we want to keep the version without reflection we need to insert a static source or a "resource"? but I don't know if at runtime we can get it also in aot mode.

That file is not added to the build if not with manual *.cs at least it wasn't working without custom source addition.

nohwnd · 2024-09-11T12:47:22Z

src/Adapter/MSTest.TestAdapter/SourceGeneratedDynamicDataOperations.cs

+
+namespace Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter;
+
+internal class SourceGeneratedDynamicDataOperations : DynamicDataOperations


This will be needed later for data driven tests probably, having extra class allows me to have all the initializations wired correctly already.

src/Adapter/MSTest.TestAdapter/VSTestAdapter/MSTestExecutor.cs

Evangelink

Can you please try to reduce new public API to the minimal? Can you also seal as much as possible?

I'll need a second review and a few manual tests

src/Adapter/MSTest.TestAdapter/DynamicDataOperations.cs

Evangelink · 2024-09-11T12:58:54Z

src/Adapter/MSTest.TestAdapter/MSTest.TestAdapter.csproj

@@ -143,4 +147,12 @@
  <!-- SDK bottom import -->
  <Import Project="Sdk.targets" Sdk="MSBuild.Sdk.Extras" Condition=" '$(OS)' == 'Windows_NT' " />
  <Import Project="Sdk.targets" Sdk="Microsoft.NET.Sdk" Condition=" '$(OS)' != 'Windows_NT' " />
+  <ItemGroup>


It's because of the line 109. @MarcoRossignoli could we workaround that and still use the MSTestVersion.cs?

src/Adapter/MSTest.TestAdapter/PlatformServiceProvider.cs

Evangelink · 2024-09-11T13:01:47Z

src/Adapter/MSTest.TestAdapter/PublicAPI/PublicAPI.Unshipped.txt

+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.ReflectionMetadataHook
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.Assembly.get -> System.Reflection.Assembly!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.Assembly.init -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.AssemblyAttributes.get -> object![]!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.AssemblyAttributes.init -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.AssemblyName.get -> string!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.AssemblyName.init -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.MyConstructorInfo
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.MyConstructorInfo.Invoker.get -> System.Func<object?[]!, object!>!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.MyConstructorInfo.MyConstructorInfo() -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.MyConstructorInfo.Parameters.get -> System.Type![]!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.SourceGeneratedReflectionDataProvider() -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeAttributes.get -> System.Collections.Generic.Dictionary<System.Type!, System.Attribute![]!>!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeAttributes.init -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeConstructors.get -> System.Collections.Generic.Dictionary<System.Type!, System.Reflection.ConstructorInfo![]!>!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeConstructors.init -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeConstructorsInvoker.get -> System.Collections.Generic.Dictionary<System.Type!, Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.MyConstructorInfo![]!>!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeConstructorsInvoker.init -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeLocation
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeLocation.FileName.get -> string!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeLocation.FileName.init -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeLocation.MethodLocations.get -> System.Collections.Generic.Dictionary<string!, int>!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeLocation.MethodLocations.init -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeLocation.TypeLocation() -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeMethodAttributes.get -> System.Collections.Generic.Dictionary<System.Type!, System.Collections.Generic.Dictionary<string!, System.Attribute![]!>!>!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeMethodAttributes.init -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeMethodLocations.get -> System.Collections.Generic.Dictionary<string!, Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeLocation!>!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeMethodLocations.init -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeMethods.get -> System.Collections.Generic.Dictionary<System.Type!, System.Reflection.MethodInfo![]!>!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeMethods.init -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeProperties.get -> System.Collections.Generic.Dictionary<System.Type!, System.Reflection.PropertyInfo![]!>!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypeProperties.init -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypePropertiesByName.get -> System.Collections.Generic.Dictionary<System.Type!, System.Collections.Generic.Dictionary<string!, System.Reflection.PropertyInfo!>!>!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypePropertiesByName.init -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.Types.get -> System.Type![]!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.Types.init -> void
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypesByName.get -> System.Collections.Generic.Dictionary<string!, System.Type!>!
+Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider.TypesByName.init -> void
+static Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.ReflectionMetadataHook.SetMetadata(Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.SourceGeneration.SourceGeneratedReflectionDataProvider! metadata) -> void


Could we avoid to have this public?

I don't think we can, the types need to be accessible to the source generated assembly, and we don't know what that will be so we cannot give it IVTs.

src/TestFramework/TestFramework/TestFramework.csproj

test/UnitTests/MSTestAdapter.UnitTests/DynamicDataAttributeTests.cs

Co-authored-by: Amaury Levé <[email protected]>

src/Adapter/MSTest.TestAdapter/DynamicDataOperations.cs

Evangelink · 2024-09-15T15:48:50Z

@nohwnd Can you please revert the verison changes so we merge this PR?!

eng/Versions.props

nohwnd · 2024-09-18T16:00:25Z

reverted

nohwnd added 16 commits June 13, 2024 09:47

Abstrace some parts of reflection

6fc0aa1

some metadata through provider, but construction is still done via ct…

99cb77e

…or callback

Allow running in native with normal adapter

e7ad398

allow-nativeaot-in-engine

d581268

move so the hook works

b983662

Merge branch 'main' into extract-reflection

28d61c7

merges

3a2e7f5

Merge branch 'main' into extract-reflection

cff718e

Merge branch 'main' into extract-reflection

e740ac4

Merge branch 'main' into extract-reflection

62cd127

main

15d568a

Merge branch 'main' into extract-reflection

4ef31e0

Fixes

eb091a6

fix

1816090

Fix various warnings

dc2c58c

Merge branch 'fix-warnings' into extract-reflection

28c32cf

nohwnd commented Sep 10, 2024

View reviewed changes

nohwnd added 7 commits September 10, 2024 13:17

Merge branch 'main' into extract-reflection

a7382b4

Replace env variable with a static class

4bc35f9

one test fixed....

d8c5ded

Fix tests

8c1c18f

Merge branch 'main' into extract-reflection

f6749ed

fix whitespace

afd042a

Fix getting memebers for data driven tests

3e731bb

nohwnd marked this pull request as ready for review September 11, 2024 12:41