Skip to content

Commit

Permalink
Add a Local Test binary, to enable local TerminalApp testing (#2294)
Browse files Browse the repository at this point in the history
In #1164 we learned that our CI doesn't support WinRT testing. This made us all sad. Since that merged, we haven't really added any TerminalApp tests, because it's a little too hard. You'd have to uncomment the entire file, and if the list of types changed you'd have to manually update the sxs manifest and appxmanifest.

Since that was all insane, I created a new Terminal App unittesting project without those problems.
1. The project is not named *Unit*Test*, so the CI won't run it, but it will run locally.
2. The project will auto-generate its SxS manifest, using the work from #1987. 
3. We'll use the SxS manifest from step 2 to generate an AppxManifest for running packaged tests.


* This is the start of me trying to enable local unittesting again

  * We've got a new unittests project that isn't named *unit*test*

  * We're manually generating the SxS manifest for it. B/C we need to use it at runtime, we need to manually combine it into one manifest file

  * the runas:UAP thing still doesn't work. We'll investigate.

* This shockingly works

but I'm still stuck with:
```
Summary of Errors Outside of Tests:
Error: TAEF: [HRESULT: 0x80270254] Failed to create the test host process for
out of process test execution. (The
IApplicationActivationManager::ActivateApplication call failed while using a
default host. TAEF's ETW logs which are gathered with the /enableEtwLogging
switch should contain events from relevant providers that may help to diagnose
the failure.)
```

* Cleaning this all up for review.

  Frankly just pushing to see if it'll work in CI

* Couple things I noticed in the diff from master

* Apply @DHowett-MSFT's suggestions from code review
  • Loading branch information
zadjii-msft authored Aug 13, 2019
1 parent 138d3b8 commit ac97e5d
Show file tree
Hide file tree
Showing 16 changed files with 568 additions and 177 deletions.
24 changes: 24 additions & 0 deletions OpenConsole.sln
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,11 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "TerminalAppLib", "src\casca
{CA5CAD1A-D7EC-4107-B7C6-79CB77AE2907} = {CA5CAD1A-D7EC-4107-B7C6-79CB77AE2907}
EndProjectSection
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "LocalTests_TerminalApp", "src\cascadia\LocalTests_TerminalApp\TerminalApp.LocalTests.vcxproj", "{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}"
ProjectSection(ProjectDependencies) = postProject
{CA5CAD1A-9A12-429C-B551-8562EC954746} = {CA5CAD1A-9A12-429C-B551-8562EC954746}
EndProjectSection
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
AuditMode|ARM64 = AuditMode|ARM64
Expand Down Expand Up @@ -1087,6 +1092,24 @@ Global
{CA5CAD1A-9A12-429C-B551-8562EC954746}.Release|x64.Build.0 = Release|x64
{CA5CAD1A-9A12-429C-B551-8562EC954746}.Release|x86.ActiveCfg = Release|Win32
{CA5CAD1A-9A12-429C-B551-8562EC954746}.Release|x86.Build.0 = Release|Win32
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.AuditMode|ARM64.ActiveCfg = AuditMode|ARM64
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.AuditMode|ARM64.Build.0 = AuditMode|ARM64
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.AuditMode|x64.ActiveCfg = AuditMode|x64
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.AuditMode|x64.Build.0 = AuditMode|x64
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.AuditMode|x86.ActiveCfg = AuditMode|Win32
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.AuditMode|x86.Build.0 = AuditMode|Win32
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.Debug|ARM64.ActiveCfg = Debug|ARM64
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.Debug|ARM64.Build.0 = Debug|ARM64
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.Debug|x64.ActiveCfg = Debug|x64
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.Debug|x64.Build.0 = Debug|x64
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.Debug|x86.ActiveCfg = Debug|Win32
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.Debug|x86.Build.0 = Debug|Win32
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.Release|ARM64.ActiveCfg = Release|ARM64
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.Release|ARM64.Build.0 = Release|ARM64
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.Release|x64.ActiveCfg = Release|x64
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.Release|x64.Build.0 = Release|x64
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.Release|x86.ActiveCfg = Release|Win32
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506}.Release|x86.Build.0 = Release|Win32
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand Down Expand Up @@ -1147,6 +1170,7 @@ Global
{34DE34D3-1CD6-4EE3-8BD9-A26B5B27EC73} = {89CDCC5C-9F53-4054-97A4-639D99F169CD}
{CA5CAD1A-9333-4D05-B12A-1905CBF112F9} = {59840756-302F-44DF-AA47-441A9D673202}
{CA5CAD1A-9A12-429C-B551-8562EC954746} = {59840756-302F-44DF-AA47-441A9D673202}
{CA5CAD1A-B11C-4DDB-A4FE-C3AFAE9B5506} = {59840756-302F-44DF-AA47-441A9D673202}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {3140B1B7-C8EE-43D1-A772-D82A7061A271}
Expand Down
44 changes: 44 additions & 0 deletions src/cascadia/LocalTests_TerminalApp/SettingsTests.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.

#include "precomp.h"

#include "../TerminalApp/ColorScheme.h"

using namespace Microsoft::Console;
using namespace TerminalApp;
using namespace WEX::Logging;
using namespace WEX::TestExecution;

namespace TerminalAppLocalTests
{
// Unfortunately, these tests _WILL NOT_ work in our CI, until we have a lab
// machine available that can run Windows version 18362.

class SettingsTests
{
// Use a custom manifest to ensure that we can activate winrt types from
// our test. This property will tell taef to manually use this as the
// sxs manifest during this test class. It includes all the cppwinrt
// types we've defined, so if your test is crashing for an unknown
// reason, make sure it's included in that file.
// If you want to do anything XAML-y, you'll need to run yor test in a
// packaged context. See TabTests.cpp for more details on that.
BEGIN_TEST_CLASS(SettingsTests)
TEST_CLASS_PROPERTY(L"ActivationContext", L"TerminalApp.LocalTests.manifest")
END_TEST_CLASS()

TEST_METHOD(TryCreateWinRTType);
};

void SettingsTests::TryCreateWinRTType()
{
winrt::Microsoft::Terminal::Settings::TerminalSettings settings{};
VERIFY_IS_NOT_NULL(settings);
auto oldFontSize = settings.FontSize();
settings.FontSize(oldFontSize + 5);
auto newFontSize = settings.FontSize();
VERIFY_ARE_NOT_EQUAL(oldFontSize, newFontSize);
}

}
110 changes: 110 additions & 0 deletions src/cascadia/LocalTests_TerminalApp/TabTests.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT license.

#include "precomp.h"

#include "../TerminalApp/ColorScheme.h"
#include "../TerminalApp/Tab.h"

using namespace Microsoft::Console;
using namespace TerminalApp;
using namespace WEX::Logging;
using namespace WEX::TestExecution;

namespace TerminalAppLocalTests
{
// Unfortunately, these tests _WILL NOT_ work in our CI, until we have a lab
// machine available that can run Windows version 18362.

class TabTests
{
// For this set of tests, we need to activate some XAML content. To do
// that, we need to be able to activate Xaml Islands(XI), using the Xaml
// Hosting APIs. Because XI looks at the manifest of the exe running, we
// can't just use the TerminalApp.Unit.Tests.manifest as our
// ActivationContext. XI is going to inspect `te.exe`s manifest to try
// and find the maxversiontested property, but te.exe hasn't set that.
// Instead, this test will run as a UAP application, as a packaged
// centenial (win32) app. We'll specify our own AppxManifest, so that
// we'll be able to also load all the dll's for the types we've defined
// (and want to use here). This does come with a minor caveat, as
// deploying the appx takes a bit, so use sparingly (though it will
// deploy once per class when used like this.)
BEGIN_TEST_CLASS(TabTests)
TEST_CLASS_PROPERTY(L"RunAs", L"UAP")
TEST_CLASS_PROPERTY(L"UAP:AppXManifest", L"TerminalApp.LocalTests.AppxManifest.xml")
END_TEST_CLASS()

// These four tests act as canary tests. If one of them fails, then they
// can help you identify if something much lower in the stack has
// failed.
TEST_METHOD(TryInitXamlIslands);
TEST_METHOD(TryCreateLocalWinRTType);
TEST_METHOD(TryCreateXamlObjects);
TEST_METHOD(TryCreateTab);

TEST_CLASS_SETUP(ClassSetup)
{
winrt::init_apartment(winrt::apartment_type::single_threaded);
// Initialize the Xaml Hosting Manager
_manager = winrt::Windows::UI::Xaml::Hosting::WindowsXamlManager::InitializeForCurrentThread();
_source = winrt::Windows::UI::Xaml::Hosting::DesktopWindowXamlSource{};

return true;
}

private:
winrt::Windows::UI::Xaml::Hosting::WindowsXamlManager _manager{ nullptr };
winrt::Windows::UI::Xaml::Hosting::DesktopWindowXamlSource _source{ nullptr };
};

void TabTests::TryInitXamlIslands()
{
// Ensures that XAML Islands was initialized correctly
VERIFY_IS_NOT_NULL(_manager);
VERIFY_IS_NOT_NULL(_source);
}

void TabTests::TryCreateLocalWinRTType()
{
// Verify we can create a WinRT type we authored
// Just creating it is enough to know that everything is working.
winrt::Microsoft::Terminal::Settings::TerminalSettings settings{};
VERIFY_IS_NOT_NULL(settings);
auto oldFontSize = settings.FontSize();
settings.FontSize(oldFontSize + 5);
auto newFontSize = settings.FontSize();
VERIFY_ARE_NOT_EQUAL(oldFontSize, newFontSize);
}

void TabTests::TryCreateXamlObjects()
{
// Verify we can create a some XAML objects
// Just creating all of them is enough to know that everything is working.
winrt::Windows::UI::Xaml::Controls::UserControl controlRoot;
VERIFY_IS_NOT_NULL(controlRoot);
winrt::Windows::UI::Xaml::Controls::Grid root;
VERIFY_IS_NOT_NULL(root);
winrt::Windows::UI::Xaml::Controls::SwapChainPanel swapChainPanel;
VERIFY_IS_NOT_NULL(swapChainPanel);
winrt::Windows::UI::Xaml::Controls::Primitives::ScrollBar scrollBar;
VERIFY_IS_NOT_NULL(scrollBar);
}

void TabTests::TryCreateTab()
{
// Just try creating all of:
// 1. one of our pure c++ types (Profile)
// 2. one of our c++winrt types (TermControl)
// 3. one of our types that uses MUX/Xaml (Tab).
// Just creating all of them is enough to know that everything is working.
const auto profileGuid{ Utils::CreateGuid() };
winrt::Microsoft::Terminal::TerminalControl::TermControl term{};
VERIFY_IS_NOT_NULL(term);

auto newTab = std::make_shared<Tab>(profileGuid, term);

VERIFY_IS_NOT_NULL(newTab);
}

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
<?xml version="1.0" encoding="utf-8"?>
<Package xmlns:rescap="http://schemas.microsoft.com/appx/manifest/foundation/windows10/restrictedcapabilities" xmlns="http://schemas.microsoft.com/appx/manifest/foundation/windows10" xmlns:uap="http://schemas.microsoft.com/appx/manifest/uap/windows10" IgnorableNamespaces="uap">

<!-- This file is used as the Appxmanifest for tests that _need_ to run in a
packaged environment. It will be copied to the test's OutDir as part of the
PostBuid step. It's highly similar to the "PackagedCwaFullTrust" manifest that
TAEF ships with, with the following modifications:
1. All of our winrt types are included in this manifest, including types from
MUX.dll. Should this list of types ever change, we'll need to manually
update this file. The easiest way of doing that is deploying the app from
VS, then copying the Extensions from the Appxmanifest.xml that's generated
under `src/cascadia/CascadiaPackage/bin/$(platform)/$(configuration)/appx`.
2. We also _NEED_ the two vclibs listed under the `PackageDependency` block.
If your test fails for whatever reason, it's likely possible you're testing a
type that's _not_ included in this file for some reason. So, here be dragons. -->

<Identity Name="TerminalApp.LocalTests.Package"
ProcessorArchitecture="neutral"
Publisher="CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington, C=US"
Version="1.0.0.0"
ResourceId="en-us" />
<Properties>
<DisplayName>TerminalApp.LocalTests.Package Host Process</DisplayName>
<PublisherDisplayName>Microsoft Corp.</PublisherDisplayName>
<Logo>taef.png</Logo>
<Description>TAEF Packaged Cwa FullTrust Application Host Process</Description>
</Properties>
<Dependencies>
<TargetDeviceFamily Name="Windows.Universal" MinVersion="10.0.18362.0" MaxVersionTested="10.0.18362.0" />
<PackageDependency Name="Microsoft.VCLibs.140.00.Debug" MinVersion="14.0.27023.1" Publisher="CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington, C=US" />
<PackageDependency Name="Microsoft.VCLibs.140.00.Debug.UWPDesktop" MinVersion="14.0.27027.1" Publisher="CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington, C=US" />
</Dependencies>
<Resources>
<Resource Language="en-us" />
</Resources>
<Applications>
<Application Id="TE.ProcessHost" Executable="TE.ProcessHost.exe" EntryPoint="Windows.FullTrustApplication">
<uap:VisualElements DisplayName="TAEF Packaged Cwa FullTrust Application Host Process" Square150x150Logo="taef.png" Square44x44Logo="taef.png" Description="TAEF Packaged Cwa Application Host Process" BackgroundColor="#222222">
<uap:SplashScreen Image="taef.png" />
</uap:VisualElements>
</Application>
</Applications>
<Capabilities>
<rescap:Capability Name="runFullTrust"/>
</Capabilities>



</Package>
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">

<!-- This file is copied into ut_app/TerminalApp.Unit.Tests.manifest as part
of the pre-build step for that project. Changes should only be made to the
WindowsTerminal version of the file. -->

<compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1">
<application>
<!-- Windows 10 1903 -->
<!-- See https://docs.microsoft.com/en-us/windows/apps/desktop/modernize/xaml-islands -->
<maxversiontested Id="10.0.18362.0"/>
<supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}" />
</application>
</compatibility>

<application xmlns="urn:schemas-microsoft-com:asm.v3">
<windowsSettings>
<dpiAwareness xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">PerMonitorV2</dpiAwareness>
<longPathAware xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">true</longPathAware>
</windowsSettings>
</application>
</assembly>
Loading

0 comments on commit ac97e5d

Please sign in to comment.