Skip to content

Commit

Permalink
[build] Build Xamarin.Android.NUnitLite.dll for API-28 (#2182)
Browse files Browse the repository at this point in the history
We've been dealing with a head-scratching annoyance: `.apk` test would
execute, pass, and have their `TestResults.xml` files extractable when
the PR builds run the `.apk` tests, but *not* when `master` does.

This makes it difficult to know whether or not we've broken anything.

One "interesting" observation we've had is that the paths *differ* on
the "downloadable" vs. "undownloadable" builds.  For example,
[master build #1166][0] uses an internal storage path:

	RunTestApks
	  ...
	  Executing: /Users/builder/android-toolchain/sdk/platform-tools/adb   shell run-as Mono.Android_Tests cat "/data/user/0/Mono.Android_Tests/files/.__override__/TestResults.xml"
	    writing stdout to file: /Users/builder/jenkins/workspace/xamarin-android/xamarin-android/tests/../bin/TestRelease/TestResult-Mono.Android_Tests.xml
	…/xamarin-android/build-tools/scripts/TestApks.targets(188,5): error :   Command /Users/builder/android-toolchain/sdk/platform-tools/adb   shell run-as Mono.Android_Tests cat "/data/user/0/Mono.Android_Tests/files/.__override__/TestResults.xml" failed with exit code 1

While the "same" command on [PR build #4003][1] uses an external path:

	RunTestApks:
	  ...
	  Executing: /Users/builder/android-toolchain/sdk/platform-tools/adb -s emulator-5570  shell run-as Mono.Android_Tests cat "/storage/emulated/0/Android/data/Mono.Android_Tests/files/Documents/TestResults.xml"
	    writing stdout to file: /Users/builder/jenkins/workspace/xamarin-android-pr-builder/xamarin-android/tests/../bin/TestDebug/TestResult-Mono.Android_Tests.xml

*Why* do these paths differ?

The paths differ -- internal storage on master, external on PR builds
-- because `Xamarin.Android.NUnitLite.dll` only uses
[external storage directories][2] when it's built against a
`$(TargetFrameworkVersion)` value >= v4.4.

This is true on PR builds, where we only build one API level,
`$(AndroidFrameworkVersion)`, currently v9.0 (API-28).

This is *not* true on master builds where we build for *all* API
levels, and "really" build `Xamarin.Android.NUnitLite.dll` for the
[*first* supported API level][3], v2.3 (API-10)!

Update the `Xamarin.Android.NUnitLite.dll` build within
`make framework-assemblies` so that instead of building for the
*first* API level, we build for the *last* API level.

This should allow our unit tests to work sanely on "full" builds.

Additionally, update/"partially revert" the
`<RunInstrumentationTests/>` task so that it *always* performs
`adb pull` to extract `TestResults.xml`, instead of using
`adb shell run-as ... cat ...`, as introduced in 2e8f96f.
The problem with `run-as ... cat` is that it doesn't work with
non-debuggable packages, e.g. if you have a
`Mono.Android_Tests-Signed.apk` built in the `Release` configuration,
`run-as` will *fail*:

	$ adb shell run-as Mono.Android_Tests cat /storage/emulated/0/Android/data/Mono.Android_Tests/files/Documents/TestResults.xml
	run-as: package not debuggable: Mono.Android_Tests

Instead of special-casing `<RunInstrumentationTests/>` to try using
`run-as` in some circumstances and `pull` in others, "revert" to
pre-2e8f96f5 behavior and always use `adb pull`.

[0]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android/1166/
[1]: https://jenkins.mono-project.com/view/Xamarin.Android/job/xamarin-android-pr-builder/4003/
[2]: https://github.com/xamarin/xamarin-android/blob/e153d42e51d36170487b3081c55310d978685b06/src/Xamarin.Android.NUnitLite/Gui/Instrumentations/TestSuiteInstrumentation.cs#L107-L110
[3]: https://github.com/xamarin/xamarin-android/blob/e153d42e51d36170487b3081c55310d978685b06/build-tools/scripts/BuildEverything.mk#L113-L118
  • Loading branch information
jonpryor authored and grendello committed Sep 14, 2018
1 parent 9081026 commit 24158d0
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -79,10 +79,7 @@ public override bool Execute ()
},

new CommandInfo {
ArgumentsGenerator = () => $"{AdbTarget} {AdbOptions} shell run-as {PackageName} cat \"{targetTestResultsPath}\"",
MergeStdoutAndStderr = false,
StdoutFilePath = NUnit2TestResultsFile,
SuppressMSbuildLog = true,
ArgumentsGenerator = () => $"{AdbTarget} {AdbOptions} pull \"{targetTestResultsPath}\" \"{NUnit2TestResultsFile}\"",
ShouldRun = () => !String.IsNullOrEmpty (targetTestResultsPath)
},
};
Expand Down
4 changes: 2 additions & 2 deletions build-tools/scripts/BuildEverything.mk
Original file line number Diff line number Diff line change
Expand Up @@ -114,8 +114,8 @@ framework-assemblies:
rm -f bin/$(conf)/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/Xamarin.Android.NUnitLite.dll; \
$(call MSBUILD_BINLOG,NUnitLite,$(_SLN_BUILD),$(conf)) $(MSBUILD_FLAGS) src/Xamarin.Android.NUnitLite/Xamarin.Android.NUnitLite.csproj \
/p:Configuration=$(conf) $(_MSBUILD_ARGS) \
/p:AndroidApiLevel=$(firstword $(API_LEVELS)) /p:AndroidPlatformId=$(word $(firstword $(API_LEVELS)), $(ALL_PLATFORM_IDS)) \
/p:AndroidFrameworkVersion=$(firstword $(FRAMEWORKS)) || exit 1; )
/p:AndroidApiLevel=$(lastword $(API_LEVELS)) /p:AndroidPlatformId=$(word $(lastword $(API_LEVELS)), $(ALL_PLATFORM_IDS)) \
/p:AndroidFrameworkVersion=$(lastword $(FRAMEWORKS)) || exit 1; )
_latest_stable_framework=$$($(MSBUILD) /p:DoNotLoadOSProperties=True /nologo /v:minimal /t:GetAndroidLatestStableFrameworkVersion build-tools/scripts/Info.targets | tr -d '[[:space:]]') ; \
$(foreach conf, $(CONFIGURATIONS), \
rm -f "bin/$(conf)/lib/xamarin.android/xbuild-frameworks/MonoAndroid/$$_latest_stable_framework"/Mono.Android.Export.* ; \
Expand Down

0 comments on commit 24158d0

Please sign in to comment.