Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge release/6.0-maui branch back into release/6.0 #64687

Merged
merged 34 commits into from
Feb 8, 2022

Conversation

akoeplinger
Copy link
Member

No description provided.

directhex and others added 28 commits November 24, 2021 18:32
Updated branch used for maui previews and get it building
* [interp] Improve logging on mobile devices

Use a single g_print in bulk for every instruction. Otherwise, the instruction ends up being displayed on multiple lines.

* [interp] Remove hack for nint/nfloat

These structures are valuetypes, but their mint_type is a primitive. This means that a LDFLD applied on the type would have attempted to do a pointer dereference, because it saw that the current top of stack is not STACK_TYPE_VT. This was fixed in the past by passing the managed pointer to the valuetype rather than the valuetype itself, against the normal call convention, which lead to inconsistencies in the code.

This commit removes that hack and fixes the problem by ignoring LDFLD applied to nint/nfloat valuetypes in the first place.
The special version part cannot exceed 20 characters. error from NuGet. This name is the same length as known-good preview
#62235)

* Exclude the managed code around libproc on iOS/tvOS (#61590)

Since libproc is a private Apple API, it is not available on iOS/tvOS and should be excluded (see #61265 (comment) and above for more details).  
This PR excludes $(CommonPath)Interop\OSX\Interop.libproc.cs on the iOS/tvOS as well as makes some methods in Process, ProcessManager, and ProcessThread classes calling that API throw PNSE so that for iOS/tvOS it's possible to re-use the respective *.UnknownUnix.cs parts.

* [iOS] Follow up changes for 61590 (#61670)

This is a follow up PR for #61590.

It includes:

- additional UnsupportedOSPlatform annotations for some System.Diagnostics.Process APIs throwing PNSE on iOS/tvOS (they started doing so after excluding some managed logic around librpoc )

- fixing a bit ugly workaround for CS0649 (see https://github.com/dotnet/runtime/pull/61590/files#r749525127) - used a local pragma in the ThreadInfo class.

- skipping the respective S.D.P. tests ( it will address [iOS/tvOS] System.Diagnostics.Tests.ProcessTests.TestGetProcesses fails on devices #60588 as well)

* Skip System.Diagnostics.TextWriterTraceListenerTests.XmlWriterTraceListenerTests on iOS/tvOS (#61807)

This marks System.Diagnostics.TextWriterTraceListenerTests.XmlWriterTraceListenerTests withSkipOnPlatform attribute for iOS/tvOS as those tests try to create a process info, which throws PNSE after S.D.Process API's around libproc have been excluded in #61590.

* Disable several failing tests on iOSSimulator arm64 #61826

A few tests popped up as failures on the rolling build due to parts of System.Diagnostics.Process throwing PNSE. Disabled the functional tests from running on arm64 as mlaunch can't detect the return code.

* Use separate partials for iOS&tvOS instead of UnknowUnix in System.Diagnostics.Process (#61871)

* Remove NoWarn removal
Fixes: #56163

PR #58523 fixed something on Windows, but it didn't actually address
our issues on Android.

A directory name like `foo Ümläüts` fails:

* `mkdir 'foo Ümläüts' ; cd 'foo Ümläüts'`
* `dotnet new android`
* `dotnet build -c Release -p:RunAOTCompilation=true` (adding
  `-p:EnableLLVM=true` complicates further)

The error:

    Precompiling failed for C:\src\foo Ümläüts\obj\Release\android-arm64\linked\System.Private.CoreLib.dll: Error: Loaded assembly 'C:\src\foo ├£ml├ñ├╝ts\obj\Release\android-arm64\linked\System.Private.CoreLib.dll' doesn't match original file name 'C:\foo ▄mlΣⁿts\obj\Release\android-arm64\linked\System.Private.CoreLib.dll'. Set MONO_PATH to the assembly's location.

Reviewing the existing AOT implementation in Xamarin.Android, I found
out *why* Xamarin.Android works:

    [AOT] response file obj\Release\120\aot\arm64-v8a\App36.dll\response.txt: --llvm "--aot=temp-path=obj\Release\120\aot\arm64-v8a\App36.dll,llvm-path=C:\Program Files\Microsoft Visual Studio\2022\Preview\MSBuild\Xamarin\Android,outfile=obj\Release\120\aot\arm64-v8a\libaot-App36.dll.so,msym-dir=obj\Release\120\aot\arm64-v8a,asmwriter,mtriple=aarch64-linux-android,tool-prefix=C:\Program Files (x86)\Android\android-sdk\ndk-bundle\toolchains\llvm\prebuilt\windows-x86_64\bin\aarch64-linux-android-,ld-name=ld.EXE,ld-flags=\"-LC:\Program Files (x86)\Android\android-sdk\ndk-bundle\toolchains\llvm\prebuilt\windows-x86_64\lib\gcc\aarch64-linux-android\4.9.x\";\"-LC:\Program Files (x86)\Android\android-sdk\ndk-bundle\platforms\android-21\arch-arm64\usr\lib\";\"C:\Program Files (x86)\Android\android-sdk\ndk-bundle\toolchains\llvm\prebuilt\windows-x86_64\lib\gcc\aarch64-linux-android\4.9.x\libgcc.a\";\"C:\Program Files (x86)\Android\android-sdk\ndk-bundle\platforms\android-21\arch-arm64\usr\lib\libc.so\";\"C:\Program Files (x86)\Android\android-sdk\ndk-bundle\platforms\android-21\arch-arm64\usr\lib\libm.so\"" C:\Users\jopepper\source\repos\App36\App36\obj\Release\120\android\assets\shrunk\App36.dll

1. Xamarin.Android passes *relative* paths. The `foo Ümläüts`
   directory name doesn't even come into play for some arguments.

2. With LLVM, `ld-flags` contains a `;`.

The existing code splits on `;` and joins on `,`:

https://github.com/dotnet/runtime/blob/25c207351c4f57cf2daa98caaf327a8b8d83edb8/src/tasks/AotCompilerTask/MonoAOTCompiler.cs#L505-L509

So we lose any `;` delimiters for the `ld-flags` value, they get
replaced by `,`.

I think the solution here is:

1. Add several missing properties to `<MonoAOTCompiler/>` so we don't
   have to rely on the `%(AotArguments)` item metadata. No splitting
   on `;` would be required, `ld-flags` can be passed in and used as-is.

2. Add a new `WorkingDirectory` property. When this is set, assume
   paths passed in might be relative -- and don't transform paths by
   calling `Path.GetFullPath()`.

Lastly, I fixed a place where the UTF8 encoding wasn't passed when
MSBuild logging the response file.

These changes I tried to make in a way where this shouldn't break
other .NET workloads like wasm. If existing MSBuild targets call this
task (not using the new properties), the behavior should remain
unchanged.

I tested these changes by commiting a modified `MonoAOTCompiler.dll`:

dotnet/android#6562

I'm able to enable several AOT tests related to #56163.

Co-authored-by: Jonathan Peppers <[email protected]>
…o-release/6.0-maui

[automated] Merge branch 'release/6.0' => 'release/6.0-maui'
…od_union_preclean (#63296)

Fixes mono/mono#21369

job_major_mod_union_preclean can race with the tarjan bridge
implementation that changes the vtable pointer by settings the three
lower bits. this results in invalid loading of the vtable
(shifted by 7 bytes)  which in turn give a wrong desc to the scan
functions

This change is released under the MIT license.

Co-authored-by: tmijieux <[email protected]>
* Bump SdkVersionForWorkloadTesting to use a 200 band SDK

#62787 bumped the manifest id.  This change bumps the prop for testing.

* Download dotnet-install script for installing workloads

Instead of trying to use the script from `.dotnet`, download the script.
`.dotnet` might not exist, for example, when the `global.json` version
matches the system installed one.

* Workloads: fix CI build when bumping to a newer version band

To resolve a workload, the current version band is used to look in
`sdk-manifests` directory. But when in development, and bumping to a
newer version band, not all the manifests might have packages for that.

This shows up when bumping the band to `6.0.200`, where we generate
the corresponding packages for wasm-tools. But the Emscripten packages
are still on `6.0.100`.

SDK [added](dotnet/sdk#19545) support for
falling back to older bands in that case.

The `InstallWorkloadFromArtifacts` task emits a warning about not being
able to find the emscripten manifest package. But this becomes an error
on CI due to warnaserror=true.

So, log a message instead of a warning. But this is only done for the
*dependent* manifest.

Co-authored-by: Steve Pfister <[email protected]>
Co-authored-by: Ankit Jain <[email protected]>
In particular this re-enables embedded PDB support on iOS and Android

The corresponding PR for main is #63365
This PR adds support for Process.Start() and Process.Kill() for Mac Catalyst. I also added a way to run Mac Catalyst tests in App Sandbox and I resolved the problems with libproc in App Sandbox (the app is unable to obtain a list of running processes).

Closes #61295, closes #61504.
… Android (#63870)

Co-authored-by: Simon Rozsival <[email protected]>
Co-authored-by: Alexander Köplinger <[email protected]>
…63936)

They aren't allowed by Apple when submitting to the App Store.

Finishes #61265
…tion (#63628)

* [Android] Fix accessing network interfaces information (#62780)

* Re-enable tests

* Dynamically load getifaddrs if necessary

* Prevent redeclaration of the ifaddrs struct

* Fix typo

* Do not close the dynamic library

* Enable the fixed functional tests in CI

* Reduce code duplication for obtaining libc file name

* Simplify usage of the function pointers

* Move typedefs

* Rename the _ensure_ function

* Remove fptr typedefs

* Update comment

* Add missing include

* Update comment

* Remove unnecessary comment

* Move static variable

* Remove unnecessary change

* Move LIBC_FILENAME to the utilities header

* Avoid error if constant is undefined

* Conditionally include ifaddrs

* Try to fix cmake_symbol_exists issue for browser

* Minor tweaks

* Try different way of detecting getifaddrs

* Use the hack only for Android builds

* Revert "Move LIBC_FILENAME to the utilities header"

This reverts commit 4e67687.

* Revert "Reduce code duplication for obtaining libc file name"

This reverts commit aca15d1.

* Simplify opening libc

* Update code style

* Fix race condition

* Prevent race condition

* Switch locking implementation for a lock-free implementation

* Enable unit test for Android

* Try using weak symbols

* Fix function name

* Revert "Fix function name"

This reverts commit f927aae.

* Revert "Try using weak symbols"

This reverts commit 46d3ede.

* Refactor code to use pthread_once

* [Android] Throw PNSE for unavailable network information (#63633)

* Update tests

* Add android specific implementation

* Add UnsupportedOSPlatform attributes

* Fix typo

* Remove unnecessary file reference

* Clean-up code

* Minor code clean-up

* Remove dictionary

* Refactoring

* Revert comment change

* Fix usage of Interop.Sys.GetNetworkInterfaces
Fixes #63693

It was discovered that Android produces duplicate TimeZone DisplayNames among all timezone IDs in GetSystemTimeZones. These duplicate DisplayNames occur across TimeZone IDs that are aliases, where all except one are backward timezone IDs.

If a name is changed, put its old spelling in the 'backward' file

From the Android TimeZone data file tzdata, it isn't obvious which TimeZone IDs are backward (I find it strange that they're included in the first place), however we discovered that on some versions of Android, there is an adjacent file tzlookup.xml that can aid us in determining which TimeZone IDs are "current" (not backward).

This PR aims to utilize tzlookup.xml when it exists and post-filter's the Populated TimeZone IDs in the AndroidTzData instance by removing IDs and their associated information (byteoffset and length) from the AndroidTzData instance if it is not found in tzlookup.xml. This is using the assumption that all non-backward TimeZone IDs make it to the tzlookup.xml file.

This PR also adds a new TimeZoneInfo Test to check whether or not there are duplicate DisplayNames in GetSystemTimeZones
…o-release/6.0-maui

[automated] Merge branch 'release/6.0' => 'release/6.0-maui'
…mblies (#64327)

* [mono] Remove support for Classic Xamarin for the Selector.GetHandle optimization, and add newer platforms. (#61989)

We haven't supported Classic mode for Xamarin for quite a few years now
(monotouch, MonoMac), so that code is not needed anymore. However, we've added
support for more platforms (tvOS, Mac Catalyst), so add support for those.

(cherry picked from commit b259f68)

# Conflicts:
#	src/mono/mono/mini/intrinsics.c

* [mono] Recognize new names for Xamarin.iOS etc assemblies (#64278)

They are being renamed in xamarin/xamarin-macios#13847

(cherry picked from commit 2b1cf1e)

Co-authored-by: Rolf Bjarne Kvinge <[email protected]>
Co-authored-by: Alexander Köplinger <[email protected]>
@akoeplinger akoeplinger added Servicing-approved Approved for servicing release area-codeflow for labeling automated codeflow labels Feb 2, 2022
@dotnet-issue-labeler
Copy link

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, to please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@akoeplinger
Copy link
Member Author

This was already approved as #64352.

@steveisok steveisok added the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label Feb 7, 2022
@steveisok
Copy link
Member

This change now has a blocking dependency on #64749. In addition, we'll need to publish 100 and 200 workload manifests - that will come in a PR shortly.

@steveisok
Copy link
Member

PR has been updated to produce a 100 and 200 mono toolchain manifest workloads. Once #64749 has been merged, this PR can go through.

@steveisok steveisok self-requested a review February 8, 2022 00:36
@steveisok steveisok removed the NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) label Feb 8, 2022
@steveisok
Copy link
Member

Merged in latest arcade bump. After CI runs, this is good to go.

@steveisok
Copy link
Member

steveisok commented Feb 8, 2022

Official build looks good... It includes the 100 and 200 manifest msi's.

https://dev.azure.com/dnceng/internal/_build/results?buildId=1599661&view=results

/cc @joeloff @marcpopMSFT

@safern
Copy link
Member

safern commented Feb 8, 2022

Failures are known. I need to put up a PR to fix them.

@safern safern merged commit 7a62f17 into release/6.0 Feb 8, 2022
@akoeplinger akoeplinger deleted the merge-maui-branch-to-6.0 branch February 9, 2022 00:00
@ghost ghost locked as resolved and limited conversation to collaborators Mar 11, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-codeflow for labeling automated codeflow new-api-needs-documentation Servicing-approved Approved for servicing release
Projects
None yet
Development

Successfully merging this pull request may close these issues.