Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inter-process Mutex doesn't work for processes of different architectures on Apple Silicon #62140

Closed
valco1994 opened this issue Nov 29, 2021 · 9 comments

Comments

@valco1994
Copy link
Contributor

Description

It's impossible to synchronize .NET processes of different architectures (x64 and aarch64) using System.Threading.Mutex on Apple Silicon

Reproduction Steps

Let's create the following simple console application:

using System;
using System.Threading;

class Program {
  static void Main(string[] args)
  {
      Mutex mutex = null;
      try
      {
          mutex = new Mutex(false, "Global\\MyUniqueMutexName");
          mutex.WaitOne();
          Thread.Sleep(TimeSpan.FromSeconds(60));
      }
      catch (Exception ex) { Console.WriteLine(ex); }
      finally { mutex?.ReleaseMutex(); }
  }
}

Then let's build it on Apple Silicon using two toolsets with .NET 6 - for x64 and aarch64.

And then launch them sequentially in any order.

Expected behavior

Both instances of the application run successfully and terminate ~60 seconds after the start of the first one (the second instance waits for the global mutex acquired by the first one).

Actual behavior

The second application is finished intermediately with following message

System.Threading.WaitHandleCannotBeOpenedException: A WaitHandle with system-wide name 'Global\MyUniqueMutexName' cannot be created. A WaitHandle of a different type might have the same name.
   at System.Threading.Mutex.CreateMutexCore(Boolean initiallyOwned, String name, Boolean& createdNew)
   at System.Threading.Mutex..ctor(Boolean initiallyOwned, String name)
   at Program.Main(String[] args) in /Users/Valentin.Grigorev/RiderProjects/GlobalMutexExperiment2/x64App/Program.cs:line 11

Regression?

No response

Known Workarounds

I don't know any workarounds

Configuration

.NET SDK (reflecting any global.json):
 Version:   6.0.100
 Commit:    9e8b04bbff

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  11.6
 OS Platform: Darwin
 RID:         osx.11.0-arm64
 Base Path:   /Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-arm64/sdk/6.0.100/

Host (useful for support):
  Version: 6.0.0
  Commit:  4822e3c3aa

.NET SDKs installed:
  6.0.100 [/Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-arm64/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.0 [/Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-arm64/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.0 [/Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-arm64/shared/Microsoft.NETCore.App]

and

.NET SDK (reflecting any global.json):
 Version:   6.0.100
 Commit:    9e8b04bbff

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  11.6
 OS Platform: Darwin
 RID:         osx.11.0-x64
 Base Path:   /Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-x64/sdk/6.0.100/

Host (useful for support):
  Version: 6.0.0
  Commit:  4822e3c3aa

.NET SDKs installed:
  6.0.100 [/Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-x64/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.0 [/Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-x64/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.0 [/Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-x64/shared/Microsoft.NETCore.App]

Other information

No response

@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Nov 29, 2021
@janvorli
Copy link
Member

cc: @kouvel

@ghost
Copy link

ghost commented Nov 29, 2021

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

It's impossible to synchronize .NET processes of different architectures (x64 and aarch64) using System.Threading.Mutex on Apple Silicon

Reproduction Steps

Let's create the following simple console application:

using System;
using System.Threading;

class Program {
  static void Main(string[] args)
  {
      Mutex mutex = null;
      try
      {
          mutex = new Mutex(false, "Global\\MyUniqueMutexName");
          mutex.WaitOne();
          Thread.Sleep(TimeSpan.FromSeconds(60));
      }
      catch (Exception ex) { Console.WriteLine(ex); }
      finally { mutex?.ReleaseMutex(); }
  }
}

Then let's build it on Apple Silicon using two toolsets with .NET 6 - for x64 and aarch64.

And then launch them sequentially in any order.

Expected behavior

Both instances of the application run successfully and terminate ~60 seconds after the start of the first one (the second instance waits for the global mutex acquired by the first one).

Actual behavior

The second application is finished intermediately with following message

System.Threading.WaitHandleCannotBeOpenedException: A WaitHandle with system-wide name 'Global\MyUniqueMutexName' cannot be created. A WaitHandle of a different type might have the same name.
   at System.Threading.Mutex.CreateMutexCore(Boolean initiallyOwned, String name, Boolean& createdNew)
   at System.Threading.Mutex..ctor(Boolean initiallyOwned, String name)
   at Program.Main(String[] args) in /Users/Valentin.Grigorev/RiderProjects/GlobalMutexExperiment2/x64App/Program.cs:line 11

Regression?

No response

Known Workarounds

I don't know any workarounds

Configuration

.NET SDK (reflecting any global.json):
 Version:   6.0.100
 Commit:    9e8b04bbff

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  11.6
 OS Platform: Darwin
 RID:         osx.11.0-arm64
 Base Path:   /Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-arm64/sdk/6.0.100/

Host (useful for support):
  Version: 6.0.0
  Commit:  4822e3c3aa

.NET SDKs installed:
  6.0.100 [/Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-arm64/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.0 [/Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-arm64/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.0 [/Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-arm64/shared/Microsoft.NETCore.App]

and

.NET SDK (reflecting any global.json):
 Version:   6.0.100
 Commit:    9e8b04bbff

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  11.6
 OS Platform: Darwin
 RID:         osx.11.0-x64
 Base Path:   /Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-x64/sdk/6.0.100/

Host (useful for support):
  Version: 6.0.0
  Commit:  4822e3c3aa

.NET SDKs installed:
  6.0.100 [/Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-x64/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.0 [/Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-x64/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.0 [/Users/Valentin.Grigorev/Downloads/dotnet-sdk-6.0.100-osx-x64/shared/Microsoft.NETCore.App]

Other information

No response

Author: valco1994
Assignees: -
Labels:

area-System.Threading, untriaged

Milestone: -

@mangod9 mangod9 removed the untriaged New issue has not been triaged by the area owner label Nov 29, 2021
@mangod9 mangod9 added this to the 7.0.0 milestone Nov 29, 2021
@kouvel
Copy link
Member

kouvel commented Dec 2, 2021

Probably the issue is that the page size is different and the code is currently strict about the size of the shared memory file, which is based on the page size. I'll see if something can be done to fix it.

@kouvel kouvel modified the milestones: 7.0.0, 6.0.x Dec 2, 2021
@k15tfu
Copy link
Contributor

k15tfu commented Dec 3, 2021

@kouvel Hi! It uses different types on macOS x64 & macOS ARM64, isn't it?

// Temporarily disabling usage of pthread process-shared mutexes on ARM/ARM64 due to functional issues that cannot easily be
// detected with code due to hangs. See https://github.com/dotnet/runtime/issues/6014.
#if HAVE_FULLY_FEATURED_PTHREAD_MUTEXES && HAVE_FUNCTIONAL_PTHREAD_ROBUST_MUTEXES && !(defined(HOST_ARM) || defined(HOST_ARM64) || defined(__FreeBSD__))
#define NAMED_MUTEX_USE_PTHREAD_MUTEX 1
#else
#define NAMED_MUTEX_USE_PTHREAD_MUTEX 0
#endif

@kouvel
Copy link
Member

kouvel commented Dec 3, 2021

It should be using file locks for both, I believe OSX doesn't have pthread robust mutexes

@k15tfu
Copy link
Contributor

k15tfu commented Dec 4, 2021

Okay I checked, the error is coming from here

else if (SharedMemoryHelpers::GetFileSize(fileDescriptor) != sharedDataTotalByteCount)
{
throw SharedMemoryException(static_cast<DWORD>(SharedMemoryError::HeaderMismatch));
}

@kouvel
Copy link
Member

kouvel commented Dec 4, 2021

Yea that would most likely be because the page size is different between x64 and arm64. The file size is set to the page size.

kouvel added a commit to kouvel/runtime that referenced this issue Dec 14, 2021
…sses

- The page size is different between arm64 processes and emulated x64 processes
- The shared memory file size is set to the page size and there was a strict check on the file size, leading to an exception from the second process of a different arch that tries to share the same mutex
- Made the file size check less strict, and allowed an arch to increase but not decrease the file size such that it can be mapped at page size granularity
- Fix for dotnet#62140 in main
kouvel added a commit to kouvel/runtime that referenced this issue Dec 14, 2021
…sses

- The page size is different between arm64 processes and emulated x64 processes
- The shared memory file size is set to the page size and there was a strict check on the file size, leading to an exception from the second process of a different arch that tries to share the same mutex
- Made the file size check less strict, and allowed an arch to increase but not decrease the file size such that it can be mapped at page size granularity
- Fix for dotnet#62140 in main
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Dec 14, 2021
kouvel added a commit that referenced this issue Dec 14, 2021
…sses (#62764)

- The page size is different between arm64 processes and emulated x64 processes
- The shared memory file size is set to the page size and there was a strict check on the file size, leading to an exception from the second process of a different arch that tries to share the same mutex
- Made the file size check less strict, and allowed an arch to increase but not decrease the file size such that it can be mapped at page size granularity
- Fix for #62140 in main
jeffschwMSFT pushed a commit that referenced this issue Jan 6, 2022
…sses (#62765)

- The page size is different between arm64 processes and emulated x64 processes
- The shared memory file size is set to the page size and there was a strict check on the file size, leading to an exception from the second process of a different arch that tries to share the same mutex
- Made the file size check less strict, and allowed an arch to increase but not decrease the file size such that it can be mapped at page size granularity
- Fix for #62140 in main
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jan 6, 2022
@kouvel
Copy link
Member

kouvel commented Jan 7, 2022

Fixed by #62765, should be available in the next release (SDK 6.0.102 / runtime 6.0.2). Both the x64 and arm64 runtimes would need to be updated for them to be able to synchronize on a named mutex.

@kouvel kouvel closed this as completed Jan 7, 2022
@ghost ghost locked as resolved and limited conversation to collaborators Feb 6, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

6 participants