v0.31.2: macOS x64 prebuilt binaries fail on macOS < 11 with "Symbol not found: _pthread_jit_write_protect_supported_np"

[nosferatu500]

Possible install-time or require-time problem

• I have read the documentation relating to installation.
• I have ensured that the architecture and platform of Node.js used for npm install is the same as the architecture and platform of Node.js used at runtime.

Are you using the latest version of sharp?

• I am using the latest version of sharp as reported by npm view sharp dist-tags.latest.

If you cannot confirm this, please upgrade to the latest version and try again before opening an issue.

If you are using another package which depends on a version of sharp that is not the latest, please open an issue against that package instead.

---

Hello! After updating Sharp from version 0.31.1 to version 0.31.2, some of my users can no longer run my application.

Dyld Error Message:
  Symbol not found: _pthread_jit_write_protect_supported_np
  Referenced from: /Applications/electron.app/Contents/Resources/app.asar.unpacked/node_modules/sharp/build/Release/../.././vendor/8.13.3/darwin-x64/lib/libvips-cpp.42.dylib
  Expected in: /usr/lib/libSystem.B.dylib

I can guess that this problem is probably inside libvips (as shown in the error message), but in the installation guide you state:

Prebuilt binaries

Ready-compiled sharp and libvips binaries are provided for use on the most common platforms:

 - macOS x64 (>= 10.13)
 - macOS ARM64
 - Linux x64 (glibc >= 2.17, musl >= 1.1.24, CPU with SSE4.2)
 - Linux ARM64 (glibc >= 2.17, musl >= 1.1.24)
 - Windows x64
 - Windows x86

So I'm just wondering if this bug should be expected to be fixed or should the supported macOS version be higher?

[bobsam]

Also happend on MacOS 10.15.7

dyld: lazy symbol binding failed: Symbol not found: _pthread_jit_write_protect_supported_np
  Referenced from: /Users/bob/Documents/workspace/git.nie/art-design-node/node_modules/sharp/build/Release/../.././vendor/8.13.3/darwin-x64/lib/libvips-cpp.42.dylib
  Expected in: /usr/lib/libSystem.B.dylib

dyld: Symbol not found: _pthread_jit_write_protect_supported_np
  Referenced from: /Users/bob/Documents/workspace/git.nie/art-design-node/node_modules/sharp/build/Release/../.././vendor/8.13.3/darwin-x64/lib/libvips-cpp.42.dylib
  Expected in: /usr/lib/libSystem.B.dylib

[lovell]

This looks like something to do with the macOS Hardened Runtime introduced in 10.14 (Mojave).

The symbol requirement has been introduced via a commit to the upstream liborc (used by libvips for SIMD) - see commit https://cgit.freedesktop.org/gstreamer/orc/commit/?id=b732715f737595884dd54d1f7860982652b2482f

The CI environment used by sharp is running macOS 11 as that's the oldest version available to us (GitHub Actions was supposed to remove 10.15 support entirely a couple of months ago, but it's still flagged as "deprecated" in the docs), so perhaps it's time to upgrade the minimum required version as that's all we can test against?

I guess this will affect only x64 hardware. Do all versions of Node.js exhibit this behaviour? Does the Node.js executable itself on these machines support the Hardened Runtime (e.g. via the com.apple.security.cs.allow-jit permission)?

[kleisauke]

I wonder why Meson returns true for cc.has_function('pthread_jit_write_protect_np') when MACOSX_DEPLOYMENT_TARGET is set to 10.9, according to commit mesonbuild/meson@a06178f it should honor the -mmacosx-version-min setting.

@eli-schwartz Can I ping you on this? Perhaps we need to explicty pass -mmacosx-version-min to the compiler flags? According to the Clang documentation this shouldn't be necessary.
https://clang.llvm.org/docs/CommandGuide/clang.html#envvar-MACOSX_DEPLOYMENT_TARGET

[kleisauke]

Ah, looking at the GitHub Actions log, it looks like it already warned about this.

Details

../orc/orccompiler.c:132:7: warning: 'pthread_jit_write_protect_supported_np' is only available on macOS 11.0 or newer [-Wunguarded-availability-new]
      pthread_jit_write_protect_supported_np());
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../orc/orcdebug.h:99:55: note: expanded from macro 'ORC_INFO'
#define ORC_INFO(...) ORC_DEBUG_PRINT(ORC_DEBUG_INFO, __VA_ARGS__)
                                                      ^~~~~~~~~~~
../orc/orcdebug.h:138:62: note: expanded from macro 'ORC_DEBUG_PRINT'
  orc_debug_print((level), __FILE__, ORC_FUNCTION, __LINE__, __VA_ARGS__); \
                                                             ^~~~~~~~~~~
/Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/pthread.h:565:5: note: 'pthread_jit_write_protect_supported_np' has been marked as being introduced in macOS 11.0 here, but the deployment target is macOS 10.9.0
int pthread_jit_write_protect_supported_np(void);
    ^
../orc/orccompiler.c:132:7: note: enclose 'pthread_jit_write_protect_supported_np' in a __builtin_available check to silence this warning
      pthread_jit_write_protect_supported_np());
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../orc/orcdebug.h:99:55: note: expanded from macro 'ORC_INFO'
#define ORC_INFO(...) ORC_DEBUG_PRINT(ORC_DEBUG_INFO, __VA_ARGS__)
                                                      ^~~~~~~~~~~
../orc/orcdebug.h:138:62: note: expanded from macro 'ORC_DEBUG_PRINT'
  orc_debug_print((level), __FILE__, ORC_FUNCTION, __LINE__, __VA_ARGS__); \
                                                             ^~~~~~~~~~~
../orc/orccompiler.c:460:3: warning: 'pthread_jit_write_protect_np' is only available on macOS 11.0 or newer [-Wunguarded-availability-new]
  pthread_jit_write_protect_np(0);
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/pthread.h:561:6: note: 'pthread_jit_write_protect_np' has been marked as being introduced in macOS 11.0 here, but the deployment target is macOS 10.9.0
void pthread_jit_write_protect_np(int enabled);
     ^
../orc/orccompiler.c:460:3: note: enclose 'pthread_jit_write_protect_np' in a __builtin_available check to silence this warning
  pthread_jit_write_protect_np(0);
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
../orc/orccompiler.c:479:3: warning: 'pthread_jit_write_protect_np' is only available on macOS 11.0 or newer [-Wunguarded-availability-new]
  pthread_jit_write_protect_np(1);
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Applications/Xcode_13.2.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/pthread.h:561:6: note: 'pthread_jit_write_protect_np' has been marked as being introduced in macOS 11.0 here, but the deployment target is macOS 10.9.0
void pthread_jit_write_protect_np(int enabled);
     ^
../orc/orccompiler.c:479:3: note: enclose 'pthread_jit_write_protect_np' in a __builtin_available check to silence this warning
  pthread_jit_write_protect_np(1);
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
3 warnings generated.

Perhaps adding -Werror=unguarded-availability-new here:
https://github.com/mesonbuild/meson/blob/0b7d935a846b8f2aa33b4e0d19fd7b4423d35df4/mesonbuild/compilers/mixins/clang.py#L110

Would cause cc.has_function return false for pthread_jit_write_protect_np when macOS 10.9 is targeted?

[lovell]

This problem appears to affect glib too. The following error occurs when attempting to compile glib with -Werror=unguarded-availability-new and MACOSX_DEPLOYMENT_TARGET=10.9.

../gio/glocalfileinfo.c:2779:9: error: 'utimensat' is only available on macOS 10.13 or newer [-Werror,-Wunguarded-availability-new]
  res = utimensat (AT_FDCWD, filename, times_n, 0);

[lovell]

PR to upgrade the minimum macOS to 10.13 as documented, introduce a build-time check for misuse of newer API, plus a temporary workaround for orc at lovell/sharp-libvips#164

[simonpeters]

is there a temporary fix this?

[leonism]

I'm having the similar issue, wondering on ways to solve the situation.

[lovell]

For macOS 10 support you'll currently need to temporarily pin sharp to v0.31.1 or earlier.

macOS 10 reached end-of-life in September this year, hence CI providers removing support for it, hence the unfortunate situation where we weren't able to test using it.

Hopefully the fixes and checks now put in place will allow sharp to re-introduce support for the documented macOS 10.13+ from the next release onwards, which is well beyond Apple's official support, and should allow those using e.g. older hardware to continue to benefit from it.

[leonism]

Hi @lovell, thanks for the advise, will try to give it a go by tonight.

[leonism]

Hi @lovell, thanks for the advise, will try to give it a go by tonight.

Well, as it turned out I don't have to wait till tonight, I have access to my local machine again. Here are the steps for everyone else that came here from a Google Search, and I noticed that it's a MacOSX specific issue. I'm on node v16.17.1, running on MacOS Mojave 10.14.6.

• npm uninstall sharp and
• npm install [email protected]

And, they're back again working as intended.

Thanks,

[nosferatu500]

@lovell Sorry to bother you, but I didn't find any information about this issue in the changelog for v0.31.3. Will the new version work on MacOS 10.13-10.15 or should we wait for another release?

[lovell]

@nosferatu500 Please subscribe to this issue for updates specific to it. There will be a future release that re-introduces support for macOS >= 10.13 on a best-effort basis, but please remember there are no longer suitable CI environments available to guarantee this.

If you are able to upgrade to macOS 11 then I highly recommend doing so as macOS 10 reached end-of-life in September and is now unsupported by Apple.

[nosferatu500]

@lovell

Got it, thank you!

PS: I personally use macOS 13, but about 20 people using my app are still using macOS 10.14. So I wanted to know how things are going with support older versions of MacOS. Thank you again.

[kleisauke]

Upstream issue: https://gitlab.freedesktop.org/gstreamer/orc/-/issues/44 (I deliberately didn't open a PR there because I don't have access to iOS devices).

[housejumper]

I am on macOS 10.15 and also getting this error with v0.31.3. According to sharp.pixelplumbing.com/install anything >= macOS 10.13 should be supported?

Also it seems like gitlab.freedesktop.org/gstreamer/orc/-/issues/44 has been merged, so maybe that will fix some things?

[conceptualspace]

Hopefully the fixes and checks now put in place will allow sharp to re-introduce support for the documented macOS 10.13+ from the next release onwards, which is well beyond Apple's official support, and should allow those using e.g. older hardware to continue to benefit from it.

can you clarify what fixes you mean and if there is a timeline for a release with them? understood it's impossible to officially support, but unfortunately macOS 10.x still has as much market share as 11/12/13 combined (at least in my userbase). it might help those of us trying to figure out how best to support them. thanks!

[neonexus]

I've been having issues myself with some dependencies of dependencies. So far, what I've found that has fixed the issue, without causing any problems, is downgrading to v0.31.1. I had to manually do this in my package-lock.json, but it was pretty easy, and everything works again!

Here is the full entry in package-lock.json, but you should only need to update the version numbers and integrity key:

{
     "node_modules/sharp": {
          "version": "0.31.1",
          "resolved": "https://registry.npmjs.org/sharp/-/sharp-0.31.1.tgz",
          "integrity": "sha512-GR8M1wBwOiFKLkm9JPun27OQnNRZdHfSf9VwcdZX6UrRmM1/XnOrLFTF0GAil+y/YK4E6qcM/ugxs80QirsHxg==",
          "dev": true,
          "hasInstallScript": true,
          "dependencies": {
               "color": "^4.2.3",
               "detect-libc": "^2.0.1",
               "node-addon-api": "^5.0.0",
               "prebuild-install": "^7.1.1",
               "semver": "^7.3.8",
               "simple-get": "^4.0.1",
               "tar-fs": "^2.1.1",
               "tunnel-agent": "^0.6.0"
          },
          "engines": {
               "node": ">=14.15.0"
          },
          "funding": {
               "url": "https://opencollective.com/libvips"
          }
     }
}

[conceptualspace]

thanks @neonexus. i wasn't sure v31.1 was an option in my case because of #3384, but pinning electron to v19 works for now. cheers 😃

[lovell]

The next release of the prebuilt binaries will include the upstream patch added to orc a couple of weeks ago (via commit lovell/sharp-libvips@d3adcca ) that provides both compile and runtime checks, which is much more comprehensive than our original quick fix. These will be made available as part of sharp v0.32.0.

[aztack]

FYI:
Make sure there is only one sharp package inside <your-project-dir>/node_modules/**/.
Add

"resolutions": {
  "sharp": "0.31.1"
}

when needed.

[lovell]

v0.32.0 now available with prebuilt binaries that contains a patched orc for macOS 10.13+ support.

[bobgravity1]

l literally just scrapped my whole computer HD and started with a fresh install of Ventura from mojave... took me a solid 2 days to get everything in order.

Now sharp supports... LOL. Anyways I guess it was for the best