Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some images are only displayed in black & white, old Intel (before 2009) and QEMU-emulated CPUs #3893

Closed
4 tasks done
Zharkan opened this issue Dec 11, 2023 · 14 comments
Closed
4 tasks done

Comments

@Zharkan
Copy link

Zharkan commented Dec 11, 2023

Possible install-time or require-time problem

Are you using the latest version of sharp?

  • I am using the latest version of sharp as reported by npm view sharp dist-tags.latest.

Are you using a supported runtime?

  • I am using Node.js with a version that satisfies ^18.17.0 || ^20.3.0 || >=21.0.0

Are you using a supported package manager and installing optional dependencies?

  • I am using npm >= 9.6.5 with --include=optional

What is the complete output of running npm install --verbose --foreground-scripts sharp in an empty directory?

npm info using [email protected]
npm info using [email protected]
npm verb title npm install sharp
npm verb argv "install" "--loglevel" "verbose" "--foreground-scripts" "sharp"
npm verb logfile logs-max:10 dir:/home/media_gestion/.npm/_logs/2023-12-11T23_18_28_144Z-
npm verb logfile /home/media_gestion/.npm/_logs/2023-12-11T23_18_28_144Z-debug-0.log
npm http fetch GET 200 https://registry.npmjs.org/sharp 353ms (cache revalidated)
npm verb reify failed optional dependency /path/to/project/node_modules/fsevents
npm verb reify failed optional dependency /path/to/project/node_modules/@next/swc-win32-x64-msvc
npm verb reify failed optional dependency /path/to/project/node_modules/@next/swc-win32-ia32-msvc
npm verb reify failed optional dependency /path/to/project/node_modules/@next/swc-win32-arm64-msvc
npm verb reify failed optional dependency /path/to/project/node_modules/@next/swc-linux-arm64-musl
npm verb reify failed optional dependency /path/to/project/node_modules/@next/swc-linux-arm64-gnu
npm verb reify failed optional dependency /path/to/project/node_modules/@next/swc-darwin-x64
npm verb reify failed optional dependency /path/to/project/node_modules/@next/swc-darwin-arm64
npm verb reify failed optional dependency /path/to/project/node_modules/@img/sharp-win32-x64
npm verb reify failed optional dependency /path/to/project/node_modules/@img/sharp-win32-ia32
npm verb reify failed optional dependency /path/to/project/node_modules/@img/sharp-wasm32
npm verb reify failed optional dependency /path/to/project/node_modules/@emnapi/runtime
npm verb reify failed optional dependency /path/to/project/node_modules/@img/sharp-linuxmusl-arm64
npm verb reify failed optional dependency /path/to/project/node_modules/@img/sharp-linux-s390x
npm verb reify failed optional dependency /path/to/project/node_modules/@img/sharp-linux-arm64
npm verb reify failed optional dependency /path/to/project/node_modules/@img/sharp-linux-arm
npm verb reify failed optional dependency /path/to/project/node_modules/@img/sharp-libvips-linuxmusl-arm64
npm verb reify failed optional dependency /path/to/project/node_modules/@img/sharp-libvips-linux-s390x
npm verb reify failed optional dependency /path/to/project/node_modules/@img/sharp-libvips-linux-arm64
npm verb reify failed optional dependency /path/to/project/node_modules/@img/sharp-libvips-linux-arm
npm verb reify failed optional dependency /path/to/project/node_modules/@img/sharp-libvips-darwin-x64
npm verb reify failed optional dependency /path/to/project/node_modules/@img/sharp-libvips-darwin-arm64
npm verb reify failed optional dependency /path/to/project/node_modules/@img/sharp-darwin-x64
npm verb reify failed optional dependency /path/to/project/node_modules/@img/sharp-darwin-arm64
npm http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/advisories/bulk 308ms

What is the output of running npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp?

  System:
    OS: Linux 5.15 Ubuntu 22.04.3 LTS 22.04.3 LTS (Jammy Jellyfish)
    CPU: (8) x64 Common KVM processor
    Memory: 2.10 GB / 5.16 GB
    Container: Yes
    Shell: 5.8.1 - /usr/bin/zsh
  Binaries:
    Node: 20.10.0 - /usr/bin/node
    Yarn: 1.22.19 - ~/.npm-global/bin/yarn
    npm: 10.2.5 - ~/.npm-global/bin/npm
  npmPackages:
    sharp: ^0.33.0 => 0.33.0 

I'm building my project with Sharp (Next.js app) and some images are black & white. There's also a message when I'm running npm run build :

 ✓ Linting and checking validity of types    
   ▲ Next.js 14.0.4
   - Environments: .env.production
   - Experiments (use at your own risk):
     · scrollRestoration

   Creating an optimized production build  ...WARNING: CPU supports 0x6000000000004000, software requires 0x4000000000005000
WARNING: CPU supports 0x6000000000004000, software requires 0x4000000000005000
WARNING: CPU supports 0x6000000000004000, software requires 0x4000000000005000
   Creating an optimized production build  .WARNING: CPU supports 0x6000000000004000, software requires 0x4000000000005000
WARNING: CPU supports 0x6000000000004000, software requires 0x4000000000005000
@lovell
Copy link
Owner

lovell commented Dec 12, 2023

WARNING: CPU supports 0x6000000000004000, software requires 0x4000000000005000

CPU: (8) x64 Common KVM processor

This is due to the use of a custom CPU, possibly QEMU-emulated, with somewhat non-standard flags.

There's an upstream fix to support this configuration at libvips/libvips#3763 that has not yet been included in a libvips release.

You can workaround it for now with one of the following:

  • ensuring the hypervisor allows the use of CPU intrinsics such as SSE4
  • setting the VIPS_NOVECTOR environment variable before importing/requiring sharp (requires v0.33.1)
  • using sharp.simd(false) to switch off the use of SIMD.

https://sharp.pixelplumbing.com/api-utility#simd

@lovell lovell changed the title Some images are only displayed in black & white (Next.js + sharp, self hosted) Some images are only displayed in black & white, QEMU-emulated CPU Dec 12, 2023
@icetee
Copy link
Contributor

icetee commented Dec 14, 2023

WARNING: CPU supports 0x6000000000004000, software requires 0x4000000000005000

CPU: (8) x64 Common KVM processor

This is due to the use of a custom CPU, possibly QEMU-emulated, with somewhat non-standard flags.

There's an upstream fix to support this configuration at libvips/libvips#3763 that has not yet been included in a libvips release.

You can workaround it for now with one of the following:

  • ensuring the hypervisor allows the use of CPU flags such as SSE4
  • setting the VIPS_NOVECTOR environment variable
  • using sharp.simd(false) to switch off the use of SIMD.

https://sharp.pixelplumbing.com/api-utility#simd

The VIPS_NOVECTOR env is not working, but sharp.simd(false) did.

@lovell
Copy link
Owner

lovell commented Dec 15, 2023

@icetee You'll need to set VIPS_NOVECTOR before calling require('sharp') (I've updated my comment to reflect this).

@icetee
Copy link
Contributor

icetee commented Dec 15, 2023

I will use Docker and I defined the env before installing anything.

If print the envs (printenv), I see VIPS_NOVECTOR=0

This is correct environment :/

@kleisauke
Copy link
Contributor

It looks like the VIPS_NOVECTOR environment variable is not effective due to this line:

simd(true);

@lovell
Copy link
Owner

lovell commented Dec 15, 2023

D'oh! I'll take a look at your PR now @icetee, thank you.

@lovell lovell changed the title Some images are only displayed in black & white, QEMU-emulated CPU Some images are only displayed in black & white, old Intel (before 2009) and QEMU-emulated CPUs Dec 18, 2023
Amir-Zouerami added a commit to Amir-Zouerami/zouerami-blog that referenced this issue Dec 21, 2023
@lovell lovell added this to the v0.33.2 milestone Dec 24, 2023
@AndrStnz
Copy link

AndrStnz commented Jan 4, 2024

WARNING: CPU supports 0x6000000000004000, software requires 0x4000000000005000

CPU: (8) x64 Common KVM processor

This is due to the use of a custom CPU, possibly QEMU-emulated, with somewhat non-standard flags.

There's an upstream fix to support this configuration at libvips/libvips#3763 that has not yet been included in a libvips release.

You can workaround it for now with one of the following:

  • ensuring the hypervisor allows the use of CPU intrinsics such as SSE4

Setting the CPU Type in Proxmox to "Host" instead of "default" did indeed solve the issue for me.
Thank you very much for the hint!

@lovell
Copy link
Owner

lovell commented Jan 12, 2024

v0.33.2 now available with prebuilt binaries that include libvips v8.15.1 and therefore the upstream fix.

@mamlzy
Copy link

mamlzy commented Feb 20, 2024

sorry, how to fix this WARNING: CPU supports 0x6000000000004000, software requires 0x4000000000005000? in nextjs with docker. from this conversation i don't reallly get it how to fix it, can someone help? thanks!

@lovell
Copy link
Owner

lovell commented Feb 20, 2024

@ImamAlfariziSyahputra Please ensure you are using at least version 0.33.2 of sharp.

@mamlzy
Copy link

mamlzy commented Feb 20, 2024

@ImamAlfariziSyahputra Please ensure you are using at least version 0.33.2 of sharp.

I am using the latest version:
image

The warning console:
image

@kachkaev
Copy link

kachkaev commented Mar 3, 2024

Same here inside a docker container on Linux:

#22 16.36 WARNING: CPU supports 0x6000000000004000, software requires 0x4000000000005000
(repeats many times)
pnpm why sharp

dependencies:
image-minimizer-webpack-plugin 3.8.3
└── sharp 0.33.2 peer
sharp 0.33.2

Is the phrasing in the error specific to sharp? I wonder if there may be some other package that’s causing it.

@lovell
Copy link
Owner

lovell commented Mar 4, 2024

If you're relying on a globally-installed libvips, please ensure it is at least version v8.15.1.

@kleisauke
Copy link
Contributor

The following warning:

WARNING: CPU supports 0x6000000000004000, software requires 0x4000000000005000

originates from Highway and indicates a discrepancy between the capabilities of the CPU and the requirements set by the prebuilt binaries. Specifically, the x86::DetectTargets() function determines that the CPU supports only SSE2, a baseline feature of x64 architectures. However, the prebuilt binaries were compiled targeting the x86-64-v2 micro-architecture, which corresponds to Highway's S-SSE3 feature 'group'.

Details
#include <cstdio>
#include <cstdint>

#define HWY_EMU128 (1LL << 61)
#define HWY_SCALAR (1LL << 62)

#define HWY_SSE2 (1LL << 14)
#define HWY_SSSE3 (1LL << 12)  // S-SSE3

int main() {
  const uint64_t bits_u = static_cast<uint64_t>(HWY_SCALAR | HWY_EMU128 | HWY_SSE2);
  const uint64_t enabled = static_cast<uint64_t>(HWY_SCALAR | HWY_SSSE3 | HWY_SSE2);
  fprintf(stderr,
          "WARNING: CPU supports 0x%08x%08x, software requires 0x%08x%08x\n",
          static_cast<uint32_t>(bits_u >> 32),
          static_cast<uint32_t>(bits_u & 0xFFFFFFFF),
          static_cast<uint32_t>(enabled >> 32),
          static_cast<uint32_t>(enabled & 0xFFFFFFFF));
  // WARNING: CPU supports 0x6000000000004000, software requires 0x4000000000005000

  return 0;
}

In summary, the appearance of this warning on an x86-64 system suggests that the ssse3 flag is absent when the CPUID instruction is queried. This could stem from either the current CPU lacking S-SSE3 support, potentially leading to a SIGILL (illegal instruction) error, or the hypervisor masking the CPUID feature flags of the underlying CPU, which would affect performance.

To check for the presence of specific features on Linux, one can examine /proc/cpuinfo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants