Sharp 0.33 does not create binaries for alpine linux / docker

[rhufsky]

Possible bug

Is this a possible bug in a feature of sharp, unrelated to installation?

• Running npm install sharp completes without error.
• Running node -e "require('sharp')" completes without error.

If you cannot confirm both of these, please open an installation issue instead.

Are you using the latest version of sharp?

• I am using the latest version of sharp as reported by npm view sharp dist-tags.latest.

If you cannot confirm this, please upgrade to the latest version and try again before opening an issue.

If you are using another package which depends on a version of sharp that is not the latest, please open an issue against that package instead.

What is the output of running npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp?

System:
OS: Linux 5.4 Ubuntu 20.04.6 LTS (Focal Fossa)
CPU: (4) x64 Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz
Memory: 29.95 GB / 31.33 GB
Container: Yes
Shell: 5.0.17 - /bin/bash
Binaries:
Node: 20.5.1 - /usr/bin/node
npm: 10.2.3 - /usr/bin/npm
npmPackages:
sharp: ^0.33 => 0.33.1

What are the steps to reproduce?

After a lot of tinkering with docker I narrowed it down to:

I want to use Alpine Linux and I guess I have to rebuild sharp for that so I do:

npm rebuild --arch=x64 --platform=linux --libc=musl sharp

For Sharp 0.33 I get:

ls node_modules/sharp/
install  lib  LICENSE  package.json  README.md  src

No binaries are generated. Hence, no binaries are moved to the docker image and sharp does not work in the resulting container.

What is the expected behaviour?

For Sharp 0.32.6 I get:

ls node_modules/sharp/
binding.gyp  build  install  lib  LICENSE  package.json  README.md  src  vendor

The vendor folder seems to contain the generated binaries. As a consequence, the generated image and the container works as desired.

Please provide a minimal, standalone code sample, without other dependencies, that demonstrates this problem

Tested on Ubuntu 22.04 as a base system for my Docker builds.

Please provide sample image(s) that help explain this problem

[lovell]

I want to use Alpine Linux and I guess I have to rebuild sharp

Hi, this sounds like an installation issue. It might be faster if you provide answers to all of the questions in https://github.com/lovell/sharp/issues/new?labels=installation&template=installation.md

Please ensure you've read the installation documentation, especially when upgrading to v0.33.0:

• Supporting multi-platform within the same installation tree - https://sharp.pixelplumbing.com/install#cross-platform
• Building from source to use a globally-installed libvips - https://sharp.pixelplumbing.com/install#building-from-source

[lovell]

@gtwilliams03 The ZSt28__throw_bad_array_new_lengthv error suggests this is a different problem. Please see #3872, which might suggest you're using an outdated version of Alpine Linux. If you're still having problems, please open a new installation issue.

[rhufsky]

As per the docs in https://sharp.pixelplumbing.com/install#cross-platform I tried:

Instead of building sharp like

RUN npm install && \
        npm rebuild --arch=x64 --platform=linux --libc=musl sharp

I installed the platform specific version

RUN npm install && \
    npm install --cpu=x64 --os=linux --libc=musl sharp

For sharp 0.32.6 this generates the binaries in node_js/sharp, sharp just works fine

/app/node_modules/sharp $ ls -al 
total 28
drwxr-xr-x    5 nextjs   nodejs        4096 Dec 18 21:15 .
drwxr-xr-x   29 nextjs   nodejs        4096 Dec 18 21:15 ..
drwxr-xr-x    3 nextjs   nodejs        4096 Dec 18 21:15 build
drwxr-xr-x    2 nextjs   nodejs        4096 Dec 18 21:15 lib
-rw-r--r--    1 nextjs   nodejs        7821 Dec 18 21:15 package.json
drwxr-xr-x    3 nextjs   nodejs        4096 Dec 18 21:15 vendor

for 0.33 this does not generate the binaries. My app then complains that it could not find sharp.

/app/node_modules/sharp $ ls -al 
total 28
drwxr-xr-x    5 nextjs   nodejs        4096 Dec 18 21:15 .
drwxr-xr-x   29 nextjs   nodejs        4096 Dec 18 21:15 ..
drwxr-xr-x    2 nextjs   nodejs        4096 Dec 18 21:15 lib

[lovell]

The binaries live in packages under the @img/ namespace. I've just tried running the exact commands in the documentation locally and it works fine:

$ npm -v
10.2.3

$ ls node_modules/@img
ls: cannot access 'node_modules/@img': No such file or directory

$ npm install --cpu=x64 --os=darwin sharp
added 13 packages, and audited 14 packages in 1s

$ ls  node_modules/@img
sharp-darwin-x64  sharp-libvips-darwin-x64

$ npm install --cpu=arm64 --os=darwin sharp
added 2 packages, and audited 16 packages in 505ms

$ ls  node_modules/@img
sharp-darwin-arm64  sharp-darwin-x64  sharp-libvips-darwin-arm64  sharp-libvips-darwin-x64

You have probably run into the npm lockfile bug npm/cli#4828 that is linked to in the documentation.

https://sharp.pixelplumbing.com/install#cross-platform

[gtwilliams03]

@gtwilliams03 The ZSt28__throw_bad_array_new_lengthv error suggests this is a different problem. Please see #3872, which might suggest you're using an outdated version of Alpine Linux. If you're still having problems, please open a new installation issue.

@lovell Thank you! I updated the first line of the Dockerfile to:

FROM node:21-alpine3.19

and all is well. I had been using alpine 3.15 and hadn't realized I had not upgraded it in a while... I'll show myself out and stop hijacking the ticket :)

[rhufsky]

The binaries live in packages under the @img/ namespace. I've just tried running the exact commands in the documentation locally and it works fine:

$ npm -v
10.2.3

$ ls node_modules/@img
ls: cannot access 'node_modules/@img': No such file or directory

$ npm install --cpu=x64 --os=darwin sharp
added 13 packages, and audited 14 packages in 1s

$ ls  node_modules/@img
sharp-darwin-x64  sharp-libvips-darwin-x64

$ npm install --cpu=arm64 --os=darwin sharp
added 2 packages, and audited 16 packages in 505ms

$ ls  node_modules/@img
sharp-darwin-arm64  sharp-darwin-x64  sharp-libvips-darwin-arm64  sharp-libvips-darwin-x64

You have probably run into the npm lockfile bug npm/cli#4828 that is linked to in the documentation.

https://sharp.pixelplumbing.com/install#cross-platform

Hmm, seems to be different for 0.32.6 and 0.33.

I don't know too much about the details of where npm stores binaries but that's what I see:

0.32.6 seems to install the binaries in node_modules/vendor
0.33 seems to install the binaries in @img

This is regardless of whether I delete package-lock.json before running npm i ...

I am just a plain user of NextJs next/image and I am not aware of how they are using sharp. Can it be that the changes of where the binaries are stored affects them?

0.32.6:

npm install --cpu=x64 --os=linux --libc=musl sharp

ls -al node_modules/@img
ls: cannot access 'node_modules/@img': No such file or directory

ls -al node_modules/sharp/
total 76
drwxr-xr-x   7 root root  4096 Dec 19 04:51 .
drwxr-xr-x 574 root root 20480 Dec 19 04:53 ..
-rw-r--r--   1 root root  7430 Dec 19 04:51 binding.gyp
drwxr-xr-x   3 root root  4096 Dec 19 04:51 build
drwxr-xr-x   2 root root  4096 Dec 19 04:51 install
drwxr-xr-x   2 root root  4096 Dec 19 04:51 lib
-rw-r--r--   1 root root 10273 Dec 19 04:51 LICENSE
-rw-r--r--   1 root root  7821 Dec 19 04:51 package.json
-rw-r--r--   1 root root  3071 Dec 19 04:51 README.md
drwxr-xr-x   3 root root  4096 Dec 19 04:51 src
drwxr-xr-x   3 root root  4096 Dec 19 04:51 vendor

0.33:

npm install --cpu=x64 --os=linux --libc=musl sharp

ls -al node_modules/@img
total 40
drwxr-xr-x   6 root root  4096 Dec 19 04:56 .
drwxr-xr-x 576 root root 20480 Dec 19 04:56 ..
drwxr-xr-x   3 root root  4096 Dec 19 04:56 sharp-libvips-linuxmusl-x64
drwxr-xr-x   3 root root  4096 Dec 19 04:56 sharp-libvips-linux-x64
drwxr-xr-x   3 root root  4096 Dec 19 04:56 sharp-linuxmusl-x64
drwxr-xr-x   3 root root  4096 Dec 19 04:56 sharp-linux-x64


ls -al node_modules/sharp/
total 60
drwxr-xr-x   5 root root  4096 Dec 19 04:56 .
drwxr-xr-x 576 root root 20480 Dec 19 04:56 ..
drwxr-xr-x   2 root root  4096 Dec 19 04:56 install
drwxr-xr-x   2 root root  4096 Dec 19 04:56 lib
-rw-r--r--   1 root root 10273 Dec 19 04:56 LICENSE
-rw-r--r--   1 root root  7547 Dec 19 04:56 package.json
-rw-r--r--   1 root root  3172 Dec 19 04:56 README.md
drwxr-xr-x   2 root root  4096 Dec 19 04:56 src

[rhufsky]

Found a solution here: https://flinect.com/blog/nextjs-standalone-docker-sharp-installation, it boils down to:

Install sharp globally via npm install -g, point to the sharp installation via NEXT_SHARP_PATH=/usr/local/lib/node_modules/sharp and NextJS seems to pick up sharp even for 0.33.

[lovell]

Hmm, seems to be different for 0.32.6 and 0.33.

This is expected and correct, please see #3750 for all the details.

Install sharp globally

Please avoid installing sharp globally, it will only lead to further problems later. If doing so appears to fix a problem then I suspect you have multiple, conflicting versions of sharp within the same installation tree or you had previously installed sharp globally and were seeing the effects of having done so.

The issue installation template asks the question: What is the output of running npx envinfo --binaries --system --npmPackages=sharp --npmGlobalPackages=sharp? The answer to this may be helpful.

[rhufsky]

I totally agree, installing sharp globally is not best practice in the first place. On the other hand, it is installed in a docker container that runs one NextJS app only, so it might be not that bad.

My guess on the whole thing is:

When deploying a NextJS app on vercel, they provide sharp for the app. No need to include sharp in package.json.
I guess that they are accessing sharp in a way that basically resembles a global installation.

So when creating a docker container that runs on a non-Vercel system, it might be OK to install sharp globally on this container and think of it as an infrastructure component.

[lovell]

@rhufsky If you still require help, the best thing to do is provide a complete, minimal, standalone repo including a Dockerfile that allows someone else to reproduce.

@tinrab The advice in your https://flinect.com/blog/nextjs-standalone-docker-sharp-installation article is somewhat misguided (e.g. installing a release candidate globally is going to end in tears). it looks like you are using Next.js standalone mode and therefore have very likely hit a known Vercel bug - see vercel/nft#371 - I'd very much appreciate it if you could edit your article to reflect this so others are not confused by it.

[lovell]

This was a Vercel bug, now fixed - see vercel/nft#371 and #3870 (comment)