Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doesn't work in the node:20-alpine image docker #402

Open
3 tasks done
MrOxMasTer opened this issue Apr 16, 2024 · 25 comments
Open
3 tasks done

Doesn't work in the node:20-alpine image docker #402

MrOxMasTer opened this issue Apr 16, 2024 · 25 comments

Comments

@MrOxMasTer
Copy link

Welcome to the issues section if it's your first time!

Before creating an issue, please be sure to:

  • Checkout to the latest version, including submodules
  • Try to find an isolated way to reproduce the behavior
  • Fill in all the blanks in the most specific way you can

Steps to reproduce

  1. Install the argon2 package on the node:20-alpine image
  2. try to use it
  3. Receive an error

image

It says from 3.18 the binary files are there and on later versions should work. No, it doesn't work on 3.19

image
tried to compile and it still doesn't work

Expected behaviour

work

Actual behaviour

Causes an error

Environment

Operating system:
Windows 11

Node version:
node:20.12.2

@MrOxMasTer
Copy link
Author

@MrOxMasTer
Copy link
Author

alpine3.18-3.19 don't work

@ranisalt
Copy link
Owner

Can you share a minimal Dockerfile + package.json that triggers this issue?

@MrOxMasTer
Copy link
Author

Can you share a minimal Dockerfile + package.json that triggers this issue?

{
  "name": "shopco",
  "version": "0.1.0",
  "private": true,
  "type": "module",
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint",
    "migration:generate": "drizzle-kit generate:pg",
    "migration:push": "drizzle-kit push:pg",
    "studio": "drizzle-kit studio --host 127.0.0.1",
    "build:icons": "npx tsx ./build-icons.mts",
    "sly": "pnpm dlx @sly-cli/sly add"
  },
  "dependencies": {
    "@auth/drizzle-adapter": "^0.8.0",
    "@hookform/resolvers": "^3.3.4",
    "@radix-ui/react-dialog": "^1.0.5",
    "@radix-ui/react-slot": "^1.0.2",
    "@t3-oss/env-nextjs": "^0.9.2",
    "argon2": "^0.40.1",
    "class-variance-authority": "^0.7.0",
    "clsx": "^2.1.0",
    "drizzle-orm": "^0.30.2",
    "drizzle-zod": "^0.5.1",
    "embla-carousel-autoplay": "8.0.0-rc17",
    "embla-carousel-react": "8.0.0-rc17",
    "ky": "^1.2.2",
    "nanoid": "^5.0.6",
    "next": "14.2.1",
    "next-auth": "5.0.0-beta.16",
    "next-themes": "^0.3.0",
    "pg": "^8.11.3",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "react-focus-lock": "^2.11.2",
    "react-hook-form": "^7.51.2",
    "redis": "^4.6.13",
    "server-only": "^0.0.1",
    "sharp": "^0.33.2",
    "sonner": "^1.4.3",
    "tailwind-merge": "^2.2.1",
    "tailwindcss-animate": "^1.0.7",
    "zod": "^3.22.4",
    "zustand": "^4.5.2"
  },
  "devDependencies": {
    "@conarti/eslint-plugin-feature-sliced": "^1.0.5",
    "@sly-cli/sly": "^1.10.0",
    "@types/fs-extra": "^11.0.4",
    "@types/node": "^20.11.28",
    "@types/pg": "^8.11.2",
    "@types/react": "^18.2.66",
    "@types/react-dom": "^18.2.22",
    "@typescript-eslint/eslint-plugin": "^6.21.0",
    "@typescript-eslint/parser": "^6.21.0",
    "autoprefixer": "^10.4.18",
    "cssnano": "^6.1.0",
    "drizzle-kit": "^0.20.14",
    "eslint": "^8.57.0",
    "eslint-config-next": "14.1.3",
    "eslint-config-prettier": "^9.1.0",
    "eslint-plugin-drizzle": "^0.2.3",
    "eslint-plugin-import": "^2.29.1",
    "eslint-plugin-promise": "^6.1.1",
    "eslint-plugin-tailwindcss": "^3.14.3",
    "fs-extra": "^11.2.0",
    "glob": "^10.3.10",
    "jiti": "^1.21.0",
    "lucide-react": "^0.303.0",
    "node-html-parser": "^6.1.12",
    "postcss": "^8.4.35",
    "postcss-flexbugs-fixes": "^5.0.2",
    "postcss-preset-env": "^9.4.0",
    "prettier": "^3.2.5",
    "prettier-plugin-tailwindcss": "^0.5.12",
    "react-devtools": "^5.0.2",
    "stylelint": "^16.3.0",
    "stylelint-config-standard": "^36.0.0",
    "stylelint-order": "^6.0.4",
    "stylelint-selector-bem-pattern": "^4.0.0",
    "svgo": "^3.2.0",
    "tailwindcss": "^3.4.1",
    "tailwindcss-debug-screens": "^2.2.1",
    "typescript": "^5.4.2"
  }
}
FROM node:20-alpine3.18 AS base

# Install dependencies only when needed
FROM base AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
WORKDIR /app

# Install dependencies based on the preferred package manager
COPY package.json pnpm-lock.yaml* yarn.lock* package-lock.json* ./
RUN \
  if [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm i --frozen-lockfile; \
  elif [ -f yarn.lock ]; then yarn --frozen-lockfile; \
  elif [ -f package-lock.json ]; then npm ci; \
  else echo "Lockfile not found." && exit 1; \
  fi

# RUN apk add build-base
# RUN gcc -v
RUN npm i -g node-gyp
RUN pnpm i argon2

# Rebuild the source code only when needed
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .

# Next.js collects completely anonymous telemetry data about general usage.
# Learn more here: https://nextjs.org/telemetry
# Uncomment the following line in case you want to disable telemetry during the build.
# ENV NEXT_TELEMETRY_DISABLED 1

RUN \
  if [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm run build; \
  elif [ -f yarn.lock ]; then yarn run build; \
  elif [ -f package-lock.json ]; then npm run build; \
  else echo "Lockfile not found." && exit 1; \
  fi

# Production image, copy all the files and run next
FROM base AS runner
WORKDIR /app

ENV NODE_ENV production
# Uncomment the following line in case you want to disable telemetry during runtime.
# ENV NEXT_TELEMETRY_DISABLED 1

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder /app/public ./public

# Set the correct permission for prerender cache
RUN mkdir .next
RUN chown nextjs:nodejs .next

# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

USER nextjs

EXPOSE 3000

ENV PORT 3000

# server.js is created by next build from the standalone output
# https://nextjs.org/docs/pages/api-reference/next-config-js/output
CMD HOSTNAME="0.0.0.0" node server.js

@MrOxMasTer
Copy link
Author

@ranisalt
Copy link
Owner

You seem to be using Next.js standalone mode, which notably fails to find anything that's not explicitly required when building - since the argon2 module needs to dynamically choose the binary, it fails to get picked.

You likely need to use outputFileTracingIncludes in one of your endpoints to include ./node_modules/argon2/** or just copy the entire node_modules/argon2 folder to the runner image

@MrOxMasTer
Copy link
Author

MrOxMasTer commented Apr 17, 2024

Owner

Tried to add for all pages and this is what came out. I can't enable for any specific point because argon is used in middleware for authorisation

experimental: {
    outputFileTracingIncludes: {
      '/': ['./node_modules/argon2/**'],
    },
  }
frontend  |  ⨯ Error: Cannot find module '@phc/format'
frontend  | Require stack:
frontend  | - /app/node_modules/argon2/argon2.cjs
frontend  | - /app/.next/server/app/(main)/auth/page.js
frontend  | - /app/node_modules/.pnpm/[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/require.js
frontend  | - /app/node_modules/.pnpm/[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/next-server.js
frontend  | - /app/node_modules/.pnpm/[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/next.js
frontend  | - /app/server.js
frontend  |     at Module._resolveFilename (node:internal/modules/cjs/loader:1143:15)
frontend  |     at /app/node_modules/.pnpm/[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/require-hook.js:55:36
frontend  |     at Module._load (node:internal/modules/cjs/loader:984:27)
frontend  |     at Module.require (node:internal/modules/cjs/loader:1231:19)
frontend  |     at mod.require (/app/node_modules/.pnpm/[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/require-hook.js:65:28)
frontend  |     at require (node:internal/modules/helpers:179:18)
frontend  |     at Object.<anonymous> (/app/node_modules/argon2/argon2.cjs:4:36)
frontend  |     at Module._compile (node:internal/modules/cjs/loader:1369:14)
frontend  |     at Module._extensions..js (node:internal/modules/cjs/loader:1427:10)
frontend  |     at Module.load (node:internal/modules/cjs/loader:1206:32) {
frontend  |   code: 'MODULE_NOT_FOUND',
frontend  |   requireStack: [
frontend  |     '/app/node_modules/argon2/argon2.cjs',
frontend  |     '/app/.next/server/app/(main)/auth/page.js',
frontend  |     '/app/node_modules/.pnpm/[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/require.js',
frontend  |     '/app/node_modules/.pnpm/[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/next-server.js',
frontend  |     '/app/node_modules/.pnpm/[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/next.js',
frontend  |     '/app/server.js'
frontend  |   ]
frontend  | }

I'm gonna try to copy the folder

@MrOxMasTer
Copy link
Author

MrOxMasTer commented Apr 17, 2024

Could this problem be due to the use of pnpm? It's just that it creates symbolic links and it's weird to copy a folder into the runner
image

@MrOxMasTer
Copy link
Author

MrOxMasTer commented Apr 17, 2024

maybe make some "argon2.config.js" that describes the binaries that will be included in the final build? Or create your own argon2 instance with the configuration in it. In my opinion this would eliminate the problem that binaries may not make it into the final build and thus not load unnecessary binaries that are not needed by a particular distribution (e.g. alpine does not need a binary for ubuntu).

@MrOxMasTer
Copy link
Author

MrOxMasTer commented Apr 17, 2024

Do I understand correctly that these are the binaries that should be included in the final build? If so, they kind of are

image

And with it switched off outputFileTracingIncludes

image

@ranisalt
Copy link
Owner

How are you using it in middleware?

@MrOxMasTer
Copy link
Author

How are you using it in middleware?

The amazing thing is that it doesn't, but it causes an error.

image
image

In the instance that is used in middleware - not used, but it is used in another NextAuth instance in the authentication function

image

@MrOxMasTer
Copy link
Author

MrOxMasTer commented Apr 23, 2024

How are you using it in middleware?

which is on par with middleware

image

@ranisalt
Copy link
Owner

I don't get the same failure with this minimal reproduction: https://github.com/ranisalt/argon2-nextjs-standalone

So I would start from there. Not sure if it makes a difference, but I used '/login' instead of '/'

Tip: the node Docker images already set up a non-root user, called node, with uid 1000.

not load unnecessary binaries that are not needed by a particular distribution

You can control this using environment variable, like this:

...
    outputFileTracingIncludes: {
      "/login": [`./node_modules/argon2/prebuilds/${process.env.ARGON2_PREBUILDS_GLOB || '**'}`],
    },
...

And in your Dockerfile, specify

ENV ARGON2_PREBUILDS_GLOB="linux-x64/*.musl.*```

Or some other variation of the same idea

@MrOxMasTer
Copy link
Author

MrOxMasTer commented Apr 23, 2024

I don't get the same failure with this minimal reproduction: https://github.com/ranisalt/argon2-nextjs-standalone

So I would start from there. Not sure if it makes a difference, but I used '/login' instead of '/'

Tip: the node Docker images already set up a non-root user, called node, with uid 1000.

Well, first of all, I have middleware working for many pages, so it looks strange to set 1 page at a time.

And secondly it won't fix the problem. I showed you standalone build before and next.js does not remove these files, so the answer should be found elsewhere

image

Docker Desktop:

image

@ranisalt
Copy link
Owner

Please raise a bug in https://github.com/vercel/next.js so we can keep track together and I can make the changes needed if the issue is with this library

@MrOxMasTer
Copy link
Author

Please raise a bug in https://github.com/vercel/next.js so we can keep track together and I can make the changes needed if the issue is with this library

Okay, thanks, I'll create it, but a little later. I have such an assumption. It can't be because of pnp that it somehow optimizes library imports, so is that such a thing? I'll try to test it later on a regular npm

@MrOxMasTer
Copy link
Author

I'm sorry that I haven't created issues yet, but I didn't have time.
Does it relate to this topic in any way?

vercel/next.js#65204

As I understand it, it says about packages that cannot be combined and it says about argon2

@MrOxMasTer
Copy link
Author

MrOxMasTer commented May 1, 2024

I decided to try switching to npm normal temporarily and see if anything happens.

Two strange situations have arisen.

  1. Next.js flat out refuses to compile pages that have dynamic data retrieval from the database.

I for a long time did not understand why I have on npm basic pages (for example: ‘/’) just did not compile and tried to understand, maybe I did something wrong, but it turned out in the matter in the server components in which I call the database.

image

I don't understand why next.js even cares at the compilation stage whether it is connected to postgresQL or not.

  1. Another situation is interesting. I remove these dynamically parts for a while and I have missing errors related to argon2 and their imports.
    Maybe it's really about pnpm and the fact that it breaks the imports in the library

image

FROM node:20-alpine3.18 AS base

# Install dependencies only when needed
FROM base AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
WORKDIR /app

# Install dependencies based on the preferred package manager
COPY package.json pnpm-lock.yaml* yarn.lock* package-lock.json* ./
RUN \
 # if [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm i --frozen-lockfile; \
  if [ -f package-lock.json ]; then npm ci; \
  elif [ -f yarn.lock ]; then yarn --frozen-lockfile; \
  else echo "Lockfile not found." && exit 1; \
  fi

# RUN apk add build-base
# RUN gcc -v
RUN npm i -g node-gyp
RUN npm i argon2

# Rebuild the source code only when needed
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .

# Next.js collects completely anonymous telemetry data about general usage.
# Learn more here: https://nextjs.org/telemetry
# Uncomment the following line in case you want to disable telemetry during the build.
# ENV NEXT_TELEMETRY_DISABLED 1
ENV ARGON2_PREBUILDS_GLOB="linux-x64/*.musl.*"

# RUN \
#   if [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm run build; \
#   elif [ -f yarn.lock ]; then yarn run build; \
#   elif [ -f package-lock.json ]; then npm run build; \
#   else echo "Lockfile not found." && exit 1; \
#   fi

RUN \
  if [ -f package-lock.json ]; then npm run build; \
#  elif [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm run build; \
  elif [ -f yarn.lock ]; then yarn run build; \
  else echo "Lockfile not found." && exit 1; \
  fi

# Production image, copy all the files and run next
FROM base AS runner
WORKDIR /app

ENV NODE_ENV production
# Uncomment the following line in case you want to disable telemetry during runtime.
# ENV NEXT_TELEMETRY_DISABLED 1

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder /app/public ./public

# Set the correct permission for prerender cache
RUN mkdir .next
RUN chown nextjs:nodejs .next

# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

USER nextjs

EXPOSE 3000

ENV PORT 3000

# server.js is created by next build from the standalone output
# https://nextjs.org/docs/pages/api-reference/next-config-js/output
CMD HOSTNAME="0.0.0.0" node server.js

@MrOxMasTer
Copy link
Author

MrOxMasTer commented May 1, 2024

I switched back to ppm and in the absence of these dynamic parts, everything works for me

image

FROM node:20-alpine3.18 AS base

# Install dependencies only when needed
FROM base AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
WORKDIR /app

# Install dependencies based on the preferred package manager
COPY package.json pnpm-lock.yaml* yarn.lock* package-lock.json* ./
RUN \
 if [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm i --frozen-lockfile; \
  # if [ -f package-lock.json ]; then npm ci; \
  elif [ -f yarn.lock ]; then yarn --frozen-lockfile; \
  else echo "Lockfile not found." && exit 1; \
  fi

# RUN apk add build-base
# RUN gcc -v
RUN npm i -g node-gyp
RUN pnpm i argon2

# Rebuild the source code only when needed
FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .

# Next.js collects completely anonymous telemetry data about general usage.
# Learn more here: https://nextjs.org/telemetry
# Uncomment the following line in case you want to disable telemetry during the build.
# ENV NEXT_TELEMETRY_DISABLED 1
ENV ARGON2_PREBUILDS_GLOB="linux-x64/*.musl.*"

# RUN \
#   if [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm run build; \
#   elif [ -f yarn.lock ]; then yarn run build; \
#   elif [ -f package-lock.json ]; then npm run build; \
#   else echo "Lockfile not found." && exit 1; \
#   fi

RUN \
  # if [ -f package-lock.json ]; then npm run build; \
  if [ -f pnpm-lock.yaml ]; then corepack enable pnpm && pnpm run build; \
  elif [ -f yarn.lock ]; then yarn run build; \
  else echo "Lockfile not found." && exit 1; \
  fi

# Production image, copy all the files and run next
FROM base AS runner
WORKDIR /app

ENV NODE_ENV production
# Uncomment the following line in case you want to disable telemetry during runtime.
# ENV NEXT_TELEMETRY_DISABLED 1

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder /app/public ./public

# Set the correct permission for prerender cache
RUN mkdir .next
RUN chown nextjs:nodejs .next

# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

USER nextjs

EXPOSE 3000

ENV PORT 3000

# server.js is created by next build from the standalone output
# https://nextjs.org/docs/pages/api-reference/next-config-js/output
CMD HOSTNAME="0.0.0.0" node server.js

@MrOxMasTer
Copy link
Author

I tried using pnpm and it works.
Now I connected to the local database and it works and does not show any errors.
I don't understand what happened. I know I updated next.js from 14.2.1 to 14.2.3(which I have now)

And I don't remember if I updated pnpm, but if you look at the latest screenshots inside the docker container, the build has changed!

Before:

image

After:
image
image

@MrOxMasTer
Copy link
Author

MrOxMasTer commented May 1, 2024

You were right, I unchecked the setting and it's showing me again that I don't have any binary files. You were right when you said next.js doesn't put those files in.

image
image

@MrOxMasTer
Copy link
Author

I added:

    outputFileTracingIncludes: {
      '/auth': [
        `./node_modules/argon2/prebuilds/${process.env.ARGON2_PREBUILDS_GLOB || '**'}`,
      ],
    },

And it worked

@MrOxMasTer
Copy link
Author

I tried adding ‘/’ for the page

    outputFileTracingIncludes: {
      '/': [
        `./node_modules/argon2/prebuilds/${process.env.ARGON2_PREBUILDS_GLOB || '**'}`,
      ],
    }

it doesn't work:
image

And the prebuilds folder is not there with the binary files:
image

But in the node_modules folder itself, an argon2 folder appears with all these binary files, but they are not in next.js itself
image

And when you add for "/auth" (as I showed above), it works. This is really weird.

@gligorot
Copy link

Similar issue as OP but with practical reproduction steps in #413

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants