Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

node modules not updated #2964

Closed
th-2021 opened this issue Jun 30, 2024 · 9 comments
Closed

node modules not updated #2964

th-2021 opened this issue Jun 30, 2024 · 9 comments

Comments

@th-2021
Copy link

th-2021 commented Jun 30, 2024

During assets compile I get error that some node modules are missing. This is with all newer versions e.g. 17.0.3

@kkimurak
Copy link
Contributor

@th-2021 Could you provide more detail? Do you mean following error on using relative url enabled (that is said that can be safely ignored)?

docker log of `sameersbn/gitlab:17.0.3`, setting `GITLAB_RELATIVE_URL=/gitlab`
warning Error running install script for optional dependency: "/home/git/gitlab/node_modules/tree-sitter-json: Command failed.
Exit code: 1
Command: node-gyp rebuild
Arguments: 
Directory: /home/git/gitlab/node_modules/tree-sitter-json
Output:
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | x64
gyp info find Python using Python version 3.8.10 found at \"/usr/bin/python3\"
gyp http GET https://nodejs.org/download/release/v20.15.0/node-v20.15.0-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v20.15.0/node-v20.15.0-headers.tar.gz

gyp http GET https://nodejs.org/download/release/v20.15.0/SHASUMS256.txt
gyp http 200 https://nodejs.org/download/release/v20.15.0/SHASUMS256.txt
gyp info spawn /usr/bin/python3
gyp info spawn args [
gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args 'binding.gyp',
gyp info spawn args '-f',
gyp info spawn args 'make',
gyp info spawn args '-I',
gyp info spawn args '/home/git/gitlab/node_modules/tree-sitter-json/build/config.gypi',
gyp info spawn args '-I',
gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args '-I',
gyp info spawn args '/home/git/.cache/node-gyp/20.15.0/include/node/common.gypi',
gyp info spawn args '-Dlibrary=shared_library',
gyp info spawn args '-Dvisibility=default',
gyp info spawn args '-Dnode_root_dir=/home/git/.cache/node-gyp/20.15.0',
gyp info spawn args '-Dnode_gyp_dir=/usr/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args '-Dnode_lib_file=/home/git/.cache/node-gyp/20.15.0/<(target_arch)/node.lib',
gyp info spawn args '-Dmodule_root_dir=/home/git/gitlab/node_modules/tree-sitter-json',
gyp info spawn args '-Dnode_engine=v8',
gyp info spawn args '--depth=.',
gyp info spawn args '--no-parallel',
gyp info spawn args '--generator-output',
gyp info spawn args 'build',
gyp info spawn args '-Goutput_dir=.'
gyp info spawn args ]
gyp ERR! build error 
gyp ERR! stack Error: not found: make
gyp ERR! stack at getNotFoundError (/usr/lib/node_modules/npm/node_modules/which/lib/index.js:16:17)
gyp ERR! stack at which (/usr/lib/node_modules/npm/node_modules/which/lib/index.js:77:9)
gyp ERR! stack at async doWhich (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:112:22)
gyp ERR! stack at async loadConfigGypi (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:77:7)
gyp ERR! stack at async build (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:35:3)
gyp ERR! stack at async run (/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js:81:18)
gyp ERR! System Linux 3.10.0-1160.119.1.el7.x86_64
gyp ERR! command \"/usr/bin/node\" \"/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js\" \"rebuild\"
gyp ERR! cwd /home/git/gitlab/node_modules/tree-sitter-json
gyp ERR! node -v v20.15.0
gyp ERR! node-gyp -v v10.1.0
gyp ERR! not ok"
info This module is OPTIONAL, you can safely ignore this error
info This module is OPTIONAL, you can safely ignore this error
warning Error running install script for optional dependency: "/home/git/gitlab/node_modules/tree-sitter-yaml: Command failed.
Exit code: 1
Command: node-gyp rebuild
Arguments: 
Directory: /home/git/gitlab/node_modules/tree-sitter-yaml
Output:
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | x64
gyp info find Python using Python version 3.8.10 found at \"/usr/bin/python3\"

gyp http GET https://nodejs.org/download/release/v20.15.0/node-v20.15.0-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v20.15.0/node-v20.15.0-headers.tar.gz
gyp http GET https://nodejs.org/download/release/v20.15.0/SHASUMS256.txt
gyp http 200 https://nodejs.org/download/release/v20.15.0/SHASUMS256.txt
gyp info spawn /usr/bin/python3
gyp info spawn args [
gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args 'binding.gyp',
gyp info spawn args '-f',
gyp info spawn args 'make',
gyp info spawn args '-I',
gyp info spawn args '/home/git/gitlab/node_modules/tree-sitter-yaml/build/config.gypi',
gyp info spawn args '-I',
gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args '-I',
gyp info spawn args '/home/git/.cache/node-gyp/20.15.0/include/node/common.gypi',
gyp info spawn args '-Dlibrary=shared_library',
gyp info spawn args '-Dvisibility=default',
gyp info spawn args '-Dnode_root_dir=/home/git/.cache/node-gyp/20.15.0',
gyp info spawn args '-Dnode_gyp_dir=/usr/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args '-Dnode_lib_file=/home/git/.cache/node-gyp/20.15.0/<(target_arch)/node.lib',
gyp info spawn args '-Dmodule_root_dir=/home/git/gitlab/node_modules/tree-sitter-yaml',
gyp info spawn args '-Dnode_engine=v8',
gyp info spawn args '--depth=.',
gyp info spawn args '--no-parallel',
gyp info spawn args '--generator-output',
gyp info spawn args 'build',
gyp info spawn args '-Goutput_dir=.'
gyp info spawn args ]
gyp ERR! build error 
gyp ERR! stack Error: not found: make
gyp ERR! stack at getNotFoundError (/usr/lib/node_modules/npm/node_modules/which/lib/index.js:16:17)
gyp ERR! stack at which (/usr/lib/node_modules/npm/node_modules/which/lib/index.js:77:9)
gyp ERR! stack at async doWhich (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:112:22)
gyp ERR! stack at async loadConfigGypi (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:77:7)
gyp ERR! stack at async build (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:35:3)
gyp ERR! stack at async run (/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js:81:18)
gyp ERR! System Linux 3.10.0-1160.119.1.el7.x86_64
gyp ERR! command \"/usr/bin/node\" \"/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js\" \"rebuild\"
gyp ERR! cwd /home/git/gitlab/node_modules/tree-sitter-yaml
gyp ERR! node -v v20.15.0
gyp ERR! node-gyp -v v10.1.0
gyp ERR! not ok"
warning Error running install script for optional dependency: "/home/git/gitlab/node_modules/tree-sitter: Command failed.
Exit code: 1
Command: prebuild-install || node-gyp rebuild
Arguments: 
Directory: /home/git/gitlab/node_modules/tree-sitter
Output:
prebuild-install warn install No prebuilt binaries found (target=20.15.0 runtime=node arch=x64 libc= platform=linux)
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | x64
gyp info find Python using Python version 3.8.10 found at \"/usr/bin/python3\"

gyp http GET https://nodejs.org/download/release/v20.15.0/node-v20.15.0-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v20.15.0/node-v20.15.0-headers.tar.gz
gyp http GET https://nodejs.org/download/release/v20.15.0/SHASUMS256.txt
gyp http 200 https://nodejs.org/download/release/v20.15.0/SHASUMS256.txt
gyp info spawn /usr/bin/python3
gyp info spawn args [
gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args 'binding.gyp',
gyp info spawn args '-f',
gyp info spawn args 'make',
gyp info spawn args '-I',
gyp info spawn args '/home/git/gitlab/node_modules/tree-sitter/build/config.gypi',
gyp info spawn args '-I',
gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args '-I',
gyp info spawn args '/home/git/.cache/node-gyp/20.15.0/include/node/common.gypi',
gyp info spawn args '-Dlibrary=shared_library',
gyp info spawn args '-Dvisibility=default',
gyp info spawn args '-Dnode_root_dir=/home/git/.cache/node-gyp/20.15.0',
gyp info spawn args '-Dnode_gyp_dir=/usr/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args '-Dnode_lib_file=/home/git/.cache/node-gyp/20.15.0/<(target_arch)/node.lib',
gyp info spawn args '-Dmodule_root_dir=/home/git/gitlab/node_modules/tree-sitter',
gyp info spawn args '-Dnode_engine=v8',
gyp info spawn args '--depth=.',
gyp info spawn args '--no-parallel',
gyp info spawn args '--generator-output',
gyp info spawn args 'build',
gyp info spawn args '-Goutput_dir=.'
gyp info spawn args ]
gyp ERR! build error 
gyp ERR! stack Error: not found: make
gyp ERR! stack at getNotFoundError (/usr/lib/node_modules/npm/node_modules/which/lib/index.js:16:17)
gyp ERR! stack at which (/usr/lib/node_modules/npm/node_modules/which/lib/index.js:77:9)
gyp ERR! stack at async doWhich (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:112:22)
gyp ERR! stack at async loadConfigGypi (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:77:7)
gyp ERR! stack at async build (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:35:3)
gyp ERR! stack at async run (/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js:81:18)
gyp ERR! System Linux 3.10.0-1160.119.1.el7.x86_64
gyp ERR! command \"/usr/bin/node\" \"/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js\" \"rebuild\"
gyp ERR! cwd /home/git/gitlab/node_modules/tree-sitter
gyp ERR! node -v v20.15.0
gyp ERR! node-gyp -v v10.1.0
gyp ERR! not ok"
info This module is OPTIONAL, you can safely ignore this error

@th-2021
Copy link
Author

th-2021 commented Jun 30, 2024

With relative url set.
When I run the image I get make and g++ not found.

With make and g++ added I get:

  • echo 'Prepare recompile assets... Installing missing node_modules for assets'
    Prepare recompile assets... Installing missing node_modules for assets
  • chown -R git: /home/git/gitlab/node_modules
  • exec_as_git yarn install --production --pure-lockfile
    ++ whoami
  • [[ root == git ]]
  • sudo -HEu git yarn install --production --pure-lockfile
    2024-06-30 10:49:54,933 INFO Included extra file "/etc/supervisor/conf.d/cron.conf" during parsing
    2024-06-30 10:49:54,933 INFO Included extra file "/etc/supervisor/conf.d/gitaly.conf" during parsing
    2024-06-30 10:49:54,933 INFO Included extra file "/etc/supervisor/conf.d/gitlab-pages.conf" during parsing
    2024-06-30 10:49:54,933 INFO Included extra file "/etc/supervisor/conf.d/gitlab-workhorse.conf" during parsing
    2024-06-30 10:49:54,933 INFO Included extra file "/etc/supervisor/conf.d/groups.conf" during parsing
    2024-06-30 10:49:54,933 INFO Included extra file "/etc/supervisor/conf.d/mail_room.conf" during parsing
    2024-06-30 10:49:54,933 INFO Included extra file "/etc/supervisor/conf.d/nginx.conf" during parsing
    2024-06-30 10:49:54,933 INFO Included extra file "/etc/supervisor/conf.d/puma.conf" during parsing
    2024-06-30 10:49:54,933 INFO Included extra file "/etc/supervisor/conf.d/sidekiq.conf" during parsing
    2024-06-30 10:49:54,933 INFO Included extra file "/etc/supervisor/conf.d/sshd.conf" during parsing
    2024-06-30 10:49:54,933 INFO Set uid to user 0 succeeded
    2024-06-30 10:49:54,936 INFO RPC interface 'supervisor' initialized
    2024-06-30 10:49:54,936 INFO supervisord started with pid 3009
    yarn install v1.22.22
    $ node ./scripts/frontend/preinstall.mjs
    [WARNING] package.json changed significantly. Removing node_modules to be sure there are no problems.
    [1/5] Validating package.json...
    [2/5] Resolving packages...
    [3/5] Fetching packages...
    [4/5] Linking dependencies...
    warning "@gitlab/application-sdk-browser > @snowplow/[email protected]" has incorrect peer dependency "@snowplow/browser-tracker@~3.13.1".
    warning " > @gitlab/[email protected]" has incorrect peer dependency "vue@^2.7.16".
    warning " > @vue/[email protected]" has incorrect peer dependency "vue@^3.1.0".
    warning " > @vue/[email protected]" has incorrect peer dependency "vue@^3.1.0".
    warning " > [email protected]" has incorrect peer dependency "vue@^3.2.0".
    warning " > [email protected]" has incorrect peer dependency "vue@^3.0.2".
    warning "@gitlab/eslint-plugin > eslint-plugin-jest > @typescript-eslint/utils > @typescript-eslint/typescript-estree > [email protected]" has unmet peer dependency "typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta".
    warning " > @vue/[email protected]" has incorrect peer dependency "[email protected]".
    warning " > @vue/[email protected]" has unmet peer dependency "@vue/compiler-dom@^3.0.1".
    warning " > @vue/[email protected]" has unmet peer dependency "@vue/server-renderer@^3.0.1".
    warning " > @vue/[email protected]" has incorrect peer dependency "vue@^3.0.1".
    warning "@vue/test-utils-vue3 > @vue/[email protected]" has incorrect peer dependency "[email protected]".
    warning " > @vue/[email protected]" has incorrect peer dependency "[email protected]".
    warning " > @vue/[email protected]" has incorrect peer dependency "[email protected]".
    warning " > @vue/[email protected]" has incorrect peer dependency "vue@^3.0.0-0".
    warning "swagger-cli > @apidevtools/swagger-cli > @apidevtools/[email protected]" has unmet peer dependency "openapi-types@>=7".
    warning Workspaces can only be enabled in private projects.
    [5/5] Building fresh packages...
    $ node ./scripts/frontend/postinstall.js
    success Dependency postinstall check passed.
    patch-package 6.5.1
    Applying patches...
    @gitlab/[email protected]
    @rails/[email protected]
    Skipping dev-only @vue/[email protected]
    Skipping dev-only @vue/[email protected]
    Skipping dev-only @vue/[email protected]
    Skipping dev-only @vue/[email protected]
    Skipping dev-only @vue/[email protected]
    [email protected]
    vue/@vue/[email protected]
    [email protected]
    Skipping dev-only [email protected]
    Done in 60.96s.

@kkimurak
Copy link
Contributor

Well, gcc and make are listed in BUILD_DEPENDENCIES in build script. They are purged at the end of the script.

We have some choise:

  • Install make and gcc on runtime only when GITLAB_RELATIVE_URL_ROOT is set
  • Do not uninstall make and gcc at the end of build script

The second idea would be acceptable.
When I was working on preparing postgresl-13 (#2685), the initial idea for that pull request was to install the appropriate version of postgresql-client at runtime. After receiving a review from the maintainer @sachilles saying that he wanted to avoid requiring an internet connection, I rewrote the patch to pre-install a possible version of postgresql-client in the image. It has been merged.

This may be pointless since we (relative url users) install the necessary node modules at runtime anyway, but I'll write this as advice anyway. I means that I have no plans to create a pull request on this subject at this time.

@th-2021
Copy link
Author

th-2021 commented Jul 2, 2024

I tried the upgrades again and the error can be ignored. With one version I had broken assets and thought that this is the issue, but it must have been something different. It worked now.

@th-2021 th-2021 closed this as completed Jul 2, 2024
@shplishka
Copy link

Is there any version where this doesn't happen?

@kkimurak
Copy link
Contributor

@shplishka No. More precisely, the root cause of this problem (a situation where gcc and make are required at runtime) is not known. According to @th-2021 's report, it may be caused by corrupted assets but not clear.

I've been running gitlab with the relative url feature enabled in all versions since at least 13.10.3 until 17.2.0 (latest version available today), and no such situation occurred on my environment.

If you are facing similar issue, providing your detailed setup may help someone who interested in this topic to debug.

@th-2021
Copy link
Author

th-2021 commented Jul 24, 2024 via email

@shplishka
Copy link

Hi @th-2021 and @kkimurak,

Thank you for your previous assistance. I am facing another problem related to this behavior. When I use a relative URL in an air-gapped cluster that doesn't have access to npm, Artifactory, etc., the image crashes when it tries to install the node modules.

I'm not sure how to resolve this issue. Any advice would be appreciated.

@kkimurak
Copy link
Contributor

@shplishka
Unfortunately, the relative URL feature should not work without an active internet connection - as this image, sameersbn/gitlab, is published with node_modules removed at the end of build to reduce image size.

Anyway, could you create a new issue about it (relative url does not work if there is no access to npm registry and so on) ?
I think it's not a good idea to continue discussing on an issue that has already been closed, and I think it's different from what was originally reported for this issue.

Submitting a merge request is much better. You should just modify the line.

diff --git a/assets/build/install.sh b/assets/build/install.sh
index 47fc4620..fb330b23 100755
--- a/assets/build/install.sh
+++ b/assets/build/install.sh
@@ -484,5 +484,5 @@ rm -rf /var/lib/apt/lists/*
 
 # clean up caches
 rm -rf ${GITLAB_HOME}/.cache ${GITLAB_HOME}/.bundle ${GITLAB_HOME}/go
-rm -rf /root/.cache /root/.bundle ${GITLAB_HOME}/gitlab/node_modules
+rm -rf /root/.cache /root/.bundle
 rm -r /tmp/*

I can report how much the image size changes.

In any case, the maintainer will make the final decision. You can mention to active maintainer (@sachilles) on demand.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants