Intermittent npm failures on github workflow docker (TAR_ENTRY_ERROR ENOENT)

[slobberchops]

We have been experiencing intermittent failures when trying to use this packages installation of pyright for a while (across all versions) and wanted to check to see if other people have also experienced this issue and how to address it. I have experienced this problem in multiple repositories in multiple github organizations.

I have searched around and the best that I can tell is that this sort of error happens when running inside a docker container, often on github. I've attempted quite a few remedies including some suggested on other reports around similar issues, but so far have not found any solutions.

Our current work around is to restart any failed jobs manually. This over time has been causing delays in much of our work, particularly for longer jobs that depend on pyright.

Has anyone encountered similar problems and understand how to go about addressing it?

---

npm ERR! code ENOTEMPTY
npm ERR! syscall rmdir
npm ERR! path /home/runner/.cache/pyright-python/1.1.307/node_modules/pyright/dist/typeshed-fallback/stdlib/unittest
npm ERR! errno -39
npm ERR! ENOTEMPTY: directory not empty, rmdir '/home/runner/.cache/pyright-python/1.1.307/node_modules/pyright/dist/typeshed-fallback/stdlib/unittest'

npm ERR! A complete log of this run can be found in: /home/runner/.npm/_logs/2023-08-04T12_00_43_386Z-debug-0.log
WARNING: there is a new pyright version available (v1.1.307 -> v1.1.320).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to latest

Traceback (most recent call last):
File "/home/runner/.cache/pypoetry/virtualenvs/trajector-cloud-api-common-HO10pT6J-py3.9/bin/pyright", line 8, in
sys.exit(entrypoint())
File "/home/runner/.cache/pypoetry/virtualenvs/trajector-cloud-api-common-HO10pT6J-py3.9/lib/python3.9/site-packages/pyright/cli.py", line 34, in entrypoint
sys.exit(main(sys.argv[1:]))
File "/home/runner/.cache/pypoetry/virtualenvs/trajector-cloud-api-common-HO10pT6J-py3.9/lib/python3.9/site-packages/pyright/cli.py", line 19, in main
return run(*args, **kwargs).returncode
File "/home/runner/.cache/pypoetry/virtualenvs/trajector-cloud-api-common-HO10pT6J-py3.9/lib/python3.9/site-packages/pyright/cli.py", line 25, in run
pkg_dir = install_pyright(args, quiet=None)
File "/home/runner/.cache/pypoetry/virtualenvs/trajector-cloud-api-common-HO10pT6J-py3.9/lib/python3.9/site-packages/pyright/_utils.py", line 61, in install_pyright
node.run(
File "/home/runner/.cache/pypoetry/virtualenvs/trajector-cloud-api-common-HO10pT6J-py3.9/lib/python3.9/site-packages/pyright/node.py", line 115, in run
return subprocess.run(node_args, env=env, **kwargs)
File "/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/usr/local/bin/npm', 'install', '[email protected]']' returned non-zero exit status 217.
npm WARN tar TAR_ENTRY_ERROR ENOENT: no such file or directory, open '/home/runner/.cache/pyright-python/1.1.307/node_modules/pyright/dist/typeshed-fallback/stdlib/collections/abc.pyi'
npm WARN tar TAR_ENTRY_ERROR ENOENT: no such file or directory, open '/home/runner/.cache/pyright-python/1.1.307/node_modules/pyright/dist/typeshed-fallback/stdlib/curses/ascii.pyi'

... continues in this manner for some time ...

es.pyi'
npm WARN tar TAR_ENTRY_ERROR ENOENT: no such file or directory, open '/home/runner/.cache/pyright-python/1.1.307/node_modules/pyright/dist/typeshed-fallback/stdlib/_typeshed/wsgi.pyi'
npm WARN tar TAR_ENTRY_ERROR ENOENT: no such file or directory, open '/home/runner/.cache/pyright-python/1.1.307/node_modules/pyright/dist/typeshed-fallback/stdlib/_typeshed/xml.pyi'

added 1 package, and audited 2 packages in 2s

found 0 vulnerabilities
WARNING: there is a new pyright version available (v1.1.307 -> v1.1.320).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to latest

Traceback (most recent call last):
File "/home/runner/.cache/pypoetry/virtualenvs/trajector-cloud-api-common-HO10pT6J-py3.9/bin/pyright", line 8, in
sys.exit(entrypoint())
File "/home/runner/.cache/pypoetry/virtualenvs/trajector-cloud-api-common-HO10pT6J-py3.9/lib/python3.9/site-packages/pyright/cli.py", line 34, in entrypoint
sys.exit(main(sys.argv[1:]))
File "/home/runner/.cache/pypoetry/virtualenvs/trajector-cloud-api-common-HO10pT6J-py3.9/lib/python3.9/site-packages/pyright/cli.py", line 19, in main
return run(*args, **kwargs).returncode
File "/home/runner/.cache/pypoetry/virtualenvs/trajector-cloud-api-common-HO10pT6J-py3.9/lib/python3.9/site-packages/pyright/cli.py", line 28, in run
raise RuntimeError(f'Expected CLI entrypoint: {script} to exist')
RuntimeError: Expected CLI entrypoint: /home/runner/.cache/pyright-python/1.1.307/node_modules/pyright/index.js to exist

[sebpretzer]

I was dealing with a similar issue that might be related:

The first time you call pyright-python from your python env, it installs pyright in ~/.cache/pyright-python/[VERSION]/node_modules/pyright/. I was running multiple processes in parallel, so the first process would try to install, then the others would error out with a similar error:

poetry run pyright
Traceback (most recent call last):
  File "~/.venv/bin/pyright", line 8, in <module>
    sys.exit(entrypoint())
  File "~/.venv/lib/python3.8/site-packages/pyright/cli.py", line 34, in entrypoint
    sys.exit(main(sys.argv[1:]))
  File "~/.venv/lib/python3.8/site-packages/pyright/cli.py", line 19, in main
    return run(*args, **kwargs).returncode
  File "~/.venv/lib/python3.8/site-packages/pyright/cli.py", line 28, in run
    raise RuntimeError(f'Expected CLI entrypoint: {script} to exist')
RuntimeError: Expected CLI entrypoint: ~/.cache/pyright-python/1.1.335/node_modules/pyright/index.js to exist

Subsequent attempts to call pyright-python led the code to think pyright was installed correctly, when it actually wasn't.

The way I resolved this was by:

1. Purging both the .venv/ and the .cache/pyright-python/
2. Reinstalling pyright-python
3. Calling pyright-python once
4. Waiting for pyright-python to properly install pyright
5. Then proceeding with more calls to pyright-python

Let me know if this helps!

[WSH032]

@sebpretzer You are right! Thanks!

Refer to: https://stackabuse.com/bytes/fixing-npm-err-code-enotempty-on-npm-install/
It seems that concurrent installations caused this issue. For me, it seems that npm install was called multithreaded when first used in pre-commit.

My solution is:
Before using pre-commit (or other uses involving multithreading), call pyright --help once before to complete the initial installation.

[copdips]

@sebpretzer You are right! Thanks!

Refer to: https://stackabuse.com/bytes/fixing-npm-err-code-enotempty-on-npm-install/ It seems that concurrent installations caused this issue. For me, it seems that npm install was called multithreaded when first used in pre-commit.

My solution is: Before using pre-commit (or other uses involving multithreading), call pyright --help once before to complete the initial installation.

I cannot confirm this works as the issue was very aleatory, but I can see that since the application of this tips, i haven't seen the error again.

[slobberchops]

I was dealing with a similar issue that might be related:

The first time you call pyright-python from your python env, it installs pyright in ~/.cache/pyright-python/[VERSION]/node_modules/pyright/. I was running multiple processes in parallel, so the first process would try to install, then the others would error out with a similar error:

poetry run pyright
Traceback (most recent call last):
  File "~/.venv/bin/pyright", line 8, in <module>
    sys.exit(entrypoint())
  File "~/.venv/lib/python3.8/site-packages/pyright/cli.py", line 34, in entrypoint
    sys.exit(main(sys.argv[1:]))
  File "~/.venv/lib/python3.8/site-packages/pyright/cli.py", line 19, in main
    return run(*args, **kwargs).returncode
  File "~/.venv/lib/python3.8/site-packages/pyright/cli.py", line 28, in run
    raise RuntimeError(f'Expected CLI entrypoint: {script} to exist')
RuntimeError: Expected CLI entrypoint: ~/.cache/pyright-python/1.1.335/node_modules/pyright/index.js to exist

Subsequent attempts to call pyright-python led the code to think pyright was installed correctly, when it actually wasn't.

The way I resolved this was by:

1. Purging both the .venv/ and the .cache/pyright-python/
2. Reinstalling pyright-python
3. Calling pyright-python once
4. Waiting for pyright-python to properly install pyright
5. Then proceeding with more calls to pyright-python

Let me know if this helps!

We did come to a rather similar conclusion. We have seemingly permanently remedied this problem by repeatedly calling pyright --help > /dev/null (as described by @WSH032 ) until it runs (or max retries) immediately after installation.

I should have circled around and closed this issue.