Plugin cache prevents vscode extension install

[tsmaeder]

I got into a state where I cannot install the gitlens extension. Here's what I do:

1. I go to the plugins view and search for "gitlens". The exension appears in the results
2. I press the "install" button on the search result
3. Nothing happens: the extension does not activate. Restarting does not help either

[tsmaeder]

Here's what happens in the console log:

2023-07-26T09:32:17.082Z root INFO [eamodio.gitlens]: trying to resolve latest version...
2023-07-26T09:32:17.499Z root INFO [eamodio.gitlens]: resolved to 'eamodio.gitlens-14.0.1'
2023-07-26T09:32:17.500Z root INFO [eamodio.gitlens-14.0.1]: trying to download from "https://open-vsx.org/api/eamodio/gitlens/14.0.1/file/eamodio.gitlens-14.0.1.vsix"... to path C:\Users\thomas\.theia\extensions
2023-07-26T09:32:19.490Z root INFO [eamodio.gitlens-14.0.1]: downloaded to C:\Users\thomas\.theia\extensions\eamodio.gitlens-14.0.1.vsix"
2023-07-26T09:32:19.491Z root INFO [eamodio.gitlens-14.0.1]: trying to decompress into "c:\Users\thomas\AppData\Local\Temp\vscode-unpacked\eamodio.gitlens-14.0.1"...
2023-07-26T09:32:19.491Z root INFO [eamodio.gitlens-14.0.1]: already found
2023-07-26T09:32:19.492Z root ERROR Failed to load plugin dependencies from 'c:\Users\thomas\AppData\Local\Temp\vscode-unpacked\eamodio.gitlens-14.0.1' path Error: ENOENT: no such file or directory, open 'c:\Users\thomas\AppData\Local\Temp\vscode-unpacked\eamodio.gitlens-14.0.1\package.json'
2023-07-26T09:32:19.492Z root INFO Deploy plugin vscode:extension/eamodio.gitlens: 2411.3 ms [Finished 31.606 s after backend start]
2023-07-26T09:32:19.492Z root ERROR Request deploy failed with error: Deployment of extension with ID vscode:extension/eamodio.gitlens failed. Error: Deployment of extension with ID vscode:extension/eamodio.gitlens failed.
    at PluginServerHandler.deploy (C:\Users\thomas\code\testdata\theia\examples\electron\lib\backend\main.js:18643:19)
    at async RpcProxyFactory.onRequest (C:\Users\thomas\code\testdata\theia\examples\electron\lib\backend\packages_core_lib_common_index_js.js:4008:24)
    at async RpcProtocol.handleRequest (C:\Users\thomas\code\testdata\theia\examples\electron\lib\backend\packages_core_lib_common_index_js.js:3447:28)
2023-07-26T09:32:19.503Z root INFO [e052bc08-9293-4843-a690-c15e4fc3d46a] Waiting for backend deployment: 7.7 ms [Finished 18.307 s after frontend start]
2023-07-26T09:32:19.503Z root INFO [e052bc08-9293-4843-a690-c15e4fc3d46a] Sync of 0 plugins: 0.1 ms [Finished 18.307 s after frontend start]
2023-07-26T09:32:19.504Z root INFO [e052bc08-9293-4843-a690-c15e4fc3d46a] Loading plugin contributions
2023-07-26T09:32:19.504Z root INFO [e052bc08-9293-4843-a690-c15e4fc3d46a] Load contributions of 0 plugins: 0.1 ms [Finished 18.307 s after frontend start]
2023-07-26T09:32:19.516Z root INFO [e052bc08-9293-4843-a690-c15e4fc3d46a] Start of 0 plugins: 8.4 ms [Finished 18.321 s after frontend start]

[tsmaeder]

no such file or directory, open 'c:\Users\thomas\AppData\Local\Temp\vscode-unpacked\eamodio.gitlens-14.0.1\package.json'

Note that the correct path to that file is ...\eamodio.gitlens-14.0.1\extension\package.json

[tsmaeder]

The strange this it that the install sometimes works.

[tsmaeder]

Removing the exploded vsix from ~AppData/Local/temp/vscode-unpacked make the install work again.

[mcg1103]

I have been trying to get gitlens to install as well. Are you using theia git or vscode git?
would you know where the vscode-unpacked folder on linux?
Thanks,
mark

[tsmaeder]

If you don't use vscode-git, gitlens will not find any repositories, ever. And no, I don't know where that folder is on Linux.

[msujew]

I assume the folder can be found at /tmp/vscode-unpacked.

[JonasHelming]

@tsmaeder I cannot reproduce this, can you reproduce this on a "fresh" Blueprint?

[tsmaeder]

@JonasHelming you have to have gitlens installed before the error occurs. Not sure what the repro scenario is, just reporting what happened in a dog-fooding env.

[JonasHelming]

I trying to find the repro scenario. Just to make sure: You assume it was an update case, i.e. Gitlens has already been installed and you installed it again? (Which I guess it shouldn't actually even allow)

[joshuami]

I ran into a similar issue using Acquia's Cloud IDE (which is Theia with some additional PHP and Drupal built in extensions). @tsmaeder 's comment about needing vscode-git triggered the solution for me. For some reason, vscode-git was no longer installed. It shows with a "(built in)" label now, but still under my "installed" extensions. That makes me think the vscode git is no longer built into Theia (or at least was lost from the version of Theia in Acquia Cloud IDE).

Getting the "no respository found" issue solved allowed me to reinstall gitlens.

Hopefully this unblocks someone else.

[JonasHelming]

@marcdumais-work Did you change anything relevant in the built-ins configuration that could explain this?

[marcdumais-work]

Hey Mark,

would you know where the vscode-unpacked folder on linux?

It's usually /tmp/vscode-unpacked

[marcdumais-work]

@marcdumais-work Did you change anything relevant in the built-ins configuration that could explain this?

@JonasHelming I don't think so, but now my curiosity is picked. I will investigate this today, first trying to reproduce the issue.

@joshuami It's a long shot, but around 1.63.0, vscode split their git extension into two, adding extension vscode.git-base / Git Base (built-in), to the existing vscode.git / Git (built-in). So, if your application were to use a version of our "Built-in extension pack" that predates the split, you might end-up not having "vscode.git-base" installed, possibly.

cc: @vince-fugnitto

[tsmaeder]

@marcdumais-work I think it's worth looking at the stack trace: #12757 (comment)

[marcdumais-work]

@tsmaeder I think I managed to reproduce your stack trace:

2023-08-09T15:22:02.583Z root INFO [eamodio.gitlens]: trying to resolve latest version...
[...]
2023-08-09T15:22:02.802Z root INFO [eamodio.gitlens]: resolved to 'eamodio.gitlens-14.2.0'
2023-08-09T15:22:02.803Z root INFO [eamodio.gitlens-14.2.0]: trying to download from "https://open-vsx.org/api/eamodio/gitlens/14.2.0/file/eamodio.gitlens-14.2.0.vsix"... to path /home/lmcmcds/.theia/extensions
2023-08-09T15:22:10.268Z root INFO [eamodio.gitlens-14.2.0]: downloaded to /home/lmcmcds/.theia/extensions/eamodio.gitlens-14.2.0.vsix"
2023-08-09T15:22:10.269Z root INFO [eamodio.gitlens-14.2.0]: trying to decompress into "/tmp/vscode-unpacked/eamodio.gitlens-14.2.0"...
2023-08-09T15:22:10.307Z root INFO [eamodio.gitlens-14.2.0]: already found
2023-08-09T15:22:18.422Z root ERROR Failed to load plugin dependencies from '/tmp/vscode-unpacked/eamodio.gitlens-14.2.0' path Error: ENOENT: no such file or directory, open '/tmp/vscode-unpacked/eamodio.gitlens-14.2.0/package.json'
2023-08-09T15:22:18.424Z root INFO Deploy plugin vscode:extension/eamodio.gitlens: 15843.6 ms [Finished 411.150 s after backend start]
2023-08-09T15:22:18.424Z root ERROR Request deploy failed with error: Deployment of extension with ID vscode:extension/eamodio.gitlens failed. Error: Deployment of extension with ID vscode:extension/eamodio.gitlens failed.
[...]
2023-08-09T15:22:18.457Z root INFO [f48bd123-2043-4817-95ec-b0c2fc522dc5] Waiting for backend deployment: 26.0 ms [Finished 101.944 s after frontend start]
2023-08-09T15:22:18.457Z root INFO [f48bd123-2043-4817-95ec-b0c2fc522dc5] Sync of 0 plugins: 0.0 ms [Finished 101.944 s after frontend start]
2023-08-09T15:22:18.457Z root INFO [f48bd123-2043-4817-95ec-b0c2fc522dc5] Loading plugin contributions
2023-08-09T15:22:18.459Z root INFO [f48bd123-2043-4817-95ec-b0c2fc522dc5] Load contributions of 0 plugins: 0.0 ms [Finished 101.944 s after frontend start]
2023-08-09T15:22:18.484Z root INFO [f48bd123-2043-4817-95ec-b0c2fc522dc5] Start of 0 plugins: 16.3 ms [Finished 101.975 s after frontend start]

[marcdumais-work]

@tsmaeder The way I did it was to go under /tmp/vscode-unpacked/eamodio.gitlens-14.2.0/ and erase folder extension, to simulate a badly corrupted extension, under there.

This reproduces even the strangest part of your trace, the mysteriously missing "extensions" folder, when trying to open the extension's package.json:
Error: ENOENT: no such file or directory, open '/tmp/vscode-unpacked/eamodio.gitlens-14.2.0/package.json'

The way that we handle the plugins under vscode-unpacked is not very sophisticated. If there is a sub-directory ion there that matches the name for the plugin that's being installed (e.g. "/tmp/vscode-unpacked/eamodio.gitlens-14.2.0/"), our PluginVsCodeFileHandler (1) assumes it contains a full copy of the plugin and exits, rather than e.g. replace it with the freshly downloaded plugin, unzipped.

I do not have a good theory how your folder might have gotten corrupted. Alternatively, maybe we have a code flow where we create the directory under vscode-unpacked and leave it in place despite not being able to finish with an extension's installation, leaving a stale folder, preventing re-installing that same version of the extension.

Also, on Linux, /tmp is usually cleared upon reboot - I am not sure if the same is true on Windows. If not, a corrupted extension under vscode-unpacked or an empty folder left there, could stay indefinitely.

(1): line 47

[tsmaeder]

Note that the directory where the plugins are uncompressed to is share among all installations of theia in the system. It might well be that an uninstall of the plugin from one instance may corrupt the uncompressed directory in /tmp when it's still used somewhere else.

[tsmaeder]

The plugins "deployment" is a bit of a mess, IMO: it confounds plugin installation, plugin deployment (uncompress to a suitable location) and discovery of plugins metadata. That makes it hard to understand and therefore prone to error. It also consumes considerable resources at startup: #9868. If we had a more static view of the installed plugins, we could cache the parsed plugin metadata in a file and only reread the metadata from the uncompressed metadata when the set of installed plugins changes, for example. The set of plugins might also be fixed in some application cases and maybe only changes through an installer. There are opportunities here!

[tsmaeder]

Maybe this one is related, as well: #12485

[marcdumais-work]

I think one relatively easy way to avoid this issue and contribute to keeping the start time shorter, would be to unzip extensions downloaded by the user, from the extension's view, that end-up in <user's home>/.theia/extensions/ on Linux, saved there in .vsix format.

vscode unzips the extension in that same case, in folder <user's home>/.vscode/extensions/

[marcdumais-work]

Some background:

I think the main reason we have this global vscode-unpacked folder is because we can't load extensions that are in .vsix format (zip). This can happen if extensions are bundled that have not been unzipped (an option of @theia/cli, theia download:plugins). In some cases it would be possible to unzip them in place, but not always. e.g. when part of an AppImage package, or installed system-wide through a .deb. or .rpm package, with read-only access to the install.

[marcdumais-work]

the directory where the plugins are uncompressed to is share among all installations of theia in the system.

I have recently noticed it's not the case on at least some Windows machines, where the temp folder is per-user, and apparently never cleaned :)

update: here's the folder: C:\Users\<Username>\AppData\local

[JonasHelming]

@marcdumais-work Thank you for the analysis. Would you by any chance be able to provide a fix? Fixing the issue and potentially imporving the start-up time would be a win-win! (@planger )

[tsmaeder]

I have recently noticed it's not the case on at least some Windows machines, where the temp folder is per-user, and apparently never cleaned :)

Yes, it's per user, but it's shared among all installations of Theia-based products for that user.

[tsmaeder]

I just got into a state where gitlens shows as uninstalled and I have two versions of the folder: eamodio.gitlens-14.2.0.vsix which seems to have the correct structure and eamodio.gitlens-14.2.0 which is missing the package.json in it's /extension folder. I have neither installed nor uninstalled gitlens recently.

[marcdumais-work]

@tsmaeder Looking locally I think I see similar presence of the two types of folders under vscode-unpacked, except all folders seem to have their proper content. I think the folders that end in .vsix are created by Blueprint and the others by the Theia Example app.

~/theia$ ll ~/.theia/extensions/
total 244
drwxr-xr-x 2 user eusers   4096 Aug 16 15:25 ./
drwxr-xr-x 8 user eusers   4096 Aug 16 15:25 ../
-rw-r--r-- 1 user eusers 240803 Aug 16 15:25 dbaeumer.vscode-eslint-2.4.2.vsix

~/theia$ ll ~/.theia-blueprint/extensions/
total 7008
drwxr-xr-x 2 user eusers    4096 Aug 10 14:00 ./
drwxr-xr-x 8 user eusers    4096 Jun  2 11:30 ../
-rw-r--r-- 1 user eusers  240240 Jun  2 11:30 dbaeumer.vscode-eslint-2.4.0.vsix
-rw-r--r-- 1 user eusers 6922979 Aug 10 14:00 eamodio.gitlens-14.2.0.vsix

[tsmaeder]

@marcdumais-work do you think there could be a fix for this in the near future? It would be a candidate to backport to the community release.

[xai]

I will take a look.

[msujew]

FYI, I've renamed the issue, because the issue is not specifically related to the gitlens extension but can happen with any extension.

[goekce]

I installed theia-browser on my system with many users. If a user installs a plugin, then it lands on /tmp/vscode-unpacked/.... If a second user wants to install the same plugin, then they get an error that they don't have permission to modify the files under /tmp/vscode-unpacked/.... So the first user has to delete their files after unpacking. I believe this problem can be solved by isolating /tmp between users or by appending a random string to the vscode-unpacked directory.

Is my problem related to this issue?

[msujew]

@goekce No it's unrelated. Theia isn't designed for multi-tenancy, see also this discussion. There are way more security related issues when you start to allow multiple users on a single system.

[xai]

I opened a draft PR that attempts to resolve our issues and that implements the suggestion by @marcdumais-work to unpack the extensions to <user's home>/.theia/extensions/.

As @msujew mentioned, there are likely plenty more problems we face when using Theia on multi-user systems (even if they are not working in parallel, but just use the system exclusively at a time). However, I would assume that at least this particular permission-related issue mentioned by @goekce is also resolved by this patch, as the only temporary directory used during the vscode extension download/deployment process now includes the username and, therefore, avoids the mentioned ownership conflict.