Implement simple file system provider to fetch files from containerized plugin #18405

vzhukovs · 2020-11-18T15:07:23Z

Is your task related to a problem? Please describe.

Use case 1:

When VS Code extension runs in dedicate container, it can provide a Webview with own html content. For example will take a look on vscode-didact.

didactWebView.ts#L358-L372:

		const completedHtml = `<!DOCTYPE html>
		<html lang="en">
		<head>
			<meta charset="UTF-8"/>
			<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
			<meta http-equiv="Content-Security-Policy" content="default-src 'none'; img-src 'self' data: https: http: blob: ${this._panel.webview.cspSource}; media-src vscode-resource: https: data:; script-src 'nonce-${nonce}' https:; style-src 'unsafe-inline' ${this._panel.webview.cspSource} https: data:; font-src ${this._panel.webview.cspSource} https: data:; object-src 'none';"/>
			<base href="${uriBase}${uriBase.endsWith('/') ? '' : '/'}"/>
			<title>Didact Tutorial</title>` + 
			stylesheetHtml + 
			`<script defer src="https://use.fontawesome.com/releases/v5.3.1/js/all.js"></script>
			</head>
		<body class="content">` + didactHtml + 
		`<script nonce="${nonce}" src="${scriptUri}"/>
		</body>
		</html>`;

src="${scriptUri}" constructs with scheme vscode-resource

didactWebView.ts#L344:

		const scriptUri = scriptPathOnDisk.with({ scheme: 'vscode-resource' });

When extension calls method to create a Webview it sets up a complete html content into a webview object:

didactWebView.ts#L409

			this._panel.webview.html = this.currentHtml;

So Theia receives the html content it performs preprocess income html content by replacing all links that have vscode-resource scheme:

webview.ts#L393:

    protected preprocessHtml(value: string): string {
        return value
            .replace(/(["'])(?:vscode|theia)-resource:(\/\/([^\s\/'"]+?)(?=\/))?([^\s'"]+?)(["'])/gi, (_, startQuote, _1, scheme, path, endQuote) => {
                if (scheme) {
                    return `${startQuote}${this.externalEndpoint}/theia-resource/${scheme}${path}${endQuote}`;
                }
                return `${startQuote}${this.externalEndpoint}/theia-resource/file${path}${endQuote}`;
            });
    }

And in result Webview's html we have got links that looks like:

https://servergokcng5l-jwtproxy-webviews.192.168.64.86.nip.io/webview/theia-resource/file/tmp/vscode-unpacked/redhat.vscode-didact.0.1.14.uteaiieoau.vscode-didact-0.1.14.vsix/extension/media/webviewslim.css

were service worker extracts path: file/tmp/vscode-unpacked/redhat.vscode-didact.0.1.14.uteaiieoau.vscode-didact-0.1.14.vsix/extension/media/webviewslim.css and tries to load such resources:

webview.ts#L444-L486

    protected async loadResource(requestPath: string): Promise<void> {
        const normalizedUri = this.normalizeRequestUri(requestPath);
        // browser cache does not support file scheme, normalize to current endpoint scheme and host
        const cacheUrl = new Endpoint({ path: normalizedUri.path.toString() }).getRestUrl().toString();

        try {
            if (this.contentOptions.localResourceRoots) {
                for (const root of this.contentOptions.localResourceRoots) {
                    if (!new URI(root).path.isEqualOrParent(normalizedUri.path)) {
                        continue;
                    }
                    let cached = await this.resourceCache.match(cacheUrl);
                    try {
                        const result = await this.fileService.readFileStream(normalizedUri, { etag: cached?.eTag });
                        const { buffer } = await BinaryBufferReadableStream.toBuffer(result.value);
                        cached = { body: () => buffer, eTag: result.etag };
                        this.resourceCache.put(cacheUrl, cached);
                    } catch (e) {
                        if (!(e instanceof FileOperationError && e.fileOperationResult === FileOperationResult.FILE_NOT_MODIFIED_SINCE)) {
                            throw e;
                        }
                    }
                    if (cached) {
                        const data = await cached.body();
                        return this.doSend('did-load-resource', {
                            status: 200,
                            path: requestPath,
                            mime: mime.getType(normalizedUri.path.toString()) || 'application/octet-stream',
                            data
                        });
                    }
                }
            }
        } catch {
            // no-op
        }

        this.resourceCache.delete(cacheUrl);
        return this.doSend('did-load-resource', {
            status: 404,
            path: requestPath
        });
    }

As the result we have got 404 error, as Theia tries to load file from local container, not from remote ones, which is obvious.

Use case 2:

From discussion with @tsmaeder it turned out similar problem, when VS Code extension that run in remote container might call: vscode.fs.watch(file, ...). Which will call watching for a file for a nonexistent file path.

Describe the solution you'd like

It is possible to modify schema from file to file-sidecar-${machineName} and get URL for links like:

https://serverh6igilxw-jwtproxy-webviews.192.168.64.88.nip.io/webview/theia-resource/file-sidecar-vscode-didact0zc/tmp/vscode-unpacked/redhat.vscode-didact.0.1.14.sdxxiurire.vscode-didact-0.1.14.vsix/extension/media/webviewslim.css

But fileService under the hood tries to load the resource by given file system provider based on the given scheme:

file-service.ts#L380-L402:

    async activateProvider(scheme: string): Promise<FileSystemProvider> {
        let provider = this.providers.get(scheme);
        if (provider) {
            return provider;
        }
        let activation = this.activations.get(scheme);
        if (!activation) {
            const deferredActivation = new Deferred<FileSystemProvider>();
            this.activations.set(scheme, activation = deferredActivation.promise);
            WaitUntilEvent.fire(this.onWillActivateFileSystemProviderEmitter, { scheme }).then(() => {
                provider = this.providers.get(scheme);
                if (!provider) {
                    const error = new Error();
                    error.name = 'ENOPRO';
                    error.message = `No file system provider found for scheme ${scheme}`;
                    throw error;
                } else {
                    deferredActivation.resolve(provider);
                }
            }).catch(e => deferredActivation.reject(e));
        }
        return activation;
    }

So we are missing file system providers for schema like file-sidecar-${machineName} which will communicate with the particular container to obtain remote files.

There is an idea to create a simple file system provider as it has been done for CheSideCarResourceResolver: che-sidecar-resource.ts#L50-L67

Describe alternatives you've considered

Investigate how HostedPluginReader works: plugin-reader.ts. This might help.
Still looking for the alternative ways how to obtain files from remote container in Theia's one.

Additional context

Fixes: #16870
Might be somehow related: eclipse-theia/theia#8429 eclipse-theia/theia#8468

Does anybody has a some thoughts about this? cc @tsmaeder

The text was updated successfully, but these errors were encountered:

gattytto · 2020-12-04T18:25:12Z

Describe alternatives you've considered

Still looking for the alternative ways how to obtain files from remote container in Theia's one.

I have some thoughts in this particular.

as for the whole in this issue, this provider will target sidecar files, and about sidecar files there's this one

thank you for all the investigation and the detail in presentation.

vzhukovs added kind/task Internal things, technical debt, and to-do tasks to be performed. area/plugins area/editors labels Nov 18, 2020

che-bot added the status/need-triage An issue that needs to be prioritized by the curator responsible for the triage. See https://github. label Nov 18, 2020

vzhukovs removed the area/plugins label Nov 18, 2020

azatsarynnyy mentioned this issue Nov 18, 2020

Editors Sprint 193 #18403

Closed

15 tasks

azatsarynnyy mentioned this issue Nov 18, 2020

Che Theia Web View doesn't work properly when a plugin is running in its own sidecar #16870

Closed

1 task

azatsarynnyy added this to the 7.23 milestone Nov 18, 2020

azatsarynnyy modified the milestones: 7.23, 7.24 Dec 2, 2020

vzhukovs mentioned this issue Dec 7, 2020

Implement simple side car file system which fetches file content eclipse-che/che-theia#946

Merged

9 tasks

azatsarynnyy mentioned this issue Dec 9, 2020

Editors Sprint 194/195 #18568

Closed

21 tasks

vzhukovs closed this as completed Dec 20, 2020

vzhukovs removed the status/in-progress This issue has been taken by an engineer and is under active development. label Dec 20, 2020

azatsarynnyy assigned vzhukovs Jan 19, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement simple file system provider to fetch files from containerized plugin #18405

Implement simple file system provider to fetch files from containerized plugin #18405

vzhukovs commented Nov 18, 2020 •

edited

Loading

gattytto commented Dec 4, 2020 •

edited

Loading

Describe alternatives you've considered

Implement simple file system provider to fetch files from containerized plugin #18405

Implement simple file system provider to fetch files from containerized plugin #18405

Comments

vzhukovs commented Nov 18, 2020 • edited Loading

Is your task related to a problem? Please describe.

Use case 1:

Use case 2:

Describe the solution you'd like

Describe alternatives you've considered

Additional context

gattytto commented Dec 4, 2020 • edited Loading

Describe alternatives you've considered

vzhukovs commented Nov 18, 2020 •

edited

Loading

gattytto commented Dec 4, 2020 •

edited

Loading