[WIP] Plugin message-rpc #38

tortmayr · 2022-04-29T12:03:06Z

Refactor plugin-ext rpc protocol to reuse the new binary message-rpc protocol.
- Remove custom RPC message encoding and handling reuse message-rpc
- Implement QueuingChannelMultiplexerthat queues messages and sends them accumlated on the next process.tick (replaces the old Multiplexer implementation
- Refactors proxy handlers and remote target handlers
- Use Channel instead of MessageConnection for creating new instances of RPCProtocol
- Implement special message encoders and decoders for the plugin communication
Adapt the HostedPluginServer and HostedPluginClient API to send/receive messages in binary format instead of strings.
Implement a message protocol for process pipe streams to be able to send messages in binary format between the hosted-plugin-process and the plugin-host.

Contributed on behalf of STMicroelectronics
Part of eclipse-theia#10684

lucas-koehler

Looks pretty good. I added some minor comments.

lucas-koehler · 2022-04-29T13:41:05Z

packages/core/src/electron-main/messaging/electron-messaging-contribution.ts

+        const sender = event.sender;
+        try {
+            // Get the multiplexer for a given window id
+            const windowChannelData = this.windowChannelMultiplexer.get(sender.id)!;


Why is there a ! if we check in the next line whether windowChannelData exists?

lucas-koehler · 2022-04-29T13:43:04Z

packages/core/src/node/messaging/ipc-connection-provider.ts

+        const onCloseEmitter = new Emitter<ChannelCloseEvent>();
+        const onMessageEmitter = new Emitter<MessageProvider>();
+        const onErrorEmitter = new Emitter<unknown>();
+        const pipe = childProcess.stdio[4] as Writable;


Should this be as Readable as we are reading data from it below?

lucas-koehler · 2022-04-29T13:51:13Z

packages/plugin-ext/src/common/rpc-protocol.ts

-    private cancel(req: string): string {
-        return `{"type":${MessageType.Cancel},"id":"${req}"}`;
+        this.registerEncoder(ObjectType.RESPONSE_ERROR, {
+            is: () => false,


Why do we add an encoder that never applies?

lucas-koehler · 2022-04-29T13:55:09Z

packages/plugin-ext/src/hosted/node/cached-process-messaging.ts

+}
+
+export function encodeMessageStart(buffer: ArrayBuffer): Uint8Array {
+    // Prepend message length before data


this is no longer what we do here. remove comment

lucas-koehler · 2022-04-29T13:59:54Z

packages/plugin-ext/src/plugin/documents.ts

+            ({
+                range: Converter.toRange(change.range),
+                rangeOffset: change.rangeOffset,
+                rangeLength: change.rangeLength,
+                text: change.text
+            }))


nothing changed here so the file should not have changed, shouldn't it?

lucas-koehler · 2022-04-29T14:00:01Z

packages/plugin-ext/src/plugin/notification.ts

@@ -35,7 +35,7 @@ export class NotificationExtImpl implements NotificationExt {
    ): Promise<R> {
        const source = new CancellationTokenSource();
        const id = new Deferred<string>();
-        const progress = task({ report: async item => this.proxy.$updateProgress(await id.promise, item)}, source.token);
+        const progress = task({ report: async item => this.proxy.$updateProgress(await id.promise, item) }, source.token);


nothing changed here so the file should not have changed, shouldn't it?

lucas-koehler · 2022-04-29T14:00:10Z

packages/plugin-ext/src/plugin/types-impl.ts

 import { es5ClassCompat } from '../common/types';
+import { ObjectsTransferrer } from '../common/rpc-protocol';


nothing changed here so the file should not have changed, shouldn't it?

The commit upgrades the `yarn.lock` file in preparation for the `v1.25.0` release. Upgrading the lockfile is important to get a better representation of what downstream applications pull. Signed-off-by: vince-fugnitto <[email protected]>

…eia#11095) Fixes a bug where `when` contexts and custom context keys were ignored.

Fixes a bug preventing Monaco from rendering keyboard shortcut hints correctly.

* [playwright] Improve getting started documentation Requires eclipse-theia/theia-playwright-template#1 Fixes eclipse-theia#11015 Change-Id: I5838de12b7e3e840e354e8230a35c752c9f25dd1 Signed-off-by: Philip Langer <[email protected]>

The commit updates the root `package.json` to use the proper `id` for builtin themes so they are not re-downloaded by the `extension-pack`. Signed-off-by: vince-fugnitto <[email protected]>

Signed-off-by: Thomas Mäder <[email protected]>

@tsmaeder

Refactors and improves the prototype of a faster JSON-RPC protocol initially contributed by @tsmaeder (See also eclipse-theia#10781). The encoding approach used in the initial POC has some performance drawbacks when encoding plain JSON objects. We refactored the protocol to improve the performance for JSON objects whilst maintaining the excellent performance for encoding objects that contain binary data. Integrates the new message-rpc prototype into the core messaging API (replacing vscode-ws-jsonrpc). This has major impacts on the Messaging API as we no longer expose a `Connection` object (which was provided by vscode-ws-jsonrpc) and directly rely on a generic transport `Channel` implementation instead. - Introduce `Channel` as the main transport concept for messages (instead of the dedicated `Connection` from vscode-jsonrpc) - Replace usage of `vscode-ws-jsonrpc` with a custom binary RPC protocol. - Refactor all connection providers to use the new binary protocol. - Ensure that the `RemoteFileSystemProvider` API uses `Uint8Arrays` over plain number arrays. This enables direct serialization as buffers and reduces the overhead of unnecessarily converting from and to `Uint8Arrays`. - Refactor terminal widget and terminal backend contribution so that the widgets communicates with the underlying terminal process using the new rpc protocol. - Rework the IPC bootstrap protocol so that it uses a binary pipe for message transmission instead of the `ipc` pipe which only supports string encoding. - Extend the `JsonRpcProxyFactory` with an optional `RpcConnectionFactory` that enables adopter to creates proxies with a that use a custom `RpcProtocol`/`RpcConnection`. The plugin API still uses its own RPC protocol implementation. Currently we have to encode/decode between binary data to handle RPC calls from a plugin context. Aligning the two protocols and zero-copy tunneling of RPC messages is planned for a follow-up PR. Contributed on behalf of STMicroelectronics. Closes eclipse-theia#10684

Address review feedback: - Merge and RpcClient and RpcServer in RpcProtocol - Only export public API in index.ts - Remove outdated comments - Refactor error value encoder - Fix casing in `MessageTypes Integrate performance improvements - Use Uint8Array instead of ArrayBuffer when feasible - Use TextEncoder.encodeInto() instead of encode() - Use improved encoding for length values - Preallocate a single TextEncoder - Consider potential buffer byteOffsets when creating new Write/Reader buffers. Contributed on behalf of STMicroelectronics

+ Remove depdendency to vscode-ws-jsonrpc Contributed on behalf of STMicroelectronics

Refactor plugin-ext rpc protocol to reuse the new binary message-rpc protocol. - - Remove custom RPC message encoding and handling reuse message-rpc -- Implement `QueuingChannelMultiplexer`that queues messages and sends them accumlated on the next process.tick (replaces the old `Multiplexer` implementation -- Refactors proxy handlers and remote target handlers -- Use `Channel` instead of `MessageConnection` for creating new instances of `RPCProtocol` -- Implement special message encoders and decoders for the plugin communication - Adapt the `HostedPluginServer` and `HostedPluginClient` API to send/receive messages in binary format instead of strings. - Implement a message protocol for process pipe streams to be able to send messages in binary format between the hosted-plugin-process and the plugin-host. Co-authored-by: Lucas Koehler <[email protected]> Part of eclipse-theia#10684 Contributed on behalf of STMicroelectronics

- add missing decoder/encoder for range, uri, vscode uri, binary buffer - cache messages in hosted plugin watcher before listeners are registered. otherwise channel open messages can be lost

- NotificationExt was instantiated in the main context - There were unused notification proxy identifiers for main and ext in the wrong contexts

…otocol

Treat them as requests to support cancellation

sdirix · 2022-11-16T14:22:37Z

Code integrated in merged upstream PR eclipse-theia#11261

Review and adapt prompt templates

tortmayr changed the title ~~Wip plugin message rpc~~ [WIP] Plugin message-rpc Apr 29, 2022

tortmayr requested a review from lucas-koehler April 29, 2022 12:03

lucas-koehler reviewed Apr 29, 2022

View reviewed changes

repo: upgrade yarn.lock

9c06e3d

The commit upgrades the `yarn.lock` file in preparation for the `v1.25.0` release. Upgrading the lockfile is important to get a better representation of what downstream applications pull. Signed-off-by: vince-fugnitto <[email protected]>

tortmayr force-pushed the wip-plugin-message-rpc branch from ce98274 to 89ad62d Compare May 2, 2022 07:00

colin-grant-work and others added 14 commits May 2, 2022 08:29

Fix MacOS browser menu keybinding display (eclipse-theia#11092)

08722fa

Ensure MonacoContexKeyDelegate is also used inside Monaco (eclipse-th…

14d8cd1

…eia#11095) Fixes a bug where `when` contexts and custom context keys were ignored.

Update Monaco keybindings on keybinding change (eclipse-theia#11101)

eb0930a

Fixes a bug preventing Monaco from rendering keyboard shortcut hints correctly.

Update keybinding widget when keybindings change (eclipse-theia#11102)

1ac9da1

Add missing PR links, update files.autoSave notice (eclipse-theia#11106)

b6caaf0

repo: give proper id for builtin themes

df80a99

The commit updates the root `package.json` to use the proper `id` for builtin themes so they are not re-downloaded by the `extension-pack`. Signed-off-by: vince-fugnitto <[email protected]>

Protoype of a faster RPC protocol

bbea4ae

Signed-off-by: Thomas Mäder <[email protected]>

Integrate improved original message encoder

deac3c1

+ Remove depdendency to vscode-ws-jsonrpc Contributed on behalf of STMicroelectronics

adapt to rebase and add client and server modes to plugin rpc protocol

27598a5

proxy debug messages

e931411

lucas-koehler force-pushed the wip-plugin-message-rpc branch from 89ad62d to e931411 Compare May 9, 2022 15:43

lucas-koehler added 5 commits May 11, 2022 16:59

fix plugin communication

891a066

- add missing decoder/encoder for range, uri, vscode uri, binary buffer - cache messages in hosted plugin watcher before listeners are registered. otherwise channel open messages can be lost

Remove incorrect notification proxy identifiers and instantiation

289a4ff

- NotificationExt was instantiated in the main context - There were unused notification proxy identifiers for main and ext in the wrong contexts

Remove ObjectTransferrer and transformations param from plugin rpc pr…

d46cbd4

…otocol

Fix handling for notifications with cancellation token

ce72120

Treat them as requests to support cancellation

debugging log messages for message tracing

ba9f1e1

sdirix closed this Nov 16, 2022

CamilleLetavernier added a commit that referenced this pull request Jul 29, 2024

Merge pull request #105 from /issues/38

666ae13

Review and adapt prompt templates

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Plugin message-rpc #38

[WIP] Plugin message-rpc #38

tortmayr commented Apr 29, 2022

lucas-koehler left a comment

lucas-koehler Apr 29, 2022

lucas-koehler Apr 29, 2022

lucas-koehler Apr 29, 2022

lucas-koehler Apr 29, 2022

lucas-koehler Apr 29, 2022

lucas-koehler Apr 29, 2022

lucas-koehler Apr 29, 2022

sdirix commented Nov 16, 2022

		import { es5ClassCompat } from '../common/types';
		import { ObjectsTransferrer } from '../common/rpc-protocol';

[WIP] Plugin message-rpc #38

[WIP] Plugin message-rpc #38

Conversation

tortmayr commented Apr 29, 2022

lucas-koehler left a comment

Choose a reason for hiding this comment

lucas-koehler Apr 29, 2022

Choose a reason for hiding this comment

lucas-koehler Apr 29, 2022

Choose a reason for hiding this comment

lucas-koehler Apr 29, 2022

Choose a reason for hiding this comment

lucas-koehler Apr 29, 2022

Choose a reason for hiding this comment

lucas-koehler Apr 29, 2022

Choose a reason for hiding this comment

lucas-koehler Apr 29, 2022

Choose a reason for hiding this comment

lucas-koehler Apr 29, 2022

Choose a reason for hiding this comment

sdirix commented Nov 16, 2022