Bundle open source SSH tunneling extension

[juliasilge]

For Public Beta, we decided to bundle the same open source SSH tunneling extension that VSCodium users tend to use:

https://open-vsx.org/extension/jeanp413/open-remote-ssh

[juliasilge]

We do have some awkward UI around bundled extensions tracked in #2115

[jmcphers]

I thought this might be as easy as bundling the extension, but it is ... not.

• The extension is hard-coded to download VS Codium into the remote. It is possible to override this with the serverDownloadUrlTemplate option, but we would need to host Positron at a public URL in order to make it possible to download it into the remote machine.
• We would need to add a large number of new builds to our matrix to support remote development, one per OS that we want to support as an SSH development target. You can see here: https://github.com/jeanp413/open-remote-ssh/blob/8191939de706952fa035f80a6fce0e85f85495f6/src/serverSetup.ts#L38 that what we need to build for this are the Remote Extension Host (REH) variants of VS Code/Positron. Currently we're only building desktop installers, which can't be used for remote development backends.
• We really haven't tested non-local development scenarios and may find that a lot of things do not work because they assume the extension host and frontend are co-located on the same machine. (I'm particularly worried about all the stuff that uses localhost servers).

In short, this is probably a week of effort if everything goes right, and more if we encounter trouble. I'm going to un-triage this for discussion.

[jmcphers]

Branch here which contains the minimal changes (bundling the extension and re-enabling the Remote Window status indicator/connection tool):

https://github.com/posit-dev/positron/tree/feature/bundle-open-remote-ssh

[bassamsdata]

Hello,
Would it be possible to bundle the other extension Open Remote -WSL as well, from the same developer that allows connection to WSL on Windows with just a click of a button?

Python and Windows often don't play well together, and I usually connect to Linux-WSL directly when opening VSCodium.

Thank you

[andrewheiss]

It turns out that it's possible to connect remotely without needing to host Positron at a special URL—the only issue is that the wrong SHA gets placed in product.json on the remote server. If that is manually edited after an installation fails, Positron can connect. Following this incredibly helpful comment here, this process works:

1. In settings.json locally, set this:

   "remote.SSH.serverDownloadUrlTemplate": "https://github.com/gitpod-io/openvscode-server/releases/download/openvscode-server-v${version}/openvscode-server-v${version}-${os}-${arch}.tar.gz"

2. Try to connect to a remote server. It will connect and download and unzip stuff that needs to be installed on the remote server, but it will then fail with an error message.

3. On the remote server, there will be a new directory at ~/.positron-server/bin/SOME_LONG_SHA_HASH. Open that and edit the product.json in there. At the very bottom of that file, there's an entry called "commit". Add the commit SHA for the currently installed version of Positron there (which also happens to be the name of the folder that it creates when downloading).

4. In settings.json locally, set this:

   "remote.SSH.experimental.serverBinaryName": "openvscode-server",

It should then connect just fine, but Positron won't be able to run anything in R or Python because it can't find an interpreter (which is being tracked, I think, here)

[jmcphers]

@andrewheiss excellent detective work!

What is effectively happening here is that you're connecting a Positron front end to a OpenVSCode Server back end. It will work -- but only partially. The only parts of the system that will function are those that are common to both products (that is, base Code OSS behavior).

In order to run R and Python interpreters or use any Positron specific features, you need a specially built copy of Positron itself on the remote host. We're working on it!

[jfsalzmann]

@jmcphers does that mean it won't be possible to use a local Positron to connect a SSH tunnel to a remote server that has only R installed, for example? It always requires a full positron installation on the other side, too?

[jmcphers]

Kind of. It's not a full Positron installation, and you won't need to install it manually -- when you connect to the remote server for the first time, Positron will automatically download the necessary binaries for you (into e.g. ~/.positron-server) and run them on the remote host (no privilege required).

[ZhimingYe]

@jmcphers I have mentioned this issue in the discussion part of this project about a Persistent SSH connection. For data scientists, this is crucial, especially when handling large datasets (like single-cell sequencing or analyzing biobank data). However, I later realized that this requirement might be more challenging than initially thought.

I encountered the same issue with VSCode's remote extension. Unfortunately, even after four years of being mentioned, this issue is still open: microsoft/vscode-remote-release#3096. This makes me worry that no matter which SSH plugin is bundled, this problem might be unavoidable.

For Python, I have been using the nbconvert command combined with nohup or tmux to address this issue over the past few years. However, I greatly miss the experience with RStudio Server, where reconnecting to a remote server seamlessly continues the output.

Given that the VSCode team has not resolved this issue in four years, I fear it might be quite a difficult problem. Therefore, I am bringing it up again here. On the VSCode issue page, someone mentioned that devtunnels (https://aka.ms/devtunnels) are not using the usual SSH but their own implementation, dev-tunnels-ssh (https://github.com/microsoft/dev-tunnels-ssh/), which states in the README:

Supports reconnecting a disconnected session without disrupting channel streams.

This might be a potential solution for your reference. Thank you for all the efforts of your team. Though it may cause some inconvenience to your work, I sincerely suggest you consider this request. Implementing this feature could allow Positron to significantly differentiate itself from VSCode, as the latter does not fully account for certain data analysis scenarios.

[e-kotov]

@ZhimingYe For me, the tunnel thing did not work as well as the SSH connection. Additionally, "resuming" (e.g., closing or crashing VSCode and reconnecting to the same session) works with some quirks when using screen (I haven't tried tmux, but I imagine the experience is mostly the same), as it requires a few hoops to make VScode and VScode-R recognise the R session. Also, of course a bit of a hassle to remember to start screen, than start R or Python manually, than attach the session with the corresponding VScode extension.

With these tunnels, you are dependent on Microsoft's infrastructure (or will be dependant on Posit if they were to set-up a similar service), whereas with SSH you are not.

It would be nice if there were more transparent options for working with screen or tmux in Positron (e.g., presets to start a new R/Python terminal through screen/tmux and checking for existing sessions when reconnecting). However, even without these features, remote SSH mode would still be great.

[e-kotov]

@jmcphers thanks, great to see this coming so quickly!

Currently, at least for me it works with some limitations. Specifically, I connect to compute nods in HPC. For that, I either need to establish an SSH tunnel manually, or do a ProxyJump command in my ssh config. With VScode it works either way perfectly, with Positron, only the manual way works.

Let me provide a few details.

Here is a snippet of how my ssh config looks:

Host host1_manual_ssh_tunel
  User user1
  HostName localhost
  Port 4589
  IdentityFile ~/.ssh/id_1

Host host1_proxy_jump
  User user1
  HostName remote_sever_1
  Port 4589
  IdentityFile ~/.ssh/id_1
  ProxyJump [email protected]

1. With ProxyJump

In VScode I just do Remote-SSH: Connect to remote Host... and choose host1_proxy_jump. And it connects. Positron cannot connect.

Here's Positron's Remote SSH log

[Info  - 17:26:25.236] Resolving ssh remote authority 'ssh-remote+host1_proxy_jump' (attemp #1)
[Trace  - 17:26:25.252] Identity keys:
/Users/u1/.ssh/id_1 ssh-rsa SHA256:<MASKED>
[Trace  - 17:26:25.254] Identity keys:
/Users/u1/.ssh/id_1 ssh-rsa SHA256:<MASKED>
[Error  - 17:27:25.261] Error resolving authority
Error: Timed out while waiting for handshake
	at Timeout.<anonymous> (/Applications/Positron.app/Contents/Resources/app/extensions/open-remote-ssh/dist/extension.js:1:164670)
	at listOnTimeout (node:internal/timers:573:17)
	at process.processTimers (node:internal/timers:514:7)

2. With manual SSH tunnel

I first do:

ssh -N -L 4589:remote_sever_1:4589 [email protected]

Then I just connect to host1_manual_ssh_tunel from the Positron's command
Remote-SSH: Connect to remote Host.... This works perfectly.

A few more considerations on remote SSH

Extension Interface

In VScode the remote ssh extension conveniently lists the hosts defined in the ssh config. Perhaps it is the fault of the Open Remote SSH, but Positron does not lists the predefined hosts, which is inconvenient.

Persistent R/Python session

SSH connection can be broken which will kill the R/Python session (I'm not sure it is actually killed, perhaps it is left dangling in the remote server's memory). So when reconnecting via Remote SSH, there is no way to recover the session. In VScode I used to solve that with screen or tmux, then running the R/python from within a screen or tmux. This way I could recover the session. In Positron it is really awesome that we have a dedicated console for R/Python, but there is no way to resume when reconnecting :(

Is there a way to address this somehow?

[testlabauto]

Verified Fixed

Positron Version(s) : 2024.08.0-48
OS Version          : OSX

Test scenario(s)

Connected from OSX to Ubuntu. Saw ports forwarded. Saw positron-server processes running on Ubuntu.

Link(s) to TestRail test cases run or created:
N/A

[sahilseth]

I am trying to connect to a remote host, and the window is stuck at this:

Cant start a console or terminal.

Have the following extensions on the server:

With R 3.6.3 install by default (later versions as rocker images):

[juliasilge]

@sahilseth Can you open a new discussion instead of posting on older, closed issues?

On a first glance, you may want to check out the requirements for Positron, which includes R 4.2 or higher.