Support TCP forwarding

[chipsenkbeil]

There are situations where TCP forwarding is needed. For SSH, this is a jump host. I've run into this situation where only certain ports are open on one machine, but I need to access a companion machine that. E.g. I'm on laptop, connect to server 1 on remote network, and then use server 1 to access server 2 to perform work.

First thought is to have jumping be baked into the request header where we include a sequence of addresses to jump through. With each submission, we peel off the first address in the list and then attempt to send data to that server.

This means that servers need to support establishing connections with other servers, which is a bit tricky. We need to both support forwarding authentication between a series of servers, and map addresses to already-established connections. In addition, how long should we keep a connection open?

This further blends the line of a manager and a server.

[jeremyjjbrown]

Can I help?

[chipsenkbeil]

@jeremyjjbrown thanks for the offer 😄 Do you have experience with Rust? I can provide code pointers and my thoughts on how this could be done, but it may not be the easiest initial contribution to the project.

I did some research a couple of weeks back on this topic (TCP forwarding) as well as supporting the equivalent of ssh ProxyCommand or ProxyJump.

Supporting TCP Forwarding

1. Update the request and response data structures in distant-core/data.rs to support opening a TCP connection, writing bytes to the connection, receiving bytes from the connection, and being able to close the connection. This would mirror the ProcSpawn (open TCP connection), ProcStdin (write bytes to TCP connection), ProcKill (close TCP connection), and ProcStdout (receive bytes from TCP connection) messages.
2. Add an interface for servers to support TCP forwarding to distant-core/api.rs
3. Implement the server-side logic for distant in distant-core/api/local.rs
4. Mark as unsupported server-side logic for ssh in distant-ssh2/api.rs (our SSH library, wezterm-ssh, doesn't support TCP forwarding yet... we'll probably need to implement and contribute it upstream to get support here)
5. Implement a client-side API method in distant-core/client/ext.rs (most likely mirroring distant-core/client/process.rs)
6. Add a command (e.g. distant proxy) to the CLI in src/cli/commands/client.rs to open a TCP proxy channel, listen on a local TCP port, and forward traffic between them
   • We would run in the foreground by default, so running distant proxy -p 8080 host.example.com:12345 would use our existing remote connection to connect to host.example.com:12345 for TCP traffic and make that available on local port 8080
   • We need to provide a --daemon option in the same way that distant manager listen and distant server listen provide one so it can fork/spawn this proxy command to run in the background

Supporting equivalent of SSH JumpHost

1. We need to have the above TCP forwarding implemented first.
2. We'd use TCP forwarding to open a local port on our machine to the remote distant (or ssh) server.
   • In this case, we don't care what port is opened, so we'd want to use an ephemeral port.
3. Finally, we'd run the programmatic equivalent of distant connect user@localhost:12345 where 12345 is whatever local port was opened in the earlier step.

It would be nice if the manager new that the connection was a proxy, but that isn't a requirement to get this to work.

ChatGPT Explanation of ProxyJump and ProxyCommand

For fun, here is ChatGPT's explanation of these commands to paint a picture of how SSH does it!

When you use ProxyJump (or ProxyCommand with the -W option), here's what happens:

1. Your local machine establishes an SSH connection to the first jump host. This connection includes a TCP forward from a local port on your machine to the target port on the next host in the chain. This connection stays open in the background.
2. Your local machine then makes a second SSH connection to the local port that's being forwarded through the first connection. This second connection is routed through the first connection and comes out on the next host in the chain, effectively making an SSH connection from the first host to the second.
3. If there are more hosts in the chain, this process repeats: each connection forwards a local port through to the next host, and then another connection is made to that forwarded port to establish the next link in the chain.
4. The final connection in the chain is the one you actually interact with. This connection can use any SSH feature, like port forwarding or SCP file transfers, and as far as it's concerned, it's a direct connection to the target host. But under the hood, its traffic is being routed through the series of connections and port forwards that were set up by ProxyJump.

So to answer your original question, the local machine is the one that's setting up these background connections and port forwards. The intermediate hosts don't need to set up any proxies or do anything special; they just need to allow SSH connections and TCP forwarding. The ProxyJump option handles all the details of setting up the connections and port forwards.

ChatGPT Explanation of SSH Proxy vs SSH Tunneling

SSH proxy and SSH tunneling are terms that are often used interchangeably, but they can refer to different techniques based on the context. They both involve using SSH to create a secure connection for sending data, but they can be used in different ways.

SSH Tunneling

SSH Tunneling, also known as SSH port forwarding, is a method of transporting arbitrary networking data over an encrypted SSH connection. It can be used to forward individual ports (Local and Remote Port Forwarding) or to create a VPN-like tunnel (Dynamic Port Forwarding).

• Local Port Forwarding: This is used to forward a port from the client machine to the server machine. The client connects to a port on its own machine, and the SSH client forwards the connection over the secure tunnel to connect to a specified address and port on the server side or any other machine the server can reach.
• Remote Port Forwarding: This is the opposite of local port forwarding. It's used to forward a port from the server machine to the client machine.
• Dynamic Port Forwarding: This sets up your SSH connection as a SOCKS proxy which can then be used for proxying all your traffic over the SSH connection.

SSH Proxy

SSH Proxy usually refers to using an SSH server as an intermediary for SSH connections to a third machine (ProxyCommand, ProxyJump). This can be used for "jumping" through one or more intermediary servers to reach the final destination. This is what the -J option does in SSH.

When it comes to ProxyCommand, it provides a way of using an arbitrary command to connect to a remote server, which could be another SSH command, netcat (nc), or anything else that connects standard input and output to a network socket. It's a more flexible but more complex way of achieving the same result as -J.

So, in summary, the main difference is that SSH tunneling is usually used to secure non-SSH traffic by transporting it over an SSH connection, while an SSH proxy is used to route SSH connections through one or more intermediary servers. However, both techniques involve creating secure connections for sending data, and they can often be used to achieve similar goals.

Resources

• ProxyJump StackExchange Explanation
• Using ProxyCommand vs ProxyJump
• OpenSSH Specifications

[chipsenkbeil]

Also, supporting a reverse proxy would be a bit different to this. We'd probably need two more message types being received by the client:

1. An indication that a new connection is being made
2. An indication that a connection has closed

And then the client would need some way to send and receive data based on specific connections. We could probably reuse whatever equivalents we made for ProcStdin and ProcStdout for the TCP forwarding solution.

[jeremyjjbrown]

I have a little experience in Rust. But I am a professional C++, C, Go and Python developer.

I am also interested in this as a part of our router stack at my job might do well in Rust and I thought maybe this would motivate me.

Let me look through the details you provided. If I feel like I grok what you saying overall then I can try some little hacks in that direction. Maybe it goes somewhere. If not I'll take care not to waste your time.

[chipsenkbeil]

@jeremyjjbrown I can handle questions if you have them. I just have other items that are higher priority for me to implement right now. 😄 So don't be shy if you need clarity or want to propose an implementation!