Re-open #2138 - WSL hangs on rsync/ssh

[bunnie]

Windows Version

Microsoft Windows [Version 10.0.22621.3085]

WSL Version

2.0.9.0

Are you using WSL 1 or WSL 2?

• WSL 2
• WSL 1

Kernel Version

Linux version 4.4.0-19041-Microsoft ([email protected]) (gcc version 5.4.0 (GCC) ) #3996-Microsoft Thu Jan 18 16:36:00 PST 2024

Distro Version

Ubuntu 20.04

Other Software

Copying via ssh/rsync to another WSL instance, this one is WSL 2.0.9.0 on windows 10.0.19045.3996

Repro Steps

I run a script to rsync several files from one computer to another:

rsync --log-file=log.log -aiv --delete [email protected]:/mnt/c/dir .

The target computer is running sshd, with public key authentication (Ed25519-only).

Expected Behavior

The rsync process should run to completion.

Actual Behavior

After a couple of minutes (so, after transferring ~ few GiB or thousands of files -- I have seen it fail with few large files, and also fail with thousands of small files), the rsync process hangs. This is evidenced by:

• Network traffic going from ~1Gbps to ~0
• rsync process consuming no more CPU
• rsync process still visible in the process table

The work-around I have for this, and have been using for years now, is to spawn in parallel a script that runs this:

while killall -CHLD ssh; do sleep 0.1; done

However, as recently as last month I forgot to spawn that and had an incomplete rsync several hours later (I seem to recall the rsync actually terminated with an error eventually, without copying all the files).

The prior issue #2138 has been open for a while and I have been holding out hoping there would be a fix for this, someday. The "keep killing SSH" helper process is a viable work-around but if I forget to run it, things fail.

Would really like my rsync to be more reliable...or at least have some satisfying explanation of root cause and why this might be specific just to rsync and not indicative of structural unsoundness inside WSL leading to race conditions.

Diagnostic Logs

No response

[github-actions]

Hi I'm an AI powered bot that finds similar issues based off the issue title.

Please view the issues below to see if they solve your problem, and if the issue describes your problem please consider closing this one and thumbs upping the other issue to help us prioritize it. Thank you!

Open similar issues:

• WSL Hangs on launch (dupe / re-open of #6980 with logs) (#7096), similarity score: 0.73
• SSH connections hanging from WSL2 (#4690), similarity score: 0.73

Closed similar issues:

• rsync daemon unable to bind in WSL2 (#5222), similarity score: 0.73
• wsl command hangs (will not start) (#9568), similarity score: 0.72
• wsl opens and closes immediately (#6242), similarity score: 0.71

Note: You can give me feedback by thumbs upping or thumbs downing this comment.

[bunnie]

I do think that the root cause of this is probably exactly what's causing #4690, since rsync relies on ssh, but there has been no satisfactory answer to that either, and I'm not willing to let #2138 close without resolution simply because it's been ignored for a year by Microsoft.

I suspect a lot of people have been experiencing this problem and probably just running the work-arounds mentioned in the issues hoping there might be a more durable fix to this someday...but I'm happy to be proven wrong.

[nvsystems]

If WSL1 is EOL please close this as WONTFIX. If WSL1 is still supported please finally fix this bug!

It's not that hard to reproduce and has been reported by SO MANY people over close to seven years.

[OneBlue]

@bunnie: Can you capture an strace of rsync and its children (strace -f ) to see what the processes get stuck on ?

[bunnie]

It's re-opened at #11157. Did not realize issues automatically close like this after a week. It's been a busy week on my side, plus CNY etc.

Must be nice to come back after Christmas break and have all your issues closed because authors aren't actively updating their tickets...