WSL 2.0: networkingMode=mirrored makes Docker unable to forward ports

[halimsamy]

Windows Version

Microsoft Windows [Version 10.0.22631.2338]

WSL Version

2.0.0.0

Are you using WSL 1 or WSL 2?

• WSL 2

Kernel Version

5.15.123.1-1

Distro Version

Ubuntu 20.04.6

Other Software

Docker version 24.0.6, build ed223bc

Repro Steps

• Change networkingMode to mirrored
• docker run -d -p 8080:80 nginx:alpine (example)
• Go to localhost:8080
• Cannot connect
• Try from WSL curl http//localhost:8080 same issue

Expected Behavior

To forward the port and be able to connect to my containers

Actual Behavior

Doesn't forward the port, so I cannot connect to my containers.

Diagnostic Logs

No response

[driver1998]

Interestingly, the port forwarding does work from another machine on the same network as host. Just not on the host machine itself.

Machine A:

• Change networkingMode to mirrored
• docker run -d -p 8080:80 nginx:alpine
• Go to localhost:8080
• Cannot connect
• Set Windows firewall rules to allow 8080 inbound
• Go to localhost:8080
• Cannot connect

Machine B:

• Go to A:8080
• works

[greenhat616]

Same issue

---

Maybe related issues:
docker/for-win#13686

[giovannicandido]

Same problem here

[lengthofrope]

Yup, it's unfortunate but I have the same issue. I am on the release channel with windows version 10.0.22621.2359

[over-star]

Same problem here

[speller]

In my case, Docker ports are not exposed to the host Windows machine at all after the upgrade to 2.0 even if I set networkingMode=NAT which is the default value. I can't see opened ports in the tcpview.exe utility. Previously, I was able to see them opened by the wslhost.exe process (or similar name).

Switching to NAT helped me.

[Yukari316]

same here

[tuxiaobei-scu]

+1

[bkfino]

+1

[ptr1120]

+1

[halimsamy]

@benhillis Any ideas/updates here?

[DukeCrimson]

same here

[tumugin]

+1

[Aerglonus]

Same issue here, the only way I found to make it "work" was adding ignoredPorts=8080 to the wslconfig, but if container has something like 4800:8080 need to add both ports to ignoredPorts=8080,4800 still doing this some containers might not work at all, also if cloudflare tunnel is running on WSL this will break to when mirrored and theres no way to know which ports is using to add it to the ignoredPorts

[halimsamy]

Same issue here, the only way I found to make it "work" was adding ignoredPorts=8080 to the wslconfig, but if container has something like 4800:8080 need to add both ports to ignoredPorts=8080,4800 still doing this some containers might not work at all, also if cloudflare tunnel is running on WSL this will break to when mirrored and theres no way to know which ports is using to add it to the ignoredPorts

I think that's a workaround that would work... but I am waiting for someone from the WSL team to answer us here, did they get to know that is the issue, and if they have any plans to fix this?

[lengthofrope]

Same issue here, the only way I found to make it "work" was adding ignoredPorts=8080 to the wslconfig, but if container has something like 4800:8080 need to add both ports to ignoredPorts=8080,4800 still doing this some containers might not work at all, also if cloudflare tunnel is running on WSL this will break to when mirrored and theres no way to know which ports is using to add it to the ignoredPorts

That seems to work. Nice find.

[zcobol]

ignorePorts is misleading. What it does is making in this case port 8080 available to request inside the container only, but is not forwarded to Windows side. Look for more details at https://learn.microsoft.com/en-us/windows/wsl/wsl-config

If you run get-nettcpConnection -LocalPort 8080 there's nothing listening on that port.

After starting the web server with docker run -d -p 8080:80 nginx:alpine requests inside the WSL container works:

zcobol@debian:~$ curl -I localhost:8080
HTTP/1.1 200 OK
Server: nginx/1.25.2
Date: Mon, 25 Sep 2023 16:41:36 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Tue, 15 Aug 2023 19:24:07 GMT
Connection: keep-alive
ETag: "64dbd0d7-267"
Accept-Ranges: bytes

Requests from Windows side timeout with:

curl: (28) Failed to connect to localhost port 8080 after 21276 ms: Couldn't connect to server

From another server it work, as mentioned by @driver1998 :

tux@raspi:~$ curl -I 192.168.1.105:8080
HTTP/1.1 200 OK
Server: nginx/1.25.2
Date: Mon, 25 Sep 2023 16:43:45 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Tue, 15 Aug 2023 19:24:07 GMT
Connection: keep-alive
ETag: "64dbd0d7-267"
Accept-Ranges: bytes

[mew1033]

Just ran into this as well. AWS sam build --use-container doesn't work. Gives the same error

 Ports are not available: exposing port TCP 127.0.0.1:5232 -> 0.0.0.0:0: listen tcp 127.0.0.1:5232: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.

(The port number changes randomly every time)

Based on this: https://github.com/aws/aws-sam-cli/blob/c5b9b1e399a1e5c938ef72934a14ede934e17bac/samcli/local/docker/container.py#L124-L125
I added every port from 5000-9000 to the ignoredPorts list. It's ugly, but for now it works.

[halimsamy]

Just ran into this as well. AWS sam build --use-container doesn't work. Gives the same error

 Ports are not available: exposing port TCP 127.0.0.1:5232 -> 0.0.0.0:0: listen tcp 127.0.0.1:5232: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.

(The port number changes randomly every time)

Based on this: https://github.com/aws/aws-sam-cli/blob/c5b9b1e399a1e5c938ef72934a14ede934e17bac/samcli/local/docker/container.py#L124-L125 I added every port from 5000-9000 to the ignoredPorts list. It's ugly, but for now it works.

It seems like it's generally a problem with WSL itself. I am looking for a fix soon, since the new network mode is so much useful but it's not useable (usefully) in the current state.

[imacarthur741]

I ma having the same issue with apache. No changes other than added --experimental but now nothing works.

(98)Address already in use: AH00072: make_sock: could not bind to address [::]:80
(98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
[no listening sockets available, shutting down](apache2.service: Control process exited, code=exited, status=1/FAILURE)

root@ACER-Nitro:/usr/sbin# lsof -nP -iTCP -sTCP:LISTEN
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
systemd-r 308 systemd-resolve 14u IPv4 22507 0t0 TCP 127.0.0.53:53 (LISTEN)
mysqld 385 mysql 21u IPv4 32886 0t0 TCP 127.0.0.1:33060 (LISTEN)
mysqld 385 mysql 23u IPv4 27396 0t0 TCP 127.0.0.1:3306 (LISTEN)
miniserv. 1867 root 5u IPv4 52863 0t0 TCP *:10000 (LISTEN)

[zohaibhassan156]

Same problem

[zohaibhassan156]

ignoredPorts=8025,1025 work for me too. I had to use mailpit in docker

docker run -d \
--name=mailpit \
--restart unless-stopped \
-p 8025:8025 \
-p 1025:1025 \
axllent/mailpit

[shigenobuokamoto]

there seem to be two issues why Docker containers cannot connect from Windows.

1. ignorePorts works
   Docker Desktop, probably

2. the port forwarding does work from another machine on the same network as host. Just not on the host machine itself.
   docker-ce that installed on Linux

temporary measures for 2 ....
disable iptables and use docker-proxy

/etc/docker/daemon.json

{
    "iptables": false
}

when using mirrored, the behavior seems to be different from the previous localhostforwarding.

use docker-proxy(listen on Linux)

windows  --->  linux(WSL)
  localhostforwarding (until)
ok    127.0.0.1   --->     127.0.0.1(lo)
ok    127.0.0.1   <---     127.0.0.1(lo)

  netowrkingMode=mirrored
ok    127.0.0.1   --->     127.0.0.1(loopback0)
ok    127.0.0.1   <---     127.0.0.1(loopback0)

interface is different, but the behavior remains the same.

use iptables(listeon on container)

windows  --->  linux(WSL) ---> Docker container
  localhostforwarding (until)
ok    127.0.0.1   ---> 127.0.0.1(lo)    /   172.x.x.1(br-xxx)  --->  172.x.x.x(eth0)
ok    127.0.0.1   <--- 127.0.0.1(lo)    /   172.x.x.1(br-xxx)  <---  172.x.x.x(eth0)

  netowrkingMode=mirrored
ok    127.0.0.1   ---> 127.0.0.1(loopback0)/127.0.0.1(br-xxxx) --->  172.x.x.x(eth0)
                                          ~~~~~~~~~
ng                                                           <- - -  172.x.x.x(eth0)

via localhostforwarding(until), source address(Windows) was the docker network gateway (=pointing to linux).

via mirrored, source address is 127.0.0.1.
for this reason that packet is returned to 127.0.0.1(localhost on container) and does not reach Windows.
in the case of access from another node(PC), there is no problem because source is his address.

[mwwhited]

Hi folks, we believe we've found a fix for this that needs to be added to the Docker engine. We've filed a bug for it and are working with the docker team to investigate adding it in: moby/moby#48056

As a note, this is not just docker. Even without docker and resetting WSL entirely this still occurs. mirrored mode does not work correctly with regular WSL images such as Ubuntu.

[felipecrs]

@mwwhited I would recommend you open a separate issue.

This one is specifically about Docker.

[mwwhited]

There are several here all describing the same issue. Opening more is not going to fix it any better.

[felipecrs]

But I can tell you this issue will very likely be closed as soon as the Docker CE issue is resolved.

[mwwhited]

As already noted above I have cross mentioned the issue to try to get an actual fix instead of just a pile of work arounds.
#11600

especially as these workarounds do not work for everyone

[jweaston]

Reference

Well it looks like the issue wasn't actually fixed for me. Not sure what I did or what changed to fix and then unfix the issue.

Also when running the sudo nft -a list chain ip nat DOCKER I get a new rule for each port I forward in the container.

table ip nat {
chain DOCKER { # handle 10
iifname "docker0" counter packets 0 bytes 0 return # handle 29
iifname "br-c360e1083765" counter packets 65 bytes 3981 return # handle 13
iifname != "br-c360e1083765" tcp dport 8085 counter packets 0 bytes 0 dnat to 172.19.0.2:54002 # handle 30
iifname != "br-c360e1083765" tcp dport 10001 counter packets 0 bytes 0 dnat to 172.19.0.2:10000 # handle 32
iifname != "br-c360e1083765" tcp dport 9001 counter packets 0 bytes 0 dnat to 172.19.0.2:9001 # handle 34
iifname != "br-c360e1083765" udp dport 1024 counter packets 0 bytes 0 dnat to 172.19.0.2:1024 # handle 36
}
}

Where it looks like the c360e1083765 is the ID of the container. Not sure if this is different because I'm using the docker engine over docker desktop.

[shigenobuokamoto]

although this issue occurs with Docker, i believe it is a compatibility issue at WSL.
the following parameters set by mirrored networking are unexpected from Docker.

net.ipv4.conf.loopback0.route_localnet = 1

my nftables rules partially cancels the effect of this parameter.
Docker can run even with the above parameters set, but we will encounter another issue where the Windows host cannot route to the networks added inside WSL.

if the Windows host could reflect the routing to the network inside WSL, the fourth point raised by @itispip would be easily solved.

[felipecrs]

I just tested WSL 2.3.11 and unfortunately the problem remains there.

[keith-horton]

@felipecrs : right - 2.3.1.1 fixes a variety of Linux container scenarios. This in particular will require Docker to change their route - tracked here: moby/moby#48056.

We are in contact with them and the fix should hopefully be released soon.

[shigenobuokamoto]

@keith-horton
(https://github.com/microsoft/WSL/releases/tag/2.3.11)[WSL 2.3.11]

Update mirrored mode loopback routing to support connections between Windows host and Linux Docker/Podman containers (solves #11136,#11468,#11758,#10926)

this improvement gives us more options.

in the daemon.json

"userland-proxy": false

this now works as well.

but

moby/moby#48075

this change is made, 'userland-proxy: false' may no longer work.
i think it needs a little adjustment.

[shigenobuokamoto]

this may seem unnecessary since moby takes care of it, but this is a new improvement plan.

WSL 2.3.11 brings several improvements to communication with Windows host.

therefore, you can now use the following rules to improve communication from Windows host to Docker containers:

$ nft add rule ip nat WSLPOSTROUTING iif "loopback0" ip saddr 127.0.0.0/8 ip daddr != 127.0.0.0/8 counter masquerade
$ sysctl -w net.ipv4.conf.all.route_localnet=1

[codeart1st]

@shigenobuokamoto with WSL pre-release 2.3.11 would you still recommend your systemd script workaround?

https://gist.github.com/shigenobuokamoto/b565d468541fc8be7d7d76a0434496a0

[felipecrs]

@shigenobuokamoto with WSL pre-release 2.3.11 would you still recommend your systemd script workaround?

https://gist.github.com/shigenobuokamoto/b565d468541fc8be7d7d76a0434496a0

As an user, I can it still works fine.

[shigenobuokamoto]

@codeart1st
work on this issue is ongoing in moby (=Docker Engine), so until that is released you can use my script to mitigate the issue.
essentially, the issue is that network within WSL behaves differently from normal Linux, so i think it would be best to deal with it on the WSL side.

WSL 2.3.11 includes some improvements to mirrored networking, so i am looking into how to use them to improve the experience. this is the prototype new network-mirrored.service. please try this too.

https://gist.github.com/shigenobuokamoto/540c5f09a03eb07149501e99a6c8d82b

• taking advantage of the fact that connect from Docker container network to Windows host is now possible, i wrote a more natural nftables.
• added support for communication between Windows and Linux using 127.0.0.0/8, so i worte the missing route.

[felipecrs]

the issue is that network within WSL behaves differently from normal Linux, so i think it would be best to deal with it on the WSL side.

Also, this would mean that whatever application that relies on the same feature as docker would have the same issue. It could be an old application that doesn't receive more support for example. Or some application that refuses to implement a fix just for WSL.

So, yeah, I totally agree. The ideal would have been the fix to happen on WSL side.

[Parsifa1]

When will this fix be built into wsl? Now we can only use service script to make it work, I don't think this is a long term solution.

Besides, I use nixos-wsl, this is my example module that can be used as a reference for nixos user, from @shigenobuokamoto

{pkgs, ...}: {
    systemd.services.network-mirrored = {
      description = "network-mirrored";
      enable = true;
      wants = ["network-pre.target"];
      wantedBy = ["multi-user.target"];
      before = ["network-pre.target" "shutdown.target"];
      serviceConfig = {
        User = "root";
        ExecStart = [
          ''
            /bin/sh -ec '\
            [ -x /usr/bin/wslinfo ] && [ "$(/usr/bin/wslinfo --networking-mode)" = "mirrored" ] || exit 0;\
            echo "\
            add chain   ip nat WSLPREROUTING { type nat hook prerouting priority dstnat - 1; policy accept; };\
            insert rule ip nat WSLPREROUTING iif loopback0  ip daddr 127.0.0.1 counter dnat to 127.0.0.1 comment mirrored;\
            "|${pkgs.nftables}/bin/nft -f -\
            '
          ''
        ];

        ExecStop = [
          ''
            /bin/sh -ec '\
              [ -x /usr/bin/wslinfo ] && [ "$(/usr/bin/wslinfo --networking-mode)" = "mirrored" ] || exit 0;\
              for chain in "ip nat WSLPREROUTING";\
              do\
                handle=$(${pkgs.nftables}/bin/nft -a list chain $chain | sed -En "s/^.*comment \\"mirrored\\" # handle ([0-9]+)$/\\1/p");\
                for n in $handle; do echo "delete rule $chain handle $n"; done;\
              done|${pkgs.nftables}/bin/nft -f -\
            '
          ''
        ];
        RemainAfterExit = "yes";
      };
    };
}

[zaaack]

here is a really dirty solution, I just wrote a nodejs tcp reverse proxy server to map the docker port to another port, so we can using another port because its created by a non-docker process.

import net from 'net'
let map = {
  15432: 5432,
  13000: 3000,
}
for (const from in map) {
  const to = map[from]
  net
    .createServer((server) => {
      const client = net.createConnection(to)
      server.pipe(client)
      client.pipe(server)
    })
    .listen(from)
    .on('listening', () => {
      console.log(`tcp-proxy: ${from} -> ${to}`)
    })
    .on('error', (err) => {
      console.error(err)
    })
}

[felipecrs]

@zaaack, if you are looking for a temporary solution, this one is the best you'll find: #10494 (comment)

[zaaack]

@zaaack, if you are looking for a temporary solution, this one is the best you'll find: #10494 (comment)

thanks, but my wsl is older and doesnt set systemd

[dannyhpy]

thanks, but my wsl is older and doesnt set systemd

@zaaack, you can use iptables -t nat -I DOCKER -i loopback0 -d 127.0.0.0/8 -j RETURN¹ or any of the variants² that have been mentioned above.

[jweaston]

Has anyone had success configuring WSL to allow connections from containers in WSL to the windows host?
I made some attempts on my own to forward used ports, 5966 in this case, from WSL to the windows host through the nft but was unsuccesfull.
Maybe some one can tell me what I did wrong.

sudo nft add chain ip nat prerouting { type nat hook prerouting priority 0 \; }
sudo nft add chain ip nat postrouting { type nat hook postrouting priority 100 \; }
sudo nft add rule ip nat prerouting tcp dport 5966 counter dnat to 127.0.0.1:5966
sudo nft add rule ip nat postrouting ip saddr 172.0.0.0/8 counter masquerade

[shigenobuokamoto]

@jweaston
you can access from Docker container in WSL to Windows host address, WSL 2.3.11 and later.

.wslconfig

[experimental]
hostAddressLoopback=true

Windows host                  WSL Linux                         Docker Container
127.0.0.1    --- loopback --- 127.0.0.1   --- not connected --- 127.0.0.1(same address but unrelated)
192.168.1.10 --- loopback --- 192.168.1.10
                                  | routed
     (unknown network)        172.17.0.1  ----- connected ----- 172.17.0.2

docker container (in WSL) can connect to 192.168.1.10:5966.
if you try connect to 127.0.0.1:5966, the packet is sent to container's 127.0.0.1. so cannot connect to windows host.

[CatalinFetoiu]

hello. Docker completed the PR that fixes this issue - moby/moby#48514
This was included in the following Docker release - https://github.com/moby/moby/releases/tag/v27.3.0

Please try Docker version 27.3.0

[felipecrs]

It works in my test. :)

[unclemusclez]

My favorite flavor of Linux is Ubuntu via Docker via Ubuntu via WSL via Windows 11.