[BUG] Client connects to server but unable to tunnel connection when started as server mode on Docker Engine >= v23 on macOS #355

puttyman · 2024-09-16T06:27:59Z

Is there an existing issue for this?

I have searched the existing issues

Current Behavior

On an Intel Mac OSX install the latest docker desktop 4.34.2 at time of this issue. Latest version has docker engine v27.
Start the a VPN Server container as per the documentation.
On a mobile phone scan the QR code.
Connect to the VPN on the client mobile phone.
Try to navigate to Google.com
Notice that the client successfully connects to the Server but the connection timeout.

Expected Behavior

The client should be able to tunnel its connection via the VPN server.

Steps To Reproduce

On an Intel Mac OSX install the latest docker desktop 4.34.2 at time of this issue. Latest version has docker engine v27.
Start the a VPN Server container as per the documentation.
On a mobile phone scan the QR code.
Connect to the VPN on the client mobile phone.
Try to navigate to Google.com
Notice that the client successfully connects to the Server but the connection timeout.
Stop the VPN server container.
Install Docker Desktop v4.18.0 / Docker Engine V20 . See: https://gist.github.com/kupietools/2f9f085228d765da579f0f0702bec33c
Start the a VPN Server container as per the documentation.
On a mobile phone scan the QR code.
Connect to the VPN on the client mobile phone.
Try to navigate to Google.com
Notice that the client successfully connects to the Server and now is able to navigate to Google.com

Environment

- OS: Mac OSX Sonoma
- How docker service was installed: Via Manual dmg download of Docker Desktop

CPU architecture

x86-64

Docker creation

docker run -d \
  --name=mobile-test-proxy \
  --cap-add=NET_ADMIN \
  --cap-add=SYS_MODULE \
  -e PUID=502 \
  -e PGID=20 \
  -e TZ=Melbourne/Australia \
  -e SERVERURL=10.0.0.80 \
  -e PEERS=2 \
  -e PEERDNS=10.0.10.1 \
  -e LOG_CONFS=true \
  -p 51820:51820/udp \
  -v /Users/mee/projects/mobile-vpn/config:/config \
  --sysctl="net.ipv4.conf.all.src_valid_mark=1" \
  --restart unless-stopped \
  linuxserver/wireguard

Container logs

2024-09-16 15:37:49 ───────────────────────────────────────
2024-09-16 15:37:49 GID/UID
2024-09-16 15:37:49 ───────────────────────────────────────
2024-09-16 15:37:49 
2024-09-16 15:37:49 User UID:    502
2024-09-16 15:37:49 User GID:    20
2024-09-16 15:37:49 ───────────────────────────────────────
2024-09-16 15:37:49 Linuxserver.io version: 1.0.20210914-r4-ls51
2024-09-16 15:37:49 Build-date: 2024-09-12T11:23:25+00:00
2024-09-16 15:37:49 ───────────────────────────────────────
2024-09-16 15:37:49     
2024-09-16 15:37:49 Uname info: Linux a4097bdb9bb1 5.15.49-linuxkit #1 SMP Tue Sep 13 07:51:46 UTC 2022 x86_64 GNU/Linux
2024-09-16 15:37:49 **** It seems the wireguard module is already active. Skipping kernel header install and module compilation. ****
2024-09-16 15:37:49 **** As the wireguard module is already active you can remove the SYS_MODULE capability from your container run/compose. ****
2024-09-16 15:37:49 ****     If your host does not automatically load the iptables module, you may still need the SYS_MODULE capability.     ****
2024-09-16 15:37:49 **** Server mode is selected ****
2024-09-16 15:37:49 **** External server address is set to 10.0.0.80 ****
2024-09-16 15:37:49 **** External server port is set to 51820. Make sure that port is properly forwarded to port 51820 inside this container ****
2024-09-16 15:37:49 **** Internal subnet is set to 10.13.13.0 ****
2024-09-16 15:26:13 [#] ip link add wg0 type wireguard
2024-09-16 15:26:13 [#] wg setconf wg0 /dev/fd/63
2024-09-16 15:26:13 [#] ip -4 address add 10.13.13.1 dev wg0
2024-09-16 15:26:13 [#] ip link set mtu 65455 up dev wg0
2024-09-16 15:26:13 [#] ip -4 route add 10.13.13.3/32 dev wg0
2024-09-16 15:26:13 [#] ip -4 route add 10.13.13.2/32 dev wg0
2024-09-16 15:26:13 [#] iptables -A FORWARD -i wg0 -j ACCEPT; iptables -A FORWARD -o wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth+ -j MASQUERADE
2024-09-16 15:35:05 [#] ip link delete dev wg0
2024-09-16 15:35:06 [#] iptables -D FORWARD -i wg0 -j ACCEPT; iptables -D FORWARD -o wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth+ -j MASQUERADE
2024-09-16 15:37:50 [#] ip link add wg0 type wireguard
2024-09-16 15:37:50 [#] wg setconf wg0 /dev/fd/63
2024-09-16 15:37:50 [#] ip -4 address add 10.13.13.1 dev wg0
2024-09-16 15:37:50 [#] ip link set mtu 1420 up dev wg0
2024-09-16 15:37:50 [#] ip -4 route add 10.13.13.3/32 dev wg0
2024-09-16 15:37:50 [#] ip -4 route add 10.13.13.2/32 dev wg0
2024-09-16 15:37:50 [#] iptables -A FORWARD -i wg0 -j ACCEPT; iptables -A FORWARD -o wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth+ -j MASQUERADE
2024-09-16 15:37:49 **** AllowedIPs for peers 0.0.0.0/0, ::/0 ****
2024-09-16 15:37:49 **** Peer DNS servers will be set to 10.247.16.1 ****
2024-09-16 15:37:49 **** Server mode is selected ****
2024-09-16 15:37:49 **** No changes to parameters. Existing configs are used. ****
2024-09-16 15:37:49 [custom-init] No custom files found, skipping...
2024-09-16 15:37:49 .:53
2024-09-16 15:37:49 CoreDNS-1.11.1
2024-09-16 15:37:49 linux/amd64, go1.22.5, 
2024-09-16 15:37:50 **** Found WG conf /config/wg_confs/wg0.conf, adding to list ****
2024-09-16 15:37:50 **** Activating tunnel /config/wg_confs/wg0.conf ****
2024-09-16 15:37:50 **** All tunnels are now active ****
2024-09-16 15:37:50 [ls.io-init] done.

github-actions · 2024-09-16T06:28:21Z

Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid.

fskaeh · 2024-09-16T20:50:23Z

Same issue here on a Mac Mini M1 running Orbstack 1.7.2, but in client mode.
The initial tunnel setup seems to go fine, but then there's no way to reach any host from inside the container.

[mod-init] Running Docker Modification Logic
[mod-init] Adding gabe565/linuxserver-mod-vuetorrent to container
[mod-init] Downloading gabe565/linuxserver-mod-vuetorrent from ghcr.io
[mod-init] Installing gabe565/linuxserver-mod-vuetorrent
[mod-init] gabe565/linuxserver-mod-vuetorrent applied to container
[migrations] started
[migrations] no migrations found
───────────────────────────────────────

      ██╗     ███████╗██╗ ██████╗
      ██║     ██╔════╝██║██╔═══██╗
      ██║     ███████╗██║██║   ██║
      ██║     ╚════██║██║██║   ██║
      ███████╗███████║██║╚██████╔╝
      ╚══════╝╚══════╝╚═╝ ╚═════╝

   Brought to you by linuxserver.io
───────────────────────────────────────

To support the app dev(s) visit:
WireGuard: https://www.wireguard.com/donations/

To support LSIO projects visit:
https://www.linuxserver.io/donate/

───────────────────────────────────────
GID/UID
───────────────────────────────────────

User UID:    501
User GID:    20
───────────────────────────────────────
Linuxserver.io version: 1.0.20210914-r4-ls51
Build-date: 2024-09-12T11:23:25+00:00
───────────────────────────────────────
    
Uname info: Linux 12c31b6beec7 6.10.7-orbstack-00280-gd3b7ec68d3d4 #4 SMP Mon Sep  2 03:51:24 UTC 2024 aarch64 GNU/Linux
**** It seems the wireguard module is already active. Skipping kernel header install and module compilation. ****
**** As the wireguard module is already active you can remove the SYS_MODULE capability from your container run/compose. ****
****     If your host does not automatically load the iptables module, you may still need the SYS_MODULE capability.     ****
**** Client mode selected. ****
[custom-init] No custom files found, skipping...
**** Disabling CoreDNS ****
**** Found WG conf /config/wg_confs/wg0.conf, adding to list ****
**** Activating tunnel /config/wg_confs/wg0.conf ****
Warning: `/config/wg_confs/wg0.conf' is world accessible
[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 10.70.36.162/32 dev wg0
[#] ip -6 address add fc00:bbbb:bbbb:bb01::7:24a1/128 dev wg0
[#] ip link set mtu 1420 up dev wg0
[#] wg set wg0 fwmark 51820
[#] ip -6 route add ::/0 dev wg0 table 51820
[#] ip -6 rule add not fwmark 51820 table 51820
[#] ip -6 rule add table main suppress_prefixlength 0
[#] ip6tables-restore -n
[#] ip -4 route add 0.0.0.0/0 dev wg0 table 51820
[#] ip -4 rule add not fwmark 51820 table 51820
[#] ip -4 rule add table main suppress_prefixlength 0
[#] iptables-restore -n
**** All tunnels are now active ****
[ls.io-init] done.

j0nnymoe · 2024-09-16T21:37:20Z

Unfortunately MacOS isn't a platform we test our containers on. While for the most part most of our containers should work on different OS's using docker desktop/orbstack, it's not something we provide primary support for: https://docs.linuxserver.io/misc/support-policy/#reasonable-endeavours-support .

thespad · 2024-09-16T21:38:43Z

As an aside you both need to check the output of wg show in the server container and confirm that there are current handshakes for the client, otherwise it means that the traffic is never reaching the server.

fskaeh · 2024-09-17T20:27:42Z

@thespad There's one hanshake at the start of the container, and new handshakes seemingly around every 2-ish minutes:

interface: wg0
  public key: ***=
  private key: (hidden)
  listening port: 32818
  fwmark: 0xca6c

peer: ***=
  endpoint: [***]:51820
  allowed ips: 0.0.0.0/0, ::/0
  latest handshake: 1 minute, 24 seconds ago
  transfer: 4.88 MiB received, 969.48 KiB sent

a couple minutes later:

interface: wg0
  public key: ***=
  private key: (hidden)
  listening port: 32818
  fwmark: 0xca6c

peer: ***=
  endpoint: [***]:51820
  allowed ips: 0.0.0.0/0, ::/0
  latest handshake: 12 seconds ago
  transfer: 5.02 MiB received, 1.13 MiB sent

At the moment of typing this, I'm even able to ping google.com from inside the container eventhough I made no change whatsoever either to the wg config or to my docker setup. I'm suspecting it may be a Mullvad issue because the connection loss seems to be intermittent. I'll keep an eye on it and report back.

EDIT: by the time I was done typing this, the issue happened again, I now cannot ping any host from inside the container 🤔

strk1204 · 2024-09-24T13:16:54Z

Just wanted to chime in and add that I'm having the same issue with a similar setup, albeit this is impacting all VPN providers (public and self-hosted.) I get about 2-4 minutes of throughput if I'm lucky, then handshakes stop. Rolling back the container to an older version seems to fix this for now.

I don't see any errors, with the only sign found in wg show, with the last handshake ever increasing.

fskaeh · 2024-10-04T13:18:00Z

Just wanted to chime in and add that I'm having the same issue with a similar setup, albeit this is impacting all VPN providers (public and self-hosted.) I get about 2-4 minutes of throughput if I'm lucky, then handshakes stop. Rolling back the container to an older version seems to fix this for now.

I don't see any errors, with the only sign found in wg show, with the last handshake ever increasing.

Could you let me know which tag you used to get it working please? I went back quite a few versions but had no such luck. 🫠

puttyman changed the title ~~[BUG] Client tunnel to VPN Server (Running as server mode) does not work when using Docker Engine >= v23 on Mac OSX~~ [BUG] Client connects to server but unable to tunnel connection when started as server mode on Docker Engine >= v23 on Mac OSX Sep 16, 2024

puttyman changed the title ~~[BUG] Client connects to server but unable to tunnel connection when started as server mode on Docker Engine >= v23 on Mac OSX~~ [BUG] Client connects to server but unable to tunnel connection when started as server mode on Docker Engine >= v23 on macOS Sep 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] Client connects to server but unable to tunnel connection when started as server mode on Docker Engine >= v23 on macOS #355

[BUG] Client connects to server but unable to tunnel connection when started as server mode on Docker Engine >= v23 on macOS #355

puttyman commented Sep 16, 2024 •

edited

Loading

github-actions bot commented Sep 16, 2024

fskaeh commented Sep 16, 2024 •

edited

Loading

j0nnymoe commented Sep 16, 2024

thespad commented Sep 16, 2024

fskaeh commented Sep 17, 2024 •

edited

Loading

strk1204 commented Sep 24, 2024

fskaeh commented Oct 4, 2024

[BUG] Client connects to server but unable to tunnel connection when started as server mode on Docker Engine >= v23 on macOS #355

[BUG] Client connects to server but unable to tunnel connection when started as server mode on Docker Engine >= v23 on macOS #355

Comments

puttyman commented Sep 16, 2024 • edited Loading

Is there an existing issue for this?

Current Behavior

Expected Behavior

Steps To Reproduce

Environment

CPU architecture

Docker creation

Container logs

github-actions bot commented Sep 16, 2024

fskaeh commented Sep 16, 2024 • edited Loading

j0nnymoe commented Sep 16, 2024

thespad commented Sep 16, 2024

fskaeh commented Sep 17, 2024 • edited Loading

strk1204 commented Sep 24, 2024

fskaeh commented Oct 4, 2024

puttyman commented Sep 16, 2024 •

edited

Loading

fskaeh commented Sep 16, 2024 •

edited

Loading

fskaeh commented Sep 17, 2024 •

edited

Loading