执行 nft list ruleset 报警告 #282

Ir1Ka · 2024-09-22T13:58:10Z

执行 nft 看到如下报错。但执行 iptables 和 ip6tables 没有报错，是正常的吗？
命令如下：

sudo nft list ruleset 1>/dev/null
sudo iptables -nvL -t nat
sudo iptables -nvL -t mangle
sudo ip6tables -nvL -t nat
sudo ip6tables -nvL -t mangle

# Warning: table ip nat is managed by iptables-nft, do not touch!
# Warning: table ip mangle is managed by iptables-nft, do not touch!
# Warning: table ip6 mangle is managed by iptables-nft, do not touch!
# Warning: XT target TPROXY not found
# Warning: XT target TPROXY not found
Warning: Extension owner is not supported, missing kernel module?
Warning: Extension owner is not supported, missing kernel module?
Warning: Extension owner is not supported, missing kernel module?
# Warning: table ip6 nat is managed by iptables-nft, do not touch!
# Warning: XT target REDIRECT not found
# Warning: XT target REDIRECT not found
Warning: Extension owner is not supported, missing kernel module?
Warning: Extension owner is not supported, missing kernel module?
# Warning: XT target REDIRECT not found
Warning: Extension owner is not supported, missing kernel module?
Warning: Extension owner is not supported, missing kernel module?
# Warning: XT target REDIRECT not found
# Warning: XT target SNAT not found

环境：
OS: VyOS 1.5-rolling-202409080006
容器：Alpine Linux 3.20.3
ss-tproxy 版本：ss-tproxy v4.8.3 (2024-07-24)

The text was updated successfully, but these errors were encountered:

zfl9 · 2024-09-22T14:05:44Z

提示很明显了，安装缺失的 iptables 内核模块。Google 下有解决方案的。

zfl9 · 2024-09-22T14:13:46Z

另外，在使用 iptables-nft 后端时，不要执行任何 nft 指令（正如给出的警告所说）。

像往常一样，使用 iptables、ip6tables 即可，内核中将自动使用 nftables 作为后端。

zfl9 · 2024-09-22T14:30:00Z

另外，执行 ss-tproxy restart 时是否存在任何报错？

Ir1Ka · 2024-09-22T15:38:38Z

检查了一下已加载内核模块，所有报错相关的模块都以加载：

vyos@Gateway:~$ lsmod | grep xt_
xt_MASQUERADE          16384  0
xt_TPROXY              12288  4
nf_tproxy_ipv6         16384  1 xt_TPROXY
nf_tproxy_ipv4         16384  1 xt_TPROXY
xt_mark                12288  2
xt_connmark            12288  8
xt_set                 20480  4
xt_addrtype            12288  12
xt_nat                 12288  2
xt_conntrack           12288  12
xt_REDIRECT            16384  8
xt_owner               12288  14
xt_tcpudp              16384  0
ip_set                 65536  2 xt_set,ip_set_hash_net
nf_nat                 57344  6 nft_nat,xt_nat,nft_masq,nft_chain_nat,xt_MASQUERADE,xt_REDIRECT
nf_conntrack          192512  10 xt_conntrack,nf_nat,nfnetlink_cthelper,nft_ct,nft_nat,xt_nat,xt_connmark,nft_masq,xt_MASQUERADE,xt_REDIRECT
nf_defrag_ipv6         24576  2 nf_conntrack,xt_TPROXY
nf_defrag_ipv4         12288  2 nf_conntrack,xt_TPROXY
x_tables               61440  13 xt_conntrack,nft_compat,xt_tcpudp,xt_addrtype,xt_nat,xt_owner,xt_set,xt_TPROXY,xt_connmark,ip_tables,xt_MASQUERADE,xt_REDIRECT,xt_mark

我尝试重启下系统

Ir1Ka · 2024-09-22T17:32:55Z

部分运行起来了，目前仅本机的 ipv4 能走透明代理，本机 ipv6、局域网内其他主机 ipv6 暂时没有走代码。并且局域网内的 ipv4 无法工作了。
继续调试。

zfl9 · 2024-09-23T07:59:45Z

先排除 nft 规则的影响，在 ss-tproxy start 之前，nft list ruleset 最好是空的。

Ir1Ka · 2024-09-23T10:15:26Z

但是部署到路由器上的的话，没法保证 ss-tproxy start 前，nft list ruleset 为空呀，只能看看啥原因了

zfl9 · 2024-09-23T10:18:19Z

pre_start 里面执行 nft flush ruleset ?

Ir1Ka · 2024-09-23T10:39:12Z

额，那网关的规则都搞没了呀，虽然能保证 nft 为空，但可能路由器的功能会异常吧

Ir1Ka · 2024-09-24T11:29:50Z

看起来 sudo nft list ruleset 中报的错都来自于 ip6 的表。

看起来这个报错是因为这个：https://forum.kaspersky.com/topic/nft-utility-errors-xt-target-tproxy-not-found-caused-by-wtpntp-task-kes-for-linux-37528/
因为如果我执行 sudo nft list table ip6 nat 和 sudo nft list table ip6 mangle 就没有这个报错。

Ir1Ka · 2024-09-24T17:40:16Z

ipv4 内网访问搞定了，需要放行 fwmark 等于 0x2333 的包。从 log 看到，0x2333 的包被默认丢掉了。
具体原因就是 ss-tproxy 会把需要代理的包通过 fwmark 和路由规则的方式路由到 lo，并进入 input 规则中，并且这些包不带 out if 信息，导致我设置的允许从 oif == lo 的规则没有生效。
通过加入规则通过所有 fwmark == 0x233 的包，ipv4 可以工作了。

Ir1Ka · 2024-09-24T19:34:28Z

加了这几行 log，然后测试 ipv6（内网），命令 curl -6 ip.sb，只打印了如下 log：

Sep 25 03:53:19 kernel: [SSTP_RULE CONNMARK set]IN=eth0 OUT= MAC=24:4b:fe:d4:3d:a0:52:54:00:ed:65:b5:86:dd SRC=xxxx::yyyy DST=2606:4700:0020:0000:0000:0000:681a:0d1f LEN=72 TC=0 HOPLIMIT=64 FLOWLBL=462883 PROTO=TCP SPT=55566 DPT=80 WINDOW=64800 RES=0x00 SYN URGP=0
Sep 25 03:53:19 kernel: [SSTP_PREROUTING tcp]IN=eth0 OUT= MAC=24:4b:fe:d4:3d:a0:52:54:00:ed:65:b5:86:dd SRC=xxxx::yyyy DST=2606:4700:0020:0000:0000:0000:681a:0d1f LEN=72 TC=0 HOPLIMIT=64 FLOWLBL=462883 PROTO=TCP SPT=55566 DPT=80 WINDOW=64800 RES=0x00 SYN URGP=0
Sep 25 03:53:20 kernel: [SSTP_PREROUTING tcp]IN=pppoe0 OUT= MAC= SRC=2606:4700:0020:0000:0000:0000:681a:0d1f DST=xxxx::yyyy LEN=72 TC=0 HOPLIMIT=52 FLOWLBL=333531 PROTO=TCP SPT=80 DPT=55566 WINDOW=65535 RES=0x00 ACK SYN URGP=0
Sep 25 03:53:20 kernel: [SSTP_PREROUTING tcp]IN=eth0 OUT= MAC=24:4b:fe:d4:3d:a0:52:54:00:ed:65:b5:86:dd SRC=xxxx::yyyy DST=2606:4700:0020:0000:0000:0000:681a:0d1f LEN=60 TC=0 HOPLIMIT=64 FLOWLBL=462883 PROTO=TCP SPT=55566 DPT=80 WINDOW=1025 RES=0x00 ACK URGP=0
Sep 25 03:53:20 kernel: [SSTP_PREROUTING tcp]IN=eth0 OUT= MAC=24:4b:fe:d4:3d:a0:52:54:00:ed:65:b5:86:dd SRC=xxxx::yyyy DST=2606:4700:0020:0000:0000:0000:681a:0d1f LEN=129 TC=0 HOPLIMIT=64 FLOWLBL=462883 PROTO=TCP SPT=55566 DPT=80 WINDOW=1025 RES=0x00 ACK PSH URGP=0
Sep 25 03:53:20 kernel: [SSTP_PREROUTING tcp]IN=pppoe0 OUT= MAC= SRC=2606:4700:0020:0000:0000:0000:681a:0d1f DST=xxxx::yyyy LEN=60 TC=0 HOPLIMIT=52 FLOWLBL=333531 PROTO=TCP SPT=80 DPT=55566 WINDOW=8 RES=0x00 ACK URGP=0
Sep 25 03:53:20 kernel: [SSTP_PREROUTING tcp]IN=pppoe0 OUT= MAC= SRC=2606:4700:0020:0000:0000:0000:681a:0d1f DST=xxxx::yyyy LEN=687 TC=0 HOPLIMIT=52 FLOWLBL=333531 PROTO=TCP SPT=80 DPT=55566 WINDOW=8 RES=0x00 ACK PSH URGP=0
Sep 25 03:53:20 kernel: [SSTP_PREROUTING tcp]IN=pppoe0 OUT= MAC= SRC=2606:4700:0020:0000:0000:0000:681a:0d1f DST=xxxx::yyyy LEN=65 TC=0 HOPLIMIT=52 FLOWLBL=333531 PROTO=TCP SPT=80 DPT=55566 WINDOW=8 RES=0x00 ACK PSH URGP=0
Sep 25 03:53:20 kernel: [SSTP_PREROUTING tcp]IN=eth0 OUT= MAC=24:4b:fe:d4:3d:a0:52:54:00:ed:65:b5:86:dd SRC=xxxx::yyyy DST=2606:4700:0020:0000:0000:0000:681a:0d1f LEN=60 TC=0 HOPLIMIT=64 FLOWLBL=462883 PROTO=TCP SPT=55566 DPT=80 WINDOW=1022 RES=0x00 ACK URGP=0
Sep 25 03:53:20 kernel: [SSTP_PREROUTING tcp]IN=eth0 OUT= MAC=24:4b:fe:d4:3d:a0:52:54:00:ed:65:b5:86:dd SRC=xxxx::yyyy DST=2606:4700:0020:0000:0000:0000:681a:0d1f LEN=60 TC=0 HOPLIMIT=64 FLOWLBL=462883 PROTO=TCP SPT=55566 DPT=80 WINDOW=1022 RES=0x00 ACK FIN URGP=0
Sep 25 03:53:20 kernel: [SSTP_PREROUTING tcp]IN=pppoe0 OUT= MAC= SRC=2606:4700:0020:0000:0000:0000:681a:0d1f DST=xxxx::yyyy LEN=60 TC=0 HOPLIMIT=52 FLOWLBL=333531 PROTO=TCP SPT=80 DPT=55566 WINDOW=8 RES=0x00 ACK FIN URGP=0
Sep 25 03:53:20 kernel: [SSTP_PREROUTING tcp]IN=eth0 OUT= MAC=24:4b:fe:d4:3d:a0:52:54:00:ed:65:b5:86:dd SRC=xxxx::yyyy DST=2606:4700:0020:0000:0000:0000:681a:0d1f LEN=60 TC=0 HOPLIMIT=64 FLOWLBL=462883 PROTO=TCP SPT=55566 DPT=80 WINDOW=1022 RES=0x00 ACK URGP=0

log 中，没有任何 matched 的 log，很奇怪。请问可能是什么原因呢？

规则：

# ip6tables -vL --line-numbers -t mangle
Chain PREROUTING (policy ACCEPT 645K packets, 358M bytes)
num   pkts bytes target     prot opt in     out     source               destination
1     645K  358M SSTP_PREROUTING  all  --  any    any     anywhere             anywhere

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 437K packets, 91M bytes)
num   pkts bytes target     prot opt in     out     source               destination
1     437K   91M SSTP_OUTPUT  all  --  any    any     anywhere             anywhere

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain SSTP_OUTPUT (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1    42235 6651K RETURN     all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL
2        0     0 RETURN     all  --  any    any     anywhere             anywhere             ctdir REPLY
3     284K   60M RETURN     all  --  any    any     anywhere             anywhere             owner GID match proxy
4        0     0 RETURN     tcp  --  any    any     anywhere             anywhere             tcp dpt:domain ! owner GID match proxy_dns
5        0     0 RETURN     udp  --  any    any     anywhere             anywhere             udp dpt:domain ! owner GID match proxy_dns
6       24  1920 SSTP_RULE  tcp  --  any    any     anywhere             anywhere             tcp flags:FIN,SYN,RST,ACK/SYN
7        0     0 SSTP_RULE  udp  --  any    any     anywhere             anywhere             ctstate NEW,RELATED
8       20  1594 LOG        tcp  --  any    any     anywhere             anywhere             LOG level warn prefix "[SSTP_OUTPUT tcp]"
9        0     0 LOG        all  --  any    any     anywhere             anywhere             connmark match  0x2333 LOG level warn prefix "[SSTP_OUTPUT matched]"
10       0     0 MARK       all  --  any    any     anywhere             anywhere             connmark match  0x2333 MARK set 0x2333

Chain SSTP_PREROUTING (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1     379K  190M RETURN     all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL
2        0     0 RETURN     all  --  any    any     anywhere             anywhere             ctdir REPLY
3    40198 2894K SSTP_RULE  tcp  --  any    any     anywhere             anywhere             tcp dpt:!domain flags:FIN,SYN,RST,ACK/SYN ADDRTYPE match src-type !LOCAL
4        0     0 SSTP_RULE  udp  --  any    any     anywhere             anywhere             udp dpt:!domain ctstate NEW,RELATED ADDRTYPE match src-type !LOCAL
5      197 37445 LOG        tcp  --  any    any     anywhere             anywhere             LOG level warn prefix "[SSTP_PREROUTING tcp]"
6        0     0 LOG        all  --  any    any     anywhere             anywhere             connmark match  0x2333 LOG level warn prefix "[SSTP_PREROUTING matched]"
7        0     0 TPROXY     tcp  --  any    any     anywhere             anywhere             connmark match  0x2333 TPROXY redirect ::1:60080 mark 0x2333/0xffffffff
8        0     0 TPROXY     udp  --  any    any     anywhere             anywhere             connmark match  0x2333 TPROXY redirect ::1:60080 mark 0x2333/0xffffffff

Chain SSTP_RULE (4 references)
num   pkts bytes target     prot opt in     out     source               destination
1      137  9864 RETURN     all  --  any    any     anywhere             anywhere             match-set sstp_white6 dst ! match-set sstp_black6 dst
2    40085 2886K CONNMARK   all  --  any    any     anywhere             anywhere             CONNMARK set 0x2333
3        5   360 LOG        all  --  any    any     anywhere             anywhere             LOG level warn prefix "[SSTP_RULE CONNMARK set]"

zfl9 · 2024-09-25T02:06:13Z

nft list ruleset 看看全部规则？

建议使用 raw 表的 -j TRACE 看日志，比手动加 LOG 更方便一些。

log 中，没有任何 matched 的 log，很奇怪。请问可能是什么原因呢？

确实很奇怪，中间也没有其他规则干扰，难道是 ipv6 的 connmark 模块有问题？

Ir1Ka · 2024-09-25T09:56:32Z

有点像 ipv6 的 connmark set 0x2333 没有生效

Ir1Ka · 2024-09-25T10:19:46Z

TRACE 需要如何加规则？我用命令 ip6tables -t raw -A PREROUTING -p tcp -j TRACE，然后测试，没有看到人的 trace 的打印。此命令是否有误；

zfl9 · 2024-09-25T10:26:52Z

你测试的本机代理还是局域网代理 ?

我看 ipv4 已经搞定了（本机、局域网）
ipv6 要不先测下本机代理？

先以 tcp 2001:4860:4860::8888 为测试用例，避免 log 过多。

ip6tables -t raw -I OUTPUT -p tcp -d 2001:4860:4860::8888 -j TRACE
ip6tables -t raw -I OUTPUT -p tcp -s 2001:4860:4860::8888 -j TRACE
ip6tables -t raw -I PREROUTING -p tcp -s 2001:4860:4860::8888 -j TRACE
ip6tables -t raw -I PREROUTING -p tcp -d 2001:4860:4860::8888 -j TRACE

然后本机执行 dig +tcp @2001:4860:4860::8888 google.com，观察 dmesg 日志。

Ir1Ka · 2024-09-25T10:49:36Z

测试的 ipv6 局域网代理，本地 ipv6 代理也不工作。均走的直连。
用 tcp dns 应该会转发到 chinadns-ng 吧？

zfl9 · 2024-09-25T10:51:58Z

好像是哈，切换下 group 身份，sg proxy_dns 'dig +tcp @2001:4860:4860::8888 google.com'，跳过 chinadns-ng。

Ir1Ka · 2024-09-25T11:56:56Z

打印的 log 如下，没有观察到 TRACE 的 log，难道是 VyOS 有问题？

Sep 25 19:55:19 kernel: [SSTP_RULE CONNMARK set]IN= OUT=pppoe0 SRC=xxxx::yyyy DST=2001:4860:4860:0000:0000:0000:0000:8888 LEN=80 TC=0 HOPLIMIT=64 FLOWLBL=502873 PROTO=TCP SPT=46469 DPT=53 WINDOW=64660 RES=0x00 SYN URGP=0
Sep 25 19:55:20 kernel: [SSTP_RULE CONNMARK set]IN= OUT=pppoe0 SRC=xxxx::yyyy DST=2001:4860:4860:0000:0000:0000:0000:8888 LEN=80 TC=0 HOPLIMIT=64 FLOWLBL=815264 PROTO=TCP SPT=45921 DPT=53 WINDOW=64660 RES=0x00 SYN URGP=0
Sep 25 19:55:20 kernel: [SSTP_RULE CONNMARK set]IN= OUT=pppoe0 SRC=xxxx::yyyy DST=2001:4860:4860:0000:0000:0000:0000:8888 LEN=80 TC=0 HOPLIMIT=64 FLOWLBL=599366 PROTO=TCP SPT=40217 DPT=53 WINDOW=64660 RES=0x00 SYN URGP=0

规则列表如下：

/ # ip6tables -vL --line-numbers -t raw
# Table `raw' contains incompatible base-chains, use 'nft' tool to list them.
Chain PREROUTING (policy ACCEPT 21365 packets, 17M bytes)
num   pkts bytes target     prot opt in     out     source               destination
1        3   245 TRACE      tcp  --  any    any     anywhere             dns.google
2       97 12843 TRACE      tcp  --  any    any     dns.google           anywhere

Chain OUTPUT (policy ACCEPT 8339 packets, 1241K bytes)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 TRACE      tcp  --  any    any     dns.google           anywhere
2       71  4842 TRACE      tcp  --  any    any     anywhere             dns.google

Chain vyos_global_rpfilter (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 RETURN     all  --  any    any     anywhere             anywhere
/ # ip6tables -vL --line-numbers -t mangle
Chain PREROUTING (policy ACCEPT 720K packets, 388M bytes)
num   pkts bytes target     prot opt in     out     source               destination
1     720K  388M SSTP_PREROUTING  all  --  any    any     anywhere             anywhere

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 495K packets, 100M bytes)
num   pkts bytes target     prot opt in     out     source               destination
1     495K  100M SSTP_OUTPUT  all  --  any    any     anywhere             anywhere

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain SSTP_OUTPUT (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1    47823 7587K RETURN     all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL
2        0     0 RETURN     all  --  any    any     anywhere             anywhere             ctdir REPLY
3     333K   67M RETURN     all  --  any    any     anywhere             anywhere             owner GID match proxy
4       53  3180 RETURN     tcp  --  any    any     anywhere             anywhere             tcp dpt:domain ! owner GID match proxy_dns
5        0     0 RETURN     udp  --  any    any     anywhere             anywhere             udp dpt:domain ! owner GID match proxy_dns
6       41  3280 SSTP_RULE  tcp  --  any    any     anywhere             anywhere             tcp flags:FIN,SYN,RST,ACK/SYN
7        0     0 SSTP_RULE  udp  --  any    any     anywhere             anywhere             ctstate NEW,RELATED
8        0     0 MARK       all  --  any    any     anywhere             anywhere             connmark match  0x2333 MARK set 0x2333

Chain SSTP_PREROUTING (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1     437K  213M RETURN     all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL
2        0     0 RETURN     all  --  any    any     anywhere             anywhere             ctdir REPLY
3    40835 2940K SSTP_RULE  tcp  --  any    any     anywhere             anywhere             tcp dpt:!domain flags:FIN,SYN,RST,ACK/SYN ADDRTYPE match src-type !LOCAL
4        0     0 SSTP_RULE  udp  --  any    any     anywhere             anywhere             udp dpt:!domain ctstate NEW,RELATED ADDRTYPE match src-type !LOCAL
5        0     0 TPROXY     tcp  --  any    any     anywhere             anywhere             connmark match  0x2333 TPROXY redirect ::1:60080 mark 0x2333/0xffffffff
6        0     0 TPROXY     udp  --  any    any     anywhere             anywhere             connmark match  0x2333 TPROXY redirect ::1:60080 mark 0x2333/0xffffffff

Chain SSTP_RULE (4 references)
num   pkts bytes target     prot opt in     out     source               destination
1      144 10368 RETURN     all  --  any    any     anywhere             anywhere             match-set sstp_white6 dst ! match-set sstp_black6 dst
2    40732 2933K CONNMARK   all  --  any    any     anywhere             anywhere             CONNMARK set 0x2333
3      652 47080 LOG        all  --  any    any     anywhere             anywhere             LOG level warn prefix "[SSTP_RULE CONNMARK set]"

zfl9 · 2024-09-25T12:00:57Z

你这个内核怕是有点问题，第一次见这种情况。

会不会是 ipv6 的 conntrack 内核模块未加载？

Ir1Ka · 2024-09-25T12:07:29Z

nft 规则列表如下，移除了隐私相关以及 ipv4 规则。

:~$ sudo nft list ruleset
# Warning: table ip nat is managed by iptables-nft, do not touch!
table inet mangle {
        chain FORWARD {
                type filter hook forward priority mangle; policy accept;
        }
}
table ip6 raw {
        chain VYOS_TCP_MSS {
                type filter hook forward priority raw; policy accept;
                oifname "pppoe0" tcp flags syn / syn,rst tcp option maxseg size set rt mtu
        }

        chain vyos_global_rpfilter {
                return
        }

        chain vyos_rpfilter {
                type filter hook prerouting priority raw; policy accept;
                counter packets 722306 bytes 389764469 jump vyos_global_rpfilter
        }

        chain VYOS_PREROUTING_HOOK {
                type filter hook prerouting priority raw; policy accept;
        }

        chain PREROUTING {
                type filter hook prerouting priority raw; policy accept;
                ip6 daddr 2001:4860:4860::8888 meta l4proto tcp counter packets 3 bytes 245 meta nftrace set 1
                ip6 saddr 2001:4860:4860::8888 meta l4proto tcp counter packets 97 bytes 12843 meta nftrace set 1
        }

        chain OUTPUT {
                type filter hook output priority raw; policy accept;
                ip6 saddr 2001:4860:4860::8888 meta l4proto tcp counter packets 0 bytes 0 meta nftrace set 1
                ip6 daddr 2001:4860:4860::8888 meta l4proto tcp counter packets 71 bytes 4842 meta nftrace set 1
        }
}
table inet vrf_zones {
        map ct_iface_map {
                typeof iifname : ct zone
        }

        chain vrf_zones_ct_in {
                type filter hook prerouting priority raw; policy accept;
        }

        chain vrf_zones_ct_out {
                type filter hook output priority raw; policy accept;
        }
}
# Warning: table ip6 mangle is managed by iptables-nft, do not touch!
table ip6 mangle {
        chain SSTP_PREROUTING {
                fib daddr type local counter packets 437076 bytes 212778279 return
                ct direction reply counter packets 0 bytes 0 return
                tcp flags syn / fin,syn,rst,ack tcp dport != 53 fib saddr type != local counter packets 40839 bytes 2940408 jump SSTP_RULE
                udp dport != 53 ct state new,related fib saddr type != local counter packets 0 bytes 0 jump SSTP_RULE
                meta l4proto tcp ct mark 0x2333 counter packets 0 bytes 0 xt target "TPROXY"
                meta l4proto udp ct mark 0x2333 counter packets 0 bytes 0 xt target "TPROXY"
        }

        chain SSTP_OUTPUT {
                fib daddr type local counter packets 47853 bytes 7590570 return
                ct direction reply counter packets 0 bytes 0 return
                skgid 38888 counter packets 332805 bytes 67225096 return
                tcp dport 53 skgid != 39999 counter packets 53 bytes 3180 return
                udp dport 53 skgid != 39999 counter packets 0 bytes 0 return
                tcp flags syn / fin,syn,rst,ack counter packets 41 bytes 3280 jump SSTP_RULE
                meta l4proto udp ct state new,related counter packets 0 bytes 0 jump SSTP_RULE
                ct mark 0x2333 counter packets 0 bytes 0 meta mark set 0x2333
        }

        chain SSTP_RULE {
                xt match "set" xt match "set" counter packets 144 bytes 10368 return
                counter packets 40736 bytes 2933320 ct mark set 0x2333
                counter packets 656 bytes 47368 log prefix "[SSTP_RULE CONNMARK set]"
        }

        chain PREROUTING {
                type filter hook prerouting priority mangle; policy accept;
                counter packets 723774 bytes 390853563 jump SSTP_PREROUTING
        }

        chain OUTPUT {
                type route hook output priority mangle; policy accept;
                counter packets 495247 bytes 99745222 jump SSTP_OUTPUT
        }
}
# Warning: table ip6 nat is managed by iptables-nft, do not touch!
table ip6 nat {
        chain SSTP_PREROUTING {
                tcp flags syn / fin,syn,rst,ack tcp dport 53 fib saddr type != local counter packets 0 bytes 0 redirect to :5353
                udp dport 53 ct state new fib saddr type != local counter packets 0 bytes 0 redirect to :5353
        }

        chain SSTP_OUTPUT {
                tcp flags syn / fin,syn,rst,ack tcp dport 53 skgid != 38888 skgid != 39999 counter packets 0 bytes 0 redirect to :5353
                udp dport 53 ct state new skgid != 38888 skgid != 39999 counter packets 0 bytes 0 redirect to :5353
        }

        chain SSTP_POSTROUTING {
                ip6 saddr != ::1 ip6 daddr ::1 counter packets 0 bytes 0 snat to ::1
        }

        chain PREROUTING {
                type nat hook prerouting priority dstnat; policy accept;
                counter packets 0 bytes 0 jump SSTP_PREROUTING
        }

        chain OUTPUT {
                type nat hook output priority dstnat; policy accept;
                counter packets 0 bytes 0 jump SSTP_OUTPUT
        }

        chain POSTROUTING {
                type nat hook postrouting priority srcnat; policy accept;
                counter packets 0 bytes 0 jump SSTP_POSTROUTING
        }
}
table ip6 vyos_filter {
        chain VYOS_IPV6_FORWARD_filter {
                type filter hook forward priority filter; policy accept;
                counter packets 8301 bytes 6776849 accept comment "FWD-filter default-action accept"
        }

        chain VYOS_IPV6_INPUT_filter {
                type filter hook input priority filter; policy accept;
                counter packets 3301 bytes 2520406 accept comment "INP-filter default-action accept"
        }

        chain VYOS_IPV6_OUTPUT_raw {
                type filter hook output priority raw; policy accept;
                counter packets 2955 bytes 444963 accept comment "OUT-raw default-action accept"
        }

        chain VYOS_IPV6_OUTPUT_filter {
                type filter hook output priority filter; policy accept;
                counter packets 2955 bytes 444963 accept comment "OUT-filter default-action accept"
        }

        chain VYOS_IPV6_PREROUTING_raw {
                type filter hook prerouting priority raw; policy accept;
                counter packets 11604 bytes 9311787 accept comment "PRE-raw default-action accept"
        }

        chain VYOS_FRAG6_MARK {
                type filter hook prerouting priority -450; policy accept;
                exthdr frag exists meta mark set 0x000ffff1 return
        }
}
table bridge vyos_filter {
        chain VYOS_FORWARD_filter {
                type filter hook forward priority filter; policy accept;
                counter packets 0 bytes 0 accept comment "FWD-filter default-action accept"
        }

        chain VYOS_INPUT_filter {
                type filter hook input priority filter; policy accept;
                counter packets 0 bytes 0 accept comment "INP-filter default-action accept"
        }

        chain VYOS_OUTPUT_filter {
                type filter hook output priority filter; policy accept;
                counter packets 0 bytes 0 accept comment "OUT-filter default-action accept"
        }

        chain VYOS_PREROUTING_filter {
                type filter hook prerouting priority filter; policy accept;
                counter packets 0 bytes 0 accept comment "PRE-filter default-action accept"
        }
}
table ip6 vyos_mangle {
        chain VYOS_PBR6_PREROUTING {
                type filter hook prerouting priority mangle; policy accept;
        }

        chain VYOS_PBR6_POSTROUTING {
                type filter hook postrouting priority mangle; policy accept;
        }
}

Ir1Ka · 2024-09-25T12:08:15Z

你这个内核怕是有点问题，第一次见这种情况。

会不会是 ipv6 的 conntrack 内核模块未加载？

有可能，还需要排除 VyOS 本身的问题。

zfl9 · 2024-09-25T12:08:28Z

lsmod | grep conntrack 看看有哪些

Ir1Ka · 2024-09-25T12:09:22Z

执行结果

:~$ lsmod | grep conntrack
xt_conntrack           12288  12
nf_conntrack          192512  9 xt_conntrack,nf_nat,nfnetlink_cthelper,nft_ct,nft_nat,xt_nat,xt_connmark,nft_masq,xt_REDIRECT
nf_defrag_ipv6         24576  2 nf_conntrack,xt_TPROXY
nf_defrag_ipv4         12288  2 nf_conntrack,xt_TPROXY
x_tables               61440  14 xt_conntrack,nft_compat,xt_TRACE,xt_LOG,xt_tcpudp,xt_addrtype,xt_nat,xt_owner,xt_set,xt_TPROXY,xt_connmark,ip_tables,xt_REDIRECT,xt_mark

zfl9 · 2024-09-25T12:16:10Z

看规则是没有问题的，ss-tproxy 中的 iptables 和 ip6tables 代码是复用的（同一个函数）。

根据上面一系列反馈，核心问题还是 ip6 的 connmark set 未生效（不知道是这个模块问题还是内核配置少了什么）。

Ir1Ka · 2024-09-28T11:14:34Z

IPv6 代理失效（connmark set 没有工作）有没有可能是流量被硬件 offload 了？

Ir1Ka · 2024-09-29T12:00:53Z

重装了一下 VyOS，重新做了所有配置，新增了 IPv6 的防火墙规则，目前本地及内网设备的 IPv6 和 IPv4 代理均已正常工作。

还需要确认之前 IPv6 代理不能工作的原因，以避免以后 IPv6 代理失效。

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

执行 nft list ruleset 报警告 #282

执行 nft list ruleset 报警告 #282

Ir1Ka commented Sep 22, 2024 •

edited

Loading

zfl9 commented Sep 22, 2024

zfl9 commented Sep 22, 2024 •

edited

Loading

zfl9 commented Sep 22, 2024

Ir1Ka commented Sep 22, 2024 •

edited

Loading

Ir1Ka commented Sep 22, 2024

zfl9 commented Sep 23, 2024

Ir1Ka commented Sep 23, 2024

zfl9 commented Sep 23, 2024

Ir1Ka commented Sep 23, 2024

Ir1Ka commented Sep 24, 2024 •

edited

Loading

Ir1Ka commented Sep 24, 2024 •

edited

Loading

Ir1Ka commented Sep 24, 2024 •

edited

Loading

zfl9 commented Sep 25, 2024 •

edited

Loading

Ir1Ka commented Sep 25, 2024

Ir1Ka commented Sep 25, 2024

zfl9 commented Sep 25, 2024 •

edited

Loading

Ir1Ka commented Sep 25, 2024

zfl9 commented Sep 25, 2024

Ir1Ka commented Sep 25, 2024 •

edited

Loading

zfl9 commented Sep 25, 2024 •

edited

Loading

Ir1Ka commented Sep 25, 2024

Ir1Ka commented Sep 25, 2024

zfl9 commented Sep 25, 2024

Ir1Ka commented Sep 25, 2024

zfl9 commented Sep 25, 2024

Ir1Ka commented Sep 28, 2024

Ir1Ka commented Sep 29, 2024

执行 nft list ruleset 报警告 #282

执行 nft list ruleset 报警告 #282

Comments

Ir1Ka commented Sep 22, 2024 • edited Loading

zfl9 commented Sep 22, 2024

zfl9 commented Sep 22, 2024 • edited Loading

zfl9 commented Sep 22, 2024

Ir1Ka commented Sep 22, 2024 • edited Loading

Ir1Ka commented Sep 22, 2024

zfl9 commented Sep 23, 2024

Ir1Ka commented Sep 23, 2024

zfl9 commented Sep 23, 2024

Ir1Ka commented Sep 23, 2024

Ir1Ka commented Sep 24, 2024 • edited Loading

Ir1Ka commented Sep 24, 2024 • edited Loading

Ir1Ka commented Sep 24, 2024 • edited Loading

zfl9 commented Sep 25, 2024 • edited Loading

Ir1Ka commented Sep 25, 2024

Ir1Ka commented Sep 25, 2024

zfl9 commented Sep 25, 2024 • edited Loading

Ir1Ka commented Sep 25, 2024

zfl9 commented Sep 25, 2024

Ir1Ka commented Sep 25, 2024 • edited Loading

zfl9 commented Sep 25, 2024 • edited Loading

Ir1Ka commented Sep 25, 2024

Ir1Ka commented Sep 25, 2024

zfl9 commented Sep 25, 2024

Ir1Ka commented Sep 25, 2024

zfl9 commented Sep 25, 2024

Ir1Ka commented Sep 28, 2024

Ir1Ka commented Sep 29, 2024

Ir1Ka commented Sep 22, 2024 •

edited

Loading

zfl9 commented Sep 22, 2024 •

edited

Loading

Ir1Ka commented Sep 22, 2024 •

edited

Loading

Ir1Ka commented Sep 24, 2024 •

edited

Loading

Ir1Ka commented Sep 24, 2024 •

edited

Loading

Ir1Ka commented Sep 24, 2024 •

edited

Loading

zfl9 commented Sep 25, 2024 •

edited

Loading

zfl9 commented Sep 25, 2024 •

edited

Loading

Ir1Ka commented Sep 25, 2024 •

edited

Loading

zfl9 commented Sep 25, 2024 •

edited

Loading