Merge Jool into nftables

[ydahhrk]

This has been mentioned several times, particularly in #140 and the survey. It appears that some people want Jool to merge with the Linux kernel.

Why, though? I get that people don't like installing from source, but wouldn't official packages fix that satisfactorily?

Personally, I sympathize with the Unix philosophy, and feel that IP translation is a tad too specific a need and too far from core to expect a kernel to ship with it by default. Wouldn't most distributions disable it?

[CodeFetch]

NAT64 keeps state information etc. It is hard to get it merged into mainline kernel. NAT46 can be easily merged as a virtual network device and will unlikely change much over the years. The kernel janitors would always keep it up to date. Thus Jool could concentrate on NAT64 and e.g. bpfilter support or whatever is coming. If Jool's NAT64 would also become a virtual network device and the design is conform with the kernel's (e.g. using the upcoming rhashtables etc.), it would also be well-suited for the mainline kernel, but it's a long journey to that point...

[ydahhrk]

(e.g. using the upcoming rhashtables etc.)

Question: Where do you hear about this kind of stuff? LWN.net?

Not sure if Jool has strayed far from the kernel's design. I probably need to update my understanding of it.

[CodeFetch]

I'm learning Linux kernel development for some years now, because I'd like to add some functionalities to mac80211, which is quite hard (MCCA). Thus I've read all the O'Reilly books and indeed a lot of LWN articles.

Actually I'm still quite bad at it. At the moment I'm working e.g. on a heavily modified NAT4(2)6 virtual network layer 2 device which does connection tracking to allow seamless roaming for clients in Babel mesh networks. I held a talk on the Battlemesh about it, but it's a bit outdated. The setup has become even more complex as I don't want TCP connections to break and don't rely on intelligent client behavior which means the module needs to do connection tracking to fake a single gateway.

That's why I've talked to some kernel devs about how a kernel module needs to look and "feel" like to be merged in mainline kernel. A simple, RFC-conform NAT46 virtual network device which is NAPI and net-utils compatible would very likely be accepted upstream.

[ydahhrk]

(Not much of an update, but I want to merge the two conversations and respond this e-mail at the same time.)

There's nothing stopping me from knocking some doors upstream, other than the higher-priority feature requests still awaiting implementation. That said, if I attempted such a thing, I'd expect to run into some obstacles:

1. Whenever a new feature is added to the kernel, I understand it needs to be justified, and I'm completely incompetent as a debater. I think it would be difficult to justify Jool's inclusion, particularly considering it can be (and currently is being) packaged. IP translation has a niche audience still and Jool might be too large to be worth shipping by default on every Debian, for example.
2. nftables and device driver modes are probably necessary modernizations, but they're not implemented yet. (Though the way the survey is going, this isn't going to remain an obstacle for long.)
3. @CodeFetch suggests (above) that SIIT and MAP-T would be likely accepted into the kernel, but NAT64 would not. Merging SIIT and MAP-T while leaving NAT64 as a module is more work.
4. The code has a lot of comments in some places. Kernel devs abhor that :p

Addressing 2, 3 and 4 is a matter of time, but I would like to hear some arguments to defeat 1.

For starters, here's a technical reason why merging Jool with the kernel would be a good idea:

1. lowest-ipv6-mtu would be far easier to implement and substantially more performant if kernel modules were allowed to inform Linux's fragmentator a maximum fragment size. (Sort of an MTU associated with a packet instead of an interface.) If Jool were merged into Linux, the latter would probably be tempted to include such a feature.

[telmich]

(Not much of an update, but I want to merge the two conversations and respond this e-mail at the same time.)

There's nothing stopping me from knocking some doors upstream, other than the higher-priority feature requests still awaiting implementation. That said, if I attempted such a thing, I'd expect to run into some obstacles:

1. Whenever a new feature is added to the kernel, I understand it needs to be justified, and I'm completely incompetent as a debater. I think it would be difficult to justify Jool's inclusion, particularly considering it can be (and currently is being) packaged. IP translation has a niche audience still and Jool might be too large to be worth shipping by default on every Debian, for example.

I can help in this regard. I've been involved a bit with the RSBAC discussion in the early 2000s (and the problems around it) and seen the wireguard approach.
It will certainly take some time and probably some modifications to fit into current kernel design/code standards, but this should not pose a real problem, I assume.

2. nftables and device driver modes are probably necessary modernizations, but they're not implemented yet. (Though the way the survey is going, this isn't going to remain an obstacle for long.)

That sounds very promising!

3. @CodeFetch suggests (above) that SIIT and MAP-T would be likely accepted into the kernel, but NAT64 would not. Merging SIIT and MAP-T while leaving NAT64 as a module is more work.

I think this is subject to discussion and maybe a talk with the netfilter group is a good thing to do. If you want to, I can reach out and
start the general discussion on the netfilter mailing list to hear about the sentiment about jool going upstream.

4. The code has a lot of comments in some places. Kernel devs abhor that :p

Addressing 2, 3 and 4 is a matter of time, but I would like to hear some arguments to defeat 1.

For starters, here's a technical reason why merging Jool with the kernel would be a good idea:

1. lowest-ipv6-mtu would be far easier to implement and substantially more performant if kernel modules were allowed to inform Linux's fragmentator a maximum fragment size. (Sort of an MTU associated with a packet instead of an interface.) If Jool were merged into Linux, the latter would probably be tempted to include such a feature.

I see a lot of convincing arguments for merging it into mainline:

• IPv6 is getting significantly more exposure
• NAT64 is getting more required / will be a default thing to do, along with MAP-E/T
• OpenBSD already has the functionality in pf
• Enabling it upstream can potentially help IPv6 migration world wide

If 2/3/4 are kind of easy from your side, I can support a bit with 1 and reach out slowly and get an understanding of how the general attitude is towards jool in upstream.

[ydahhrk]

I think this is subject to discussion and maybe a talk with the netfilter group is a good thing to do. If you want to, I can reach out and
start the general discussion on the netfilter mailing list to hear about the sentiment about jool going upstream.

Ok. I'm currently writing a message I should post on the Netfilter Developer Mailing List tomorrow.

From the archives it seems there have been a couple of attempts to raise a bit of hype about NAT64 many years ago, but they seem to have been unsuccessful.

So feel free to chime in, or even beat me to it. I can't imagine my message being persuasive on its own.

[ydahhrk]

https://marc.info/?l=netfilter-devel&m=157807496904032&w=2

[ordex]

hey @ydahhrk it seems that the mail thread you posted in your last comment just stopped with no further action.
Is the idea of merging with nftables still on the table? what are your thoughts? I'd like to help with this issue, but I am not sure about the current direction that project wants to take.

Personally I believe that having a NAT64 implementation in the kernel would be a great idea, especially if it could just sit next to other NAT'ing mechanism existing in the kernel.

[ydahhrk]

Merging Jool into nftables is still very much the ideal/intended path, but I can't start yet because we're critically understaffed at the moment. Right now I'm charged with the maintenance of both Jool and Fort, as well as the development of half of a new internal project. It's pretty bad. Even Jool 4.2.0 is half a year overdue. I don't know when things will settle down, and I can't offer much right now aside from maintenance and support.

…

On Sat, Jul 10, 2021, 11:06 AM Antonio Quartulli ***@***.***> wrote: hey @ydahhrk <https://github.com/ydahhrk> it seems that the mail thread you posted in your last comment just stopped with no further action. Is the idea of merging with nftables still on the table? what are your thoughts? I'd like to help with this issue, but I am not sure about the current direction that project wants to take. Personally I believe that having a NAT64 implementation in the kernel would be a great idea, especially if it could just sit next to other NAT'ing mechanism existing in the kernel. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#273 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AASHNF2CPMVAPKRNPA6W6C3TXBV2HANCNFSM4GR4VOVQ> .

[tiagogaspar8]

Hi everyone!

@ydahhrk Great work! I'm sorry to hear you're understaffed and under lots of work, hope things got better in the meantime.
I just wanted to let you know that Openwrt is starting the transition to nftables for it's next release (which should be right around the corner, about a month maybe). jool is one of the packages that should migrate to nftables and I was thinking about doing that but I have now found out that this seems to be an actual impossible task without modifying netfilter (which the core OpenWRT devs won't accept).
So I'm here to let you know of this situation and let you know that if you need any help (if I can help of course) just let me know!
Since it's been almost a year since your last update and not wanting to bother I'd also wanted to ask for an update of the status.

Hope you had great festivities and want to wish you a great year! 😄

[ydahhrk]

The situation has done nothing but worsen over time. To be perfectly honest, I (and I speak for myself only) would like to sunset the project at this point, but the higher ups won't allow it. Then again, there are no resources, so I don't know what to tell you.

If somebody would be willing to pay me a salary to work on this, I suppose it could be pushed, but otherwise, I don't see it happening.

Although, if the problem is simply that OpenWRT is about to drop the iptables glue code, I could release a Jool variant that lacks the iptables dependency. Or make it optional. (Maybe it already is; I don't remember.) You'd be forced to use Netfilter mode, but otherwise, nothing would change.

[tiagogaspar8]

Really? Oh wow, that's very sad to hear, this project implements a important feature like no other...

As you might guess I can't do that, I wish I could but I can't 😢

Yes, that's the issue, OpenWRT is dropping the old iptables and moving to nftables. That would be great cause that way I can update the package and it would continue working in the future! I'm not actually sure if actually is, but if the logic is right, if the netfilter mode can work with both iptables and nftables, then removing iptables as a dependency might actually work right? It might be worth it to add a compile flag or check to see if iptables exists and disallow it's use.
Well, I'll leave the implementation to you, but if you could do that it would be great!

Really, thanks for your work, and I hope that in the future things get better, we're all needing it 😄

[myxal]

Let me also express frustrations at this news, as I consider this the old financing-open-source-projects problem rearing its head once again. I'm also in no position to employ someone, but would love to chip in and help get this feature to mainline.

@ydahhrk have you (not just you personally, but the organisation in general) considered organising Kickstarter or some similar fundraiser?

Back in 2018, Bootlin (this was around the same time they changed their name from Free Electrons, so news articles might reference the old name) set up a Kickstarter in a somewhat similar situation - they had code that was working on either deprecated APIs (VDPAU), some initial reversing work done by community, and binary blob from a non-cooperating hardware vendor targeting at that point a long-EOLed kernel versions. They found enough people and companies who wanted a proper mainline driver for the hardware and were willing to put their money into the project. Over the course of a year or two, they succeeded in the goals they set up - driver was developed for the new V4L2 API, made to run with kodi player (where the team also pushed some changes, I think), driver pushed to the kernel before being accepted.

• News article from the announcement: https://linuxgizmos.com/project-crowdfunds-effort-to-bring-allwinner-vpu-support-to-the-linux-kernel/
• Their blog posts detailing the progress: https://bootlin.com/blog/tag/kickstarter/

[ordex]

Have you considered to ask NLnet to fund this project (well at least $some development)? https://nlnet.nl/
It may well fit their scope.

[telmich]

I have to second everyone on this list that Jool has become a very important project. As I see others are also thinking about chipping in: I am working for ungleich[0] in Switzerland, which is registered ltd. I'm open to put 100$/month to support Jool development and it would be easy to collect other payments in Switzerland and send them in one batch to Alberto. At no cost. [1] I know there are other projects to support monthly payments and I'm also open to use that, depending on what's easiest for Alberto. So, who else is in? Best regards, Nico [0] www.ungleich.ch, ungleich glarus ag [1] No cost = ungleich passes it through, minus banking fees. Alberto Leiva Popper ***@***.***> writes:

…

The situation has done nothing but worsen over time. To be perfectly honest, I (and I speak for myself only) would like to sunset the project at this point, but the higher ups won't allow it. Then again, there are no resources, so I don't know what to tell you. If somebody would be willing to pay me a salary to work on this, I suppose it could be pushed, but otherwise, I don't see it happening. Although, if the problem is simply that OpenWRT is about to drop the iptables glue code, I could release a Jool variant that lacks the iptables dependency. Or make it optional. (Maybe it already is; I don't remember.) You'd be forced to use Netfilter mode, but otherwise, nothing would change.

-- Sustainable and modern Infrastructures by ungleich.ch

[ordex]

@telmich I know you're pretty active in the RIPE community - maybe you could circulate this idea there and see which other company would be willing to help?
On top of that, I know RIPE had a call for funding projects not long ago and jool would have probably been a good candidate. Maybe there is still a chance for late applications?

[ydahhrk]

Status: Sorry, but I'm too busy right now. Will work on the optional iptables dependency in the weekend, and figure out a plan for nftables over the second half of January.

[ydahhrk]

Ok, iptables should now be optional in the optional-iptables branch.

Is it possible to test it in the iptablesless OpenWRT without me generating a release?

During the configure step:

./autogen.sh
./configure
make

It detects whether the libxtables-dev (or equivalent) dependency is installed, and disables iptables if it's not.

The kernel module is a bit different:

cd src/mod
make

Compiles with iptables enabled, and

cd src/mod
make JOOL_FLAGS=-DXTABLES_DISABLED

compiles with iptables disabled.

Is this enough?

[tiagogaspar8]

Hi @ydahhrk

I'm really gonna need some pointers here as I believe the way OpenWRT compiles jool is different from the way you "officially" support. Ignoring the source of the code (I have yet to change the link, checksum, etc.) there are two issues I'm facing:

• Where to put the JOOL_FLAGS=-DXTABLES_DISABLED section in the makefile
• Will the build system detect that iptables is not present? I believe since it isn't a dependency, when building jool iptables will be absent and jool will be built without iptables support, is that correct?
• There is a section in the original makefile install section that copies libxt_XXXX.so files, are this needed in the nftables version?

This is the make file I came up with:

#
# Copyright (C) 2016-2017 Dan Luedtke <[email protected]>
#
# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.

include $(TOPDIR)/rules.mk
include $(INCLUDE_DIR)/kernel.mk

PKG_NAME:=jool
PKG_VERSION:=4.1.6
PKG_RELEASE:=$(AUTORELEASE)

PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz
PKG_SOURCE_URL:=https://github.com/NICMx/Jool/releases/download/v$(PKG_VERSION)
PKG_HASH:=dccfde88e3b97a7d1c44c53b6541b74873a04f50bb8cfd187eb399d5ae8d2c81

PKG_MAINTAINER:=
PKG_LICENSE:=GPL-2.0-only
PKG_LICENSE_FILES:=COPYING

PKG_INSTALL:=1
PKG_BUILD_PARALLEL:=1
PKG_BUILD_DEPENDS:=!USE_GLIBC:argp-standalone

include $(INCLUDE_DIR)/package.mk

define Build/Compile
	$(MAKE) -C "$(LINUX_DIR)" \
		KERNEL_DIR="$(LINUX_DIR)" \
		ARCH="$(LINUX_KARCH)" \
		CROSS_COMPILE="$(TARGET_CROSS)" \
		M="$(PKG_BUILD_DIR)/src/mod/common" \
		V="$(V)" \
		modules
	$(MAKE) -C "$(LINUX_DIR)" \
		KERNEL_DIR="$(LINUX_DIR)" \
		ARCH="$(LINUX_KARCH)" \
		CROSS_COMPILE="$(TARGET_CROSS)" \
		M="$(PKG_BUILD_DIR)/src/mod/nat64" \
		V="$(V)" \
		modules
	$(MAKE) -C "$(LINUX_DIR)" \
		KERNEL_DIR="$(LINUX_DIR)" \
		ARCH="$(LINUX_KARCH)" \
		CROSS_COMPILE="$(TARGET_CROSS)" \
		M="$(PKG_BUILD_DIR)/src/mod/siit" \
		V="$(V)" \
		modules
	$(call Build/Compile/Default)
endef

--------------------------------------------------------------------------------package defaults
define Package/jool/Default
  SECTION:=net
  CATEGORY:=Network
  URL:=https://www.jool.mx
  DEPENDS:=+kmod-jool +libnl
endef

define Package/jool/Default/description
  Jool is an Open Source SIIT and NAT64 for Linux.
endef

--------------------------------------------------------------------------------kernel defaults
define KernelPackage/jool/Default
  SECTION:=kernel
  CATEGORY:=Kernel modules
  SUBMENU:=Network Support
  TITLE:=Jool kernel module
  DEPENDS:= \
    @IPV6 \
    +kmod-crypto-md5 \
    +kmod-nf-conntrack \
    +kmod-nf-conntrack6
  FILES:= \
    $(PKG_BUILD_DIR)/src/mod/common/jool_common.$(LINUX_KMOD_SUFFIX) \
    $(PKG_BUILD_DIR)/src/mod/nat64/jool.$(LINUX_KMOD_SUFFIX) \
    $(PKG_BUILD_DIR)/src/mod/siit/jool_siit.$(LINUX_KMOD_SUFFIX)
  JOOL_AUTOLOAD:= \
    jool \
    jool_siit
  AUTOLOAD:=$(call AutoLoad,46,jool_common)
  AUTOLOAD:=$(call AutoLoad,48,$(JOOL_AUTOLOAD))
endef


define KernelPackage/jool/Default/description
  $(call Package/jool/Default/description)

  This package provides the kernel module for Jool.
endef

--------------------------------------------------------------------------------kernel iptables
define KernelPackage/jool-iptables
	$(call KernelPackage/jool/Default)
  DEPENDS:= \
		+kmod-nf-ipt
endef


define KernelPackage/jool-iptables/description
  $(call KernelPackage/jool/Default/description)

  This package provides the kernel module for Jool with iptables support.
endef

--------------------------------------------------------------------------------kernel nftables
define KernelPackage/jool-nftables
	$(call KernelPackage/jool/Default)
  DEPENDS:= \
		+kmod-nft-core
endef


define KernelPackage/jool-nftables/description
  $(call KernelPackage/jool/Default/description)

  This package provides the kernel module for Jool with nftables support.
endef

--------------------------------------------------------------------------------package iptables
define Package/jool-tools-iptables
  $(call Package/jool/Default)
  TITLE:=Jool userspace control programs (iptables)
  DEPENDS:=+libxtables
endef

define Package/jool-tools-iptables/description
  $(call Package/jool/Default/description)

  This package provides the userspace control programs for Jool with iptables support.
endef

--------------------------------------------------------------------------------package nftables
define Package/jool-tools-nftables
  $(call Package/jool/Default)
  TITLE:=Jool userspace control programs- (nftables)
  DEPENDS:=+lib_nftnl
endef

define Package/jool-tools-nftables/description
  $(call Package/jool/Default/description)

  This package provides the userspace control programs for Jool with nftables support.
endef


CONFIGURE_ARGS += \
	--disable-shared \
	--without-bash-completion-dir

CONFIGURE_VARS += \
	ac_cv_func_memset=yes \
	ac_cv_func_strcasecmp=yes

JOOL_AUTOLOAD:= \
	jool_common \
	jool \
	jool_siit

define Package/jool-tools/Default
	$(INSTALL_DIR) $(1)/usr/bin/
	$(INSTALL_BIN) $(PKG_INSTALL_DIR)/usr/bin/jool		$(1)/usr/bin/
	$(INSTALL_BIN) $(PKG_INSTALL_DIR)/usr/bin/joold		$(1)/usr/bin/
	$(INSTALL_BIN) $(PKG_INSTALL_DIR)/usr/bin/jool_siit	$(1)/usr/bin/

	$(INSTALL_DIR) $(1)/etc/config
	$(INSTALL_CONF) ./files/jool.config $(1)/etc/config/jool

	$(INSTALL_DIR) $(1)/etc/init.d
	$(INSTALL_BIN) ./files/jool.init $(1)/etc/init.d/jool

	$(INSTALL_DIR) $(1)/etc/jool
	$(INSTALL_DATA) ./files/jool-nat64.conf.json $(1)/etc/jool/jool-nat64.conf.json.default
	$(INSTALL_DATA) ./files/jool-siit.conf.json $(1)/etc/jool/jool-siit.conf.json.default
	$(INSTALL_DATA) ./files/readme.md $(1)/etc/jool/readme.md
endef


define Package/jool-tools-iptables/install
	$(INSTALL_DIR) $(1)/usr/lib/iptables
	$(INSTALL_BIN) $(PKG_INSTALL_DIR)/usr/lib/iptables/libxt_JOOL_SIIT.so \
		$(1)/usr/lib/iptables
	$(INSTALL_BIN) $(PKG_INSTALL_DIR)/usr/lib/iptables/libxt_JOOL.so \
		$(1)/usr/lib/iptables
endef


define Package/jool-tools-nftables/install
	$(INSTALL_DIR) $(1)/usr/lib/iptables
	$(INSTALL_BIN) $(PKG_INSTALL_DIR)/usr/lib/iptables/libxt_JOOL_SIIT.so \
		$(1)/usr/lib/iptables
	$(INSTALL_BIN) $(PKG_INSTALL_DIR)/usr/lib/iptables/libxt_JOOL.so \
		$(1)/usr/lib/iptables
endef

$(eval $(call KernelPackage,jool-iptables))
$(eval $(call KernelPackage,jool-nftables))
$(eval $(call BuildPackage,jool-tools-iptables))
$(eval $(call BuildPackage,jool-tools-nftables))

[ydahhrk]

Huh. Is it just me, or they documented this way better now?

I just uploaded an improvement patch. Forget JOOL_FLAGS; use CFLAGS_MODULE instead.

Looking at your script, it's probably in these:

$(MAKE) -C "$(LINUX_DIR)" \
	KERNEL_DIR="$(LINUX_DIR)" \
	ARCH="$(LINUX_KARCH)" \
	CROSS_COMPILE="$(TARGET_CROSS)" \
	M="$(PKG_BUILD_DIR)/src/mod/common" \
	V="$(V)" \
	modules

Something like this:

$(MAKE) -C "$(LINUX_DIR)" \
	KERNEL_DIR="$(LINUX_DIR)" \
	ARCH="$(LINUX_KARCH)" \
	CROSS_COMPILE="$(TARGET_CROSS)" \
	M="$(PKG_BUILD_DIR)/src/mod/common" \
	V="$(V)" \
	CFLAGS_MODULE=-DXTABLES_DISABLED \
	modules

Don't forget the backslash! Note, you have to do it three times.

If you can see the output of make, I left some temporary pragmas early in compilation:

al@ubuntu20:~/git/jool/src/mod$ make
(...)
   16 | #pragma message("Xtables enabled")
	  |         ^~~~~~~
^C

al@ubuntu20:~/git/jool/src/mod$ make CFLAGS_MODULE="-DXTABLES_DISABLED"
(...)
   14 | #pragma message("Xtables disabled")
	  |         ^~~~~~~
^C

[ydahhrk]

Yet another improvement patch: Userspace iptables can also be excluded through a flag now.

./configure                    # xtables included
./configure --with-xtables     # xtables included
./configure --with-xtables=yes # xtables included
./configure --with-xtables=no  # xtables excluded

("xtables" and "iptables" are synonyms.)

Note, this only applies to userspace. For kernel modules, keep doing

make (...) CFLAGS_MODULE=-DXTABLES_DISABLED modules

There is a section in the original makefile install section that copies libxt_XXXX.so files, are this needed in the nftables version?

No.

(Note: If you're still finding problems, please open a new issue. This thread is kind of unrelated.)

[agowa]

Is the work of merging jool into nftables still active? Is there a status update on this?

[ydahhrk]

The entire project is more or less frozen. I'm doing minimal maintenance in my free time.

We should get some funding next year, but I can't be any more specific than that because it's still in the planning phase.