Support for VETH interfaces #368

daniloegea · 2023-06-15T11:17:56Z

Design: Netplan support for VETH

Rationale

Veths are Virtual Ethernet interfaces that are commonly used to
connect bridges or network namespaces. They act like a link between
two networks, simply bypassing traffic from one veth peer to the other.

Veths work in peers, meaning that users will always create two of them.

This implementation allows users to create peers of veths, configure them with
IPs, routes and etc, change their MAC addresses and use them as member
interfaces for bridges and bonds for example. It doesn't support adding veths
to different network namespaces, although it could be done by an external tool.

Changes in the schema

As both peers are configurable with IPs and etc, they must be defined in
separate stanzas.

A new type of network called virtual-ethernets was created and one extra key called
peer was introduced.

network:
  version: 2
  renderer: networkd
  virtual-ethernets:
    veth0-peer1:
      peer: veth0-peer2
    veth0-peer2:
      peer: veth0-peer1

  bridges:
    br0:
      interfaces:
        - veth0-peer1
    br1:
      interfaces:
        - veth0-peer2

Even though veth only work in pairs, it's not mandatory to define the peer
stanza (the peer key is mandatory) when using NetworkManager as renderer. The peer interface is not required
because Network Manager requires the creation of two connections, one for each
peer. As our keyfile parser maps a keyfile to a YAML file (a 1:1 mapping), if
we forced the creation of both peers, we'd had problems with the
Netplan-NetworkManager integration.

systemd-networkd backend

Networkd requires a single .netdev file to create both veths.
So the configuration above will generate the .netdev file below:

[NetDev]
Name=veth0-peer1
Kind=veth

[Peer]
Name=veth0-peer2

If further configuration is required, one .network file per veth peer must be
created.

As only one .netdev is needed, we sort both peer names and use the first one.
If the peer stanza is not defined, it will not generate any files.

Network Manager backend

On Network Manager, one keyfile per peer is required. So, creating a veth pair
requires 2 connections.

veth-peer1.nmconnection:

[connection]
id=veth-peer1
uuid=64cec75d-1f76-4379-bd82-684bc77484b9
type=veth
interface-name=veth-peer1

[veth]
peer=veth-peer2

veth-peer2.nmconnection:

[connection]
id=veth-peer2
uuid=a5d3f350-15f7-4e21-a2d4-be549531fba3
type=veth
interface-name=veth-peer2

[veth]
peer=veth-peer1

Netplan Everywhere

One of the implications of these changes is that Network Manager will
create YAML files for virtual-ethernets automatically.

veth devices will be seen as unmanaged by Network Manager until the file
/var/run/NetworkManager/conf.d/10-globally-managed-devices.conf is
created by the generator and Network Manager is restarted. The implication
is that, veth pairs created with NM on a system that is not already
managed by NM will not come up online until the user either calls
netplan apply or restart NM.

Keyfile parser

The keyfile parser will create a 1:1 mapping between the keyfile
and the interface in the YAML file it generates. That's the main
reason why it's not mandatory to defined both peers at once.

That approach enable Netplan Everywhere to work out of the box
with virtual-ethernets.

Alternative solution

$NOTE$: this was the original approach implemented. But due to complications
with the integration with Network Manager, we used the approach described
above.

We could make it mandatory to define both peers at once. It makes sense
because virtual-ethernets only work in pairs.

When creating the veth interface from the keyfile, a stanza for the peer
is also created and linked to the veth. That basically means that a keyfile
for a veth interface will generate 2 interfaces in the YAML file, the
interface itself and its peer. The same will happen when the second veth
(the peer) is created in a second API call to Network Manager. In the end,
2 Netplan YAML files will be created. When the generator is called, it will
merge both files, creating the full veth configuration.

This approach will require some changes in the Network Manager integration with Netplan.
Currently, Network Manager assumes a single netdef is present in the
Netplan state, but now we will have 2 netdefs. Both netdefs must be part of
the same YAML file so, instead of writing the netdef to the file, we will
need to write the entire netplan state (which will contain 2 netdefs).

After some changes to the Network Manager's integration patch,
that's the results:

Adding the first veth interface:

nmcli con add \
  con-name veth-nm-peer1 \
  type veth \
  peer veth-nm-peer2 \
  connection.interface-name veth-nm-peer1

This will create the file
/etc/netplan/90-NM-d4afd034-9634-4a67-a3ee-0c52154eefa3.yaml with the
following content:

network:
  version: 2
  virtual-ethernets:
    veth-nm-peer1:
      renderer: NetworkManager
      dhcp4: true
      dhcp6: true
      peer: "veth-nm-peer2"
      networkmanager:
        uuid: "d4afd034-9634-4a67-a3ee-0c52154eefa3"
        name: "veth-nm-peer1"
        passthrough:
          ipv6.addr-gen-mode: "default"
          ipv6.ip6-privacy: "-1"
          proxy._: ""
    veth-nm-peer2:
      renderer: NetworkManager
      peer: "veth-nm-peer1"

Now, adding the veth peer with the following command:

nmcli con add \
  con-name veth-nm-peer2 \
  type veth \
  peer veth-nm-peer1 \
  connection.interface-name veth-nm-peer2

This will create the file
/etc/netplan/90-NM-505fc532-a910-4346-8bd8-19e849b696e6.yaml with the
following content:

network:
  version: 2
  virtual-ethernets:
    veth-nm-peer2:
      renderer: NetworkManager
      dhcp4: true
      dhcp6: true
      peer: "veth-nm-peer1"
      networkmanager:
        uuid: "505fc532-a910-4346-8bd8-19e849b696e6"
        name: "veth-nm-peer2"
        passthrough:
          ipv6.addr-gen-mode: "default"
          ipv6.ip6-privacy: "-1"
          proxy._: ""
    veth-nm-peer1:
      renderer: NetworkManager
      peer: "veth-nm-peer2"

Network Manager now has both connections:

NAME           UUID                                  TYPE  DEVICE
veth-nm-peer1  d4afd034-9634-4a67-a3ee-0c52154eefa3  veth  --
veth-nm-peer2  505fc532-a910-4346-8bd8-19e849b696e6  veth  --

netplan get will then produce the following result:

network:
  version: 2
  virtual-ethernets:
    veth-nm-peer2:
      renderer: NetworkManager
      dhcp4: true
      dhcp6: true
      peer: "veth-nm-peer1"
      networkmanager:
        uuid: "505fc532-a910-4346-8bd8-19e849b696e6"
        name: "veth-nm-peer2"
        passthrough:
          ipv6.addr-gen-mode: "default"
          ipv6.ip6-privacy: "-1"
          proxy._: ""
    veth-nm-peer1:
      renderer: NetworkManager
      dhcp4: true
      dhcp6: true
      peer: "veth-nm-peer2"
      networkmanager:
        uuid: "d4afd034-9634-4a67-a3ee-0c52154eefa3"
        name: "veth-nm-peer1"
        passthrough:
          ipv6.addr-gen-mode: "default"
          ipv6.ip6-privacy: "-1"
          proxy._: ""

Deleting connections

Netplan's netplan_delete_connection() will try to remove one of the peers
from the Netplan's state. As a veth peer cannot exist without the other,
netplan_delete_connection() will need to remove both connections when
any one of them is deleted.

The problem with this approach is that, after deleting one of the veths,
both of them will disappear from the YAML files but one connection will be kept
in the Network Manager's state.

So, deleting an interface, would require the deletion of both of them. If
we delete a single interface, Netplan will fail to parse the one that's left.
If we delete both of them from netplan, after running nmcli con del, one would
still be in the Network Manager's state so it would be out of sync with netplan.

Checklist

Runs make check successfully.
Retains 100% code coverage (make check-coverage).
New/changed keys in YAML format are documented.
(Optional) Adds example YAML for new feature.
(Optional) Closes an open bug in Launchpad.

slyon · 2023-08-02T15:09:02Z

Schema wise, I like the proposal from the PR description. I think it makes sense to introduce a new interface type for veths, as networkd is using a top-level [NetDev].Kind=veth setting and NetworkManager is also using a top-level [connection].type=veth setting. I'm a bit undecided about the new peer setting, though. Calling correlated veths "peers" is the common term across the networking stack. OTOH, in Netplan we're using the term "link" for SRIOV, VLAN & VXLAN, and we could use it here, too, for consistency.
I'm leaning towards still calling it peer, as link is more of a one-sided relationship between interfaces (parent-child), while "peer" is linking interfaces on the same level.

@vorlonofportland What's your stance on this?

network:
  veths:
    IFACE1:
      peer: IFACE2
      ...
    IFACE2:
      peer: IFACE1
      ...

The implementation details are a bit tricky on this one, due to the cross-linking of interfaces and the declarative vs imperative architectural differences with NetworkManager. I think it's an elegant solution to make the validation a bit less strict for the NM backed in order to support this. Will do a full review soon.

slyon

Thank you, this is looking very good implementation wise!

I added a few inline remarks, but no major changes required, IMO.

+1 (but let's block merging util we have the new YAML schema approved)

tests/integration/veths.py

slyon · 2023-08-03T09:52:33Z

doc/netplan-yaml.md

+
+Virtual Ethernets acts as tunnels forwarding traffic from one interface to the other.
+They can be used to connect two separate virtual networks such as network namespaces and
+bridges. It's not possible to move `veths` to different namespaces through Netplan at the


thought (non-blocking): We could mention using hooks as described in https://netplan.io/faq#use-pre-up-post-up-etc-hook-scripts as a workaround for moving it into a namepsace.

hm I don't know how the backends would react after moving the interface to another namespace... we might want to investigate it before adding to the documentation.

src/abi.h

tests/generator/test_veths.py

tests/integration/veths.py

tests/parser/test_keyfile.py

src/parse.c

src/networkd.c

src/validation.c

vorlonofportland · 2023-08-04T22:54:42Z

veth is short for virtual ethernet. Elsewhere in the schema we spell out ethernets. Should this be virtual-ethernets instead?

What if they were just called ethernets, and ethernets with the mandatory peer key were automatically veths, and ethernets without this key were physical devices?

daniloegea · 2023-08-08T08:29:55Z

Regarding the name, veth is the term used by both backends and, as far as I know, it's the most common term people will use to refer to this type of devices. But apart from that, I don't have a strong opinion against using virtual-ethernets.

One argument against generalizing ethernets and allowing it to also create veths is that it would allow users to try to use configuration intended to be used only for physical devices with virtual devices, such as match for example.

slyon · 2023-08-08T08:45:56Z

Regarding the name, veth is the term used by both backends and, as far as I know, it's the most common term people will use to refer to this type of devices. But apart from that, I don't have a strong opinion against using virtual-ethernets.

I'd also be fine with virtual-ethernets, as indeed we're usually using the more verbose spelling in Netplan.

One argument against generalizing ethernets and allowing it to also create veths is that it would allow users to try to use configuration intended to be used only for physical devices with virtual devices, such as match for example.

Exactly. This is a problem Danilo and I discussed previously. ethernets are pure physical devices, allowing for settings such as match, set-name, *-offload, etc, while virtual-ethernets/veths are pure virtual devices, where the NetplanID is the interface name and it can have the peer setting. peer and match (or other physical-devices properties) are mutually exclusive, and it's hard to check this condition at the YAML parsing stage, as those settings might be defined in arbitrary order (could be checked in the validation stage, though). But mixing those physical vs virtual properties could be confusing to our users.

So I'd suggest going with virtual-ethernets. Are you fine with that @vorlonofportland ?

vorlonofportland · 2023-08-11T03:49:17Z

Yes, +1 from me for virtual-ethernets then.

The new function validate_veth_pair is intended to be used by the backend code to do the final validation as we might have missed something during parsing

When generating configuration for veth, only one .netdev file should be created for the pair. The logic used to select which netdef will be used to generate the file is simple: we sort the pair names and use the first one. If the veth's pair wasn't defined in the YAML file we don't hard fail, just skip it.

daniloegea · 2023-08-11T13:49:03Z

I replaced veths by virtual-ethernets.

The debci test is failing because api.ftp-master.debian.org is having problems apparently...

curl: (7) Failed to connect to api.ftp-master.debian.org port 443 after 177 ms: Connection refused

slyon · 2023-08-14T09:16:19Z

+1 Thanks for the renaming. I've adopted the PR description accordingly for virtual-ethernets. The DebCi test failure looks unrelated.

LGTM.

daniloegea added the schema change This PR includes a change to netplan's YAML schema, which needs sign-off label Jun 15, 2023

daniloegea force-pushed the veth_support branch 2 times, most recently from d5c4a39 to b2d5483 Compare June 29, 2023 14:03

daniloegea requested a review from slyon June 30, 2023 14:20

daniloegea force-pushed the veth_support branch from b2d5483 to 4f33530 Compare July 24, 2023 16:16

slyon approved these changes Aug 3, 2023

View reviewed changes

daniloegea force-pushed the veth_support branch from 4f33530 to 9a684bc Compare August 8, 2023 12:38

daniloegea added 10 commits August 11, 2023 10:13

parser: add support for virtual-ethernet interfaces

d5bbb56

virtual-ethernet: add validation rules for virtual-ethernets

359a6a3

The new function validate_veth_pair is intended to be used by the backend code to do the final validation as we might have missed something during parsing

virtual-ethernet: add support for virtual-ethernets to the YAML emitter

1d37d93

examples: add a virtual-ethernets example file

c701983

nm: add support for virtual-ethernets

5c7cfae

parse-nm: add support for virtual-ethernets

32350c2

tests: new generator tests for virtual-ethernets

6e25345

docs: update the documentation with the new virtual-ethernets devices

b834fe8

tests: add integration tests for virtual-ethernets

c313812

daniloegea force-pushed the veth_support branch from 9a684bc to c313812 Compare August 11, 2023 09:21

slyon added the schema ok label Aug 14, 2023

slyon merged commit 78ab30f into canonical:main Aug 14, 2023
13 of 14 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for VETH interfaces #368

Support for VETH interfaces #368

daniloegea commented Jun 15, 2023 •

edited by slyon

Loading

slyon commented Aug 2, 2023 •

edited

Loading

slyon left a comment

slyon Aug 3, 2023

daniloegea Aug 8, 2023

slyon Aug 10, 2023

vorlonofportland commented Aug 4, 2023

daniloegea commented Aug 8, 2023

slyon commented Aug 8, 2023

vorlonofportland commented Aug 11, 2023

daniloegea commented Aug 11, 2023

slyon commented Aug 14, 2023

Support for VETH interfaces #368

Support for VETH interfaces #368

Conversation

daniloegea commented Jun 15, 2023 • edited by slyon Loading

Design: Netplan support for VETH

Rationale

Changes in the schema

systemd-networkd backend

Network Manager backend

Netplan Everywhere

Keyfile parser

Alternative solution

Deleting connections

Checklist

slyon commented Aug 2, 2023 • edited Loading

slyon left a comment

Choose a reason for hiding this comment

slyon Aug 3, 2023

Choose a reason for hiding this comment

daniloegea Aug 8, 2023

Choose a reason for hiding this comment

slyon Aug 10, 2023

Choose a reason for hiding this comment

vorlonofportland commented Aug 4, 2023

daniloegea commented Aug 8, 2023

slyon commented Aug 8, 2023

vorlonofportland commented Aug 11, 2023

daniloegea commented Aug 11, 2023

slyon commented Aug 14, 2023

daniloegea commented Jun 15, 2023 •

edited by slyon

Loading

slyon commented Aug 2, 2023 •

edited

Loading