FreeBSD 15 comes with a new bridging implementation which has native support for VLANs. They have also soft-deprecated the ability to have any layer 3 addresses on member interfaces which makes it behave like a real hardware switch. The net.link.bridge.member_ifaddrs sysctl controls this behavior and it will be removed in FreeBSD 16.0-RELEASE, same as if set to zero.
🚧 There are concerns about how this will work with router-on-a-stick style setups, but that's a future problem.
One of the primary benefits of the new implementation is that you can have a single bridge for everything and the packet processing has been optimized. Previously the switch performance would degrade as the number of member interfaces increased, but I'm not clear on how significant it really was. It didn't impact gigabit for me, but perhaps you'd see it if you were trying to get 10gbit line rate out of it.
Old Bridging The old design when working with bridges and VLANs was like so: Create a bridge for a specific VLAN
Create a VLAN interface off a physical interface
Attach the VLAN interface to the bridge
Now all your bridge members are able to communicate to that VLAN As you can imagine this starts to get messy especially if you have a lot of VLANs to work with or try to deploy jails that are meant to communicate over multiple VLANs. An example from my old rc.conf looked something like this: # Bhyve and VNET ifconfig_ix1 = "up -tso4 -tso6 -vlanhwfilter -vlanmtu -vlanhwtso -vlanhwtag -vlanhwcsum -lro" vlans_ix1 = "vlan2 vlan3 vlan128" create_args_vlan2 = "vlan 2" create_args_vlan3 = "vlan 3" create_args_vlan128 = "vlan 128" cloned_interfaces = "bridge0 bridge1 bridge2" ifconfig_bridge0_name = "vlan2bridge" ifconfig_vlan2bridge = "addm vlan2 up" ifconfig_bridge1_name = "vlan3bridge" ifconfig_vlan3bridge = "addm vlan3 up" ifconfig_bridge2_name = "vlan128bridge" ifconfig_vlan128bridge = "addm vlan128 addm" It's a lot of work for something that feels so simple if you're familiar with real network gear.
New Bridging The new design allows you to have a single bridge and specify tagged and/or untagged VLANs for each bridge member. Here's what all of that condenses down to. # Bhyve and VNET ifconfig_ix1 = "up -tso4 -tso6 -vlanhwfilter -vlanmtu -vlanhwtso -vlanhwtag -vlanhwcsum -lro" cloned_interfaces = "bridge0" ifconfig_bridge0 = "vlanfilter addm ix1 tagged 2,3,128" That's it. Much simpler, isn't it? Please don't over look the vlanfilter flag on the bridge. If you are missing this you can add members to the bridge with untagged VLANs but you'll get this error when trying to add tagged VLANs: ifconfig: BRDGSIFVLANSET ix1: Invalid argument (extended error VLAN filtering not enabled)
VNET Jails VNET jails posed a problem for me because I was relying on the "unofficial" /usr/share/examples/jails/jib script which can handle creating epair(4) devices for the jails. I'd copy this script into my PATH , make it executable, and then I could use it. A simplified example of this jail configuration looks like this: webserver { exec.prestart += "jib addm $name vlan2"; exec.poststop = "jib destroy $name"; vnet; vnet.interface = "e0b_$name"; } This would handle creating a stable name of epair(4) devices and attach it to the right bridge. e.g., "vlan2" becomes "vlan2bridge" in the jib script. The jib script is old and has some features that I don't believe are necessary anymore. One feature tries to ensure your MAC addreses are stable, but this seems to be a native kernel feature of epair(4) now: $ sysctl -d net.link.epair.ether_gen_addr net.link.epair.ether_gen_addr: Generate MAC with FreeBSD OUI using ether_gen_addr ( 9 ) And ether_gen_addr(9) says: By default, ether_gen_addr attempts to generate a stable MAC address using the hostid of the jail that the ifp is being added to. During early boot, the hostid may not be set on machines that haven't yet pop- ulated /etc/hostid, or on machines that do not use loader(8). Great, we seem to be covered there. But I still need a way to make stable epair device names for the jails and attach them to the bridge the new way, so I wrote a script for that: #!/bin/sh # /scripts/vnetif set -eu ENAME = $1 BRIDGE = $2 VLAN = $3 NEW_EPAIR = $( ifconfig -D epair create -vlanhwfilter up ) EPAIR_NUM = $( echo ${ NEW_EPAIR ##epair } | tr -d '[a]' ) # Do not even want ipv6 link local on here, layer3 not allowed # anymore on bridge members ifconfig ${ NEW_EPAIR } inet6 ifdisabled -auto_linklocal -accept_rtadv no_radr ifconfig epair ${ EPAIR_NUM } a name e0a_ ${ ENAME } ifconfig epair ${ EPAIR_NUM } b name e0b_ ${ ENAME } ifconfig ${ BRIDGE } addm e0a_ ${ ENAME } untagged ${ VLAN } This is not well designed but it gets the job done. Note, on the epair device I had to set -vlanhwfilter -- another hidden requirement! Now that this is sorted, my VNET jail config looks like this: webserver { exec.prestart += "/scripts/vnetif $name bridge0 2"; vnet; vnet.interface = "e0b_$name"; } It works! The VNET interface gets added to the bridge with the correct VLAN membership. It even seems to start the jail faster now... Notice I don't have an exec.poststop anymore. This seems to not be required as the epair interface is automatically destroyed when the jail is torn down now.