Improving Network Reliability on FreeBSD

Kevin Way — Fri, 27 Jun 2008 16:05:00 -0400

If you didn’t notice it during the FreeBSD 7.0/6.3 release, FreeBSD got a great new tool from OpenBSD. The lagg device.

This device allows you to setup links with failover, or to combine them using LACP, and the operation is dead simple. Here’s an example rc.conf, that just does a basic link failover:


cloned_interfaces="lagg0" 
ifconfig_bge0="up" 
ifconfig_bge1="up" 
ifconfig_lagg0="laggproto failover laggport bge1 laggport bge0 192.168.1.5 netmask 255.255.255.0"

Or if you use 802.1q trunks


cloned_interfaces="lagg0 vlan0" 
ifconfig_em0="up" 
ifconfig_em1="up" 
ifconfig_lagg0="laggproto failover laggport em0 laggport em1" 
ifconfig_vlan0="vlan 22 vlandev lagg0 192.168.1.5 netmask 255.255.255.0"

The only downside of this, at all, is you need to write a quick nagios plugin to check for dead links, but fortunately, that’s easy enough to do as well.

Note: We don’t currently use LACP, because we’ve had some issues with it losing connectivity altogether, after alternating link failures.

Example ifconfig output from a successful lagg setup:


em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
    ether 00:15:17:73:61:f4
    media: Ethernet 100baseTX <full-duplex>
    status: active
    lagg: laggdev lagg0
em4: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
    ether 00:15:17:73:61:f4
    media: Ethernet 100baseTX <full-duplex>
    status: active
    lagg: laggdev lagg0
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
    ether 00:15:17:73:61:f4
    inet 192.168.1.52 netmask 0xffffff00 broadcast 192.168.1.255
    media: Ethernet autoselect
    status: active
    laggproto failover
    laggport: em4 flags=0<>
    laggport: em0 flags=5<MASTER,ACTIVE>

Example ifconfig output from a successful vlan and lagg combination:


bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
    ether 00:11:0a:30:21:04
    media: Ethernet autoselect (1000baseTX <full-duplex>)
    status: active
    lagg: laggdev lagg0
bge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
    ether 00:11:0a:30:21:04
    media: Ethernet autoselect (1000baseTX <full-duplex>)
    status: active
    lagg: laggdev lagg0
lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
    ether 00:11:0a:30:21:04
    media: Ethernet autoselect
    status: active
    laggproto failover
    laggport: bge0 flags=0<>
    laggport: bge1 flags=5<MASTER,ACTIVE>
vlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=3<RXCSUM,TXCSUM>
    ether 00:11:0a:30:21:04
    inet 192.168.1.145 netmask 0xffffff00 broadcast 192.168.1.255
    media: Ethernet autoselect
    status: active
    vlan: 22 parent interface: lagg0

blog@insidesystems.net: Improving Network Reliability on FreeBSD

Improving Network Reliability on FreeBSD