Channel Bonding – Points

arp_ip_target
Specifies the IP addresses to use as ARP monitoring peers when         arp_interval is > 0.  These are the targets of the ARP request sent to determine the health of the link to the targets. Specify these values in ddd.ddd.ddd.ddd format.  Multiple IP addresses must be separated by a comma.  At least one IP address must be given for ARP monitoring to function.  The maximum number of targets that can be specified is 16.  The default value is no IP addresses.

downdelay
Specifies the time, in milliseconds, to wait before disabling a slave after a link failure has been detected.  This option is only valid for the miimon link monitor.  The downdelay value should be a multiple of the miimon value; if not, it  will be rounded down to the nearest multiple.  The default  value is 0.

max_bonds
Specifies the number of bonding devices to create for this instance of the bonding driver.  E.g., if max_bonds is 3, and the bonding driver is not already loaded, then bond0, bond1 & bond2 will be created.  The default value is 1.

miimon
Specifies the MII link monitoring frequency in milliseconds. This determines how often the link state of each lave is inspected for link failures.  A value of zero disables MII link monitoring.  A value of 100 is a good starting point. The use_carrier option, below, affects how the link state is determined.  See the High Availability section for additional information.  The default value is 0.

mode
Specifies one of the bonding policies. The default is balance-rr (round robin).  Possible values are:

balance-rr or 0
Round-robin policy: Transmit packets in sequential order from the first available slave through the last.  This mode provides load balancing and fault tolerance.

active-backup or 1
Active-backup policy: Only one slave in the bond is active.  A different slave becomes active if, and only if, the active slave fails.  The bond’s MAC address is externally visible on only one port (network adapter) to avoid confusing the switch. In bonding version 2.6.2 or later, when a failover  occurs in active-backup mode, bonding will issue one  or more gratuitous ARPs on the newly active slave. One gratutious ARP is issued for the bonding master interface and each VLAN interfaces configured above it, provided that the interface has at least one IP   address configured.  Gratuitous ARPs issued for VLAN interfaces are tagged with the appropriate VLAN id.     This mode provides fault tolerance.  The primary option, documented below, affects the behavior of this mode.

balance-xor or 2
XOR policy: Transmit based on the selected transmit hash policy.  The default policy is a simple [(source           MAC address XOR’d with destination MAC address) modulo slave count].  Alternate transmit policies may be    selected via the xmit_hash_policy option, described  below. This mode provides load balancing and fault tolerance.

broadcast or 3
Broadcast policy: transmits everything on all slave interfaces.  This mode provides fault tolerance.

802.3ad or 4
IEEE 802.3ad Dynamic link aggregation.  Creates aggregation groups that share the same speed and duplex settings.  Utilizes all slaves in the active aggregation according to the 802.3ad specification. Save selection for outgoing traffic is done according to the transmit hash policy, which may be changed from the default simple XOR policy via the xmit_hash_policy option, documented below.  Note that not all transmit policies may be 802.3ad compliant, particularly in regards to the packet mis-ordering requirements of section 43.2.4 of the 802.3ad standard.  Differing peer implementations will have varying tolerances for noncompliance.

Prerequisites:
1. Ethtool support in the base drivers for retrieving the speed and duplex of each slave.
2. A switch that supports IEEE 802.3ad Dynamic link aggregation. Most switches will require some type of configuration to enable 802.3ad mode.

balance-tlb or 5
Adaptive transmit load balancing: channel bonding that does not require any special switch support.  The        outgoing traffic is distributed according to the current load (computed relative to the speed) on each slave.  Incoming traffic is received by the current slave.  If the receiving slave fails, another slave takes over the MAC address of the failed receiving slave.

Prerequisite:
Ethtool support in the base drivers for retrieving the speed of each slave.

balance-alb or 6
Adaptive load balancing: includes balance-tlb plus receive load balancing (rlb) for IPV4 traffic, and does not require any special switch support.  The receive load balancing is achieved by ARP negotiation. The bonding driver intercepts the ARP Replies sent by the local system on their way out and overwrites the source hardware address with the unique hardware address of one of the slaves in the bond such that different peers use different hardware addresses for the server. Receive traffic from connections created by the server is also balanced.  When the local system sends an ARP Request the bonding driver copies and saves the peer’s IP information from the ARP packet.  When the ARP Reply arrives from the peer, its hardware address is retrieved and the bonding driver initiates an ARP reply to this peer assigning it to one of the slaves in the bond.  A problematic outcome of using ARP negotiation for balancing is that each time that an ARP request is broadcast it uses the hardware address of the bond.  Hence, peers learn the hardware address of the bond and the balancing of receive traffic collapses to the current slave.  This is handled by sending updates (ARP Replies) to all the peers with their individually assigned hardware address such that the traffic is redistributed.  Receive traffic is also redistributed when a new slave is added to the bond and when an inactive slave is re-activated.  The receive load is distributed sequentially (round robin) among the group of highest speed slaves in the bond. When a link is reconnected or a new slave joins the Bond the receive traffic is redistributed among all  active slaves in the bond by initiating ARP Replies with the selected mac address to each of the clients. The updelay parameter (detailed below) must be set to a value equal or greater than the switch’s forwarding delay so that the ARP Replies sent to the peers will not be blocked by the switch.

Prerequisites:
1. Ethtool support in the base drivers for retrieving the speed of each slave.
2. Base driver support for setting the hardware address of a device while it is open.  This is required so that there will always be one slave in the team using the bond hardware address (the curr_active_slave) while having a unique hardware address for each slave in the bond.  If the curr_active_slave fails its hardware address is          swapped with the new curr_active_slave that was chosen.

arp_interval
Specifies the ARP link monitoring frequency in milliseconds. If ARP monitoring is used in an etherchannel compatible mode (modes 0 and 2), the switch should be configured in a mode that evenly distributes packets across all links. If the switch is configured to distribute the packets in an XOR fashion, all replies from the ARP targets will be received on the same link which could cause the other team members to fail.  ARP monitoring should not be used in conjunction with miimon.  A value of 0 disables ARP monitoring.  The default value is 0.

Channel Bonding

Channel bonding (also known as “Ethernet bonding”) is a computer networking arrangement in which two or more network interfaces on a host computer are combined for redundancy or increased throughput.

On Ethernet interfaces, channel bonding requires assistance from both the Ethernet switch and the host computer’s operating system, which must “stripe” the delivery of frames across the network interfaces in the same manner that I/O is striped across disks in a RAID array. For this reason, channel bonding is sometimes also called RAIN, or “redundant array of independent network interfaces”.

Multiple dial-up links over POTS can be channel-bonded together in the same manner and can come closer to achieving their aggregate bandwidth than routing schemes which simply load-balance outgoing network connections over the links. This is known as modem bonding.

The Linux bonding driver provides a method for aggregating multiple network interfaces into a single logical bonded interface. The behavior of the bonded interfaces depends upon the mode; generally speaking, modes provide either hot standby or load balancing services. Additionally, link integrity monitoring may be performed.

RHEL/CentOS allows administrators to bind NICs together into a single channel using the bonding kernel module and a special network interface, called a channel bonding interface. Channel bonding enables two or more network interfaces to act as one, simultaneously increasing the bandwidth and providing redundancy.

Following is the simple process to configure NIC bonding on a linux server:-

1. Create/Edit a file called ifcfg-bond0 in /etc/sysconfig/network-scripts/

# vi /etc/sysconfig/network-scripts/ifcfg-bond0

Append following lines to it:

DEVICE=bond0
IPADDR=192.168.1.10
NETWORK=192.168.1.0
NETMASK=255.255.255.0
BROADCAST=192.168.1.255
GATEWAY=192.168.1.1
USERCTL=no
BOOTPROTO=none
ONBOOT=yes

Replace above IP address with your actual IP address. Save file and exit to shell prompt.

2. Modify eth0 and eth1 config files:

Open both configuration using vi text editor and make sure file read as follows for eth0 interface

# vi /etc/sysconfig/network-scripts/ifcfg-eth0

and Modify/append directive as follows:

DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none

Open eth1 configuration file using vi text editor:

# vi /etc/sysconfig/network-scripts/ifcfg-eth1

and Make sure file read as follows for eth1 interface:

DEVICE=eth1
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none

Save file and exit to shell prompt.

3. Load bond driver/module

Make sure bonding module is loaded when the channel-bonding interface (bond0) is brought up. You need to modify kernel modules configuration file:

# vi /etc/modprobe.conf  (in RHEL 5)  “OR”
# vi /etc/modprobe.d/modprobe.conf  (in RHEL 6)

Append following two lines:

# vi /etc/modprobe.conf

alias bond0 bonding
options bond0 miimon=100 mode=1

Mode value is used to determine the operational mode of Bond e.g. active-backup or active-active. Different bonding modes are documented in /usr/share/doc/kernel-doc-2.4.21/networking/bonding.txt

There are different modes in network bonding :
mode=0 (Balance-rr) – This mode provides load balancing and fault tolerance.
mode=1 (active-backup) – This mode provides fault tolerance.
mode=2 (balance-xor) – This mode provides load balancing and fault tolerance.
mode=3 (broadcast) – This mode provides fault tolerance.
mode=4 (802.3ad) – This mode provides load balancing and fault tolerance.
mode=5 (balance-tlb) – Prerequisite: Ethtool support in the base drivers for retrieving the speed of each slave.
mode=6 (Balance-alb) – Prerequisite: Ethtool support in the base drivers for retrieving the speed of each slave.

Run following commands to activate bonding; make sure you are connected to console otherwise you will loose your session as network services will be restarted.

# modprobe bonding
# modprobe -r -v bonding
# modeprobe -r -b bonding -o bond0

Restart networking service in order to bring up bond0 interface:

# service network restart

To verify that bond is working; use following commands:

# less /proc/net/bonding/bond0
# ifconfig -a