epairs and duplicate address (DAD) warnings.

After the initial step of determining the set of commands needed to bring up a VIMAGE/jail network (described here), I started reorganizing it to more closely mimic the order in which these commands would be called by a Mininet script. The typical order of operations and their corresponding commands are roughly:

  1. Instantiate a topology (Topo) object: (no corresponding step)
  2. Add Switches and Hosts to Topo object: Start up some jails with bridges, if they are switches, and shells, if they’re hosts
  3. Interconnect the Switches and Hosts by adding Links to the Topo object: create epairs, and move the interfaces to the jails)
  4. Initialize the network with the Topo object as its topology: Bring the interfaces up, and if the jails represent switches, add the interface to the bridge

But, while trying to recreate the same –linear,2 network, I noticed the following messages in dmesg:

epair2b: DAD detected duplicate IPv6 address fe80:2::ff:70ff:fe00:40b: NS in/out/loopback=4/1/0, NA in=0
epair2b: DAD complete for fe80:2::ff:70ff:fe00:40b - duplicate found
epair2b: manual intervention required
epair2b: possible hardware address duplication detected, disable IPv6
epair3b: DAD detected duplicate IPv6 address fe80:2::ff:70ff:fe00:40b: NS in/out/loopback=4/1/0, NA in=0
epair3b: DAD complete for fe80:2::ff:70ff:fe00:40b - duplicate found
epair3b: manual intervention required
epair3b: possible hardware address duplication detected, disable IPv6

And a bit above, the following:

epair2a: Ethernet address: 02:ff:20:00:03:0a
epair2b: Ethernet address: 02:ff:70:00:04:0b
epair2a: link state changed to UP
epair2b: link state changed to UP
epair3a: Ethernet address: 02:ff:20:00:03:0a
epair3b: Ethernet address: 02:ff:70:00:04:0b
epair3a: link state changed to UP
epair3b: link state changed to UP

As the DAD (Duplicate Address Detection) warnings suggested, the same MAC (hardware) addresses were indeed being reused for the epair* interfaces being created.

Searching for the warnings eventually brought me to a thread describing the exact mechanics behind the issue – I had interleaved the steps for creating and moving the interfaces. The MAC addresses for epair* interfaces are generated from a globally tracked if_index counter. This value increases by one for each interface created (so +2 for each ifconfig epair create), and decreases by one for each destroyed or moved to a VIMAGE jail. The problem arises when epair creation is interleaved with moving them to jails; The if_index:

  1. increases by two for the first epairs created
  2. drops back down by two when they are moved, and
  3. take on the same values as for the first epairs when the next epairs are created

In fact, since the value of if_index is used directly in the 4th and 5th byte of the MAC address (first and last are hard-coded and the rest, set by other means), we can see in the dmesg output that the index values 3 and 4 are being reused repeatedly.

It also explains why I didn’t see this issue initially, since I was creating all of the epairs at once and then moving them later on, making the changes in if_index monotonic. While I could reorganize the commands so that it both follows Mininet’s conventions and the if_index doesn’t fluctuate, I manually assigned unique addresses to each epair for the time being:

# ifconfig epair1a ether 02:ff:00:00:01:14    #from s1 (jid 1) to h1 (jid 4)
# ifconfig epair1b ether 02:ff:00:00:01:41    #from s1 (jid 4) to h1 (jid 1)
...

Of course, I’ll use a less manual approach for generating a unique MAC address in Mininet.

[update]: if_index is not guaranteed to monotonically increase, but the number in the interface name (the ‘n’ in epair[n]) does, so I decided to use that as a base for my unique MACs.


References:

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s