Adding a new Switch class to Mininet.

This is a quick post about making Mininet support new network objects, like switches.

Depending on the amount of features that a software switch (or other program) supports, and the degree to which you want to access those features through Mininet, adding a custom node type to Mininet can be a relatively simple task. For something as simple as a classic learning switch, this may be a matter of of implementing just a handful of methods within a child class of Switch.

An example: the LinuxBridge class

A good example of such a custom node is the LinuxBridge, prepackaged with Mininet and found in nodelib.py. This node, which implements a non-OpenFlow learning switch using the classic Linux bridge(8) device, just implements five methods, ignoring its constructor:

  • connected( self ):
    Returns True if the switch is connected to a controller or False if not. For a non-SDN switch, a connected state is when it is fully started up and therefore, ready for interaction. For the LinuxBridge, this is when brctl(8) shows that it is in the “forwarding” state.

  • start( self, _controllers ):
    Initializes and configures the software switch underlying the Mininet Switch object. For our example, this method invokes a series of brctl commands to initialize the switch, add ports, and to apply the necessary configurations (e.g. enable STP) to get it to the connected state. An OpenFlow switch such as OVS would also be configured to connect with a set of controllers passed into this method.

  • stop( self, deleteIntfs=True ):
    Carries out the teardown and removal of the underlying software switch backing the Mininet object. This method is the complement to start.

  • dpctl( self, *args ):
    Invokes the configuration utility for the software switch, parameterized to point to the particular instance associated with the Mininet object. In our case, this is, again, brctl.

  • setup( cls ):
    Checks that all prerequisites are met in order to start up and configure the software switch. This may entail checking that certain kernel modules are loaded, packages like bridge-utils are installed, and certain features, such as the firewall, are configured to allow the switch to function.

A bit of detail to keep note of is that the Switch class does its own housekeeping in its __init__ and stop methods, so these should be called from your custom class as well:

class LinuxBridge( Switch ):

    def __init__( self, name, stp=False, prio=None, **kwargs ):
        # snipped out initialization things here
        Switch.__init__( self, name, **kwargs )

    # snip...

    def stop( self, deleteIntfs=True ):
        # snipped out cleanup things here
        super( LinuxBridge, self ).stop( deleteIntfs )

Specifically, __init__ would be tied to initializing any major lightweight virtualization functions and facilities for sending commands to the process(es) underlying the switch object, and stop, the teardown and cleanup of the interfaces on the switch.

This meant that, by implementing the above methods in a IfBridge class, I was able to quickly put together a learning switch using if_bridge(4) and ifconfig(8) for my experimental Mininet port – barring one easily circumvented but odd bit of behavior. As a convention, custom nodes usually end up in nodelib.py, so that is where I ended up adding my IfBridge node.

Integrating a custom element into `mn`

mn allows you to launch a Mininet topology from the shell, and is one of the first things that the Mininet quickstart has you run to sanity check an install. It’s also handy for launching a topology to test with, without having to write a Python script. Among the options that mn comes with is the class of various network elements (switches, links, etc) to use when building the topology:

$ mn --help
Usage: mn [options]
(type mn -h for details)

The mn utility creates Mininet network from the command line. It can create
parametrized topologies, invoke the Mininet CLI, and run tests.

Options:
  -h, --help            show this help message and exit
  --switch=SWITCH       default|ivs|lxbr|ovs|ovsbr|ovsk|user[,param=value...]
                        ovs=OVSSwitch default=OVSSwitch ovsk=OVSSwitch
                        lxbr=LinuxBridge user=UserSwitch ivs=IVSSwitch
                        ovsbr=OVSBridge
  --host=HOST           cfs|proc|rt[,param=value...]
                        rt=CPULimitedHost{'sched': 'rt'} proc=Host
                        cfs=CPULimitedHost{'sched': 'cfs'}
...

Adding your custom Switch to this list is a matter of a few changes in mn. mn keeps a mapping between the various Node and Link classes and their aliases (e.g. default for OVS, lxbr for LinuxBridge …). So in my case, I have my IfBridge custom switch, which I aliased to ‘ifbr’ in the SWITCHES map:

SWITCHES = { 'user': UserSwitch,
             'ovs': OVSSwitch,
             ...
             'ifbr': IfBridge }

At this point, your custom class should be displayed alongside the other choices in the help message:

  --switch=SWITCH       default|ifbr|ivs|lxbr|ovs|ovsbr|ovsk|user[,param=value
                        ...] ovs=OVSSwitch lxbr=LinuxBridge user=UserSwitch
                        ivs=IVSSwitch default=OVSSwitch ovsk=OVSSwitch
                        ovsbr=OVSBridge ifbr=IfBridge

The rest of the script more or less sanity-checks the combination of classes that had been chosen during invocation, before launching a Mininet object. The sanity checks done for each network element type depends on their features. For example, if --switch=default is specified, mn checks for an OpenFlow controller, since the OVS switch would need a controller to connect to. If a non-functional dummy controller (--controller=none) had been used in the invocation, it will fall back to using bridge-mode OVS (ovsbr) even if ‘default’ was chosen for the switch.

In the case of my custom ‘ifbr’ switch, since it is a good-old-fashioned learning switch, the only extra work I needed to do was to add it to the list of non-OpenFlow switch types that are available for use with the ‘none’ controller:

    def begin( self ):
        "Create and run mininet."
        # snip...
        if not self.options.controller:
            if not CONTROLLERS[ 'default' ]:
            # snip...
                elif self.options.switch not in ( 'ovsbr', 'lxbr', 'ifbr' ):
                    raise Exception( "Could not find a default controller "
                                     "for switch %s" %
                                     self.options.switch )

At this point, I can run mn with my custom switch as so:

sudo ./mn --switch=ifbr --controller=none --test=pingall

Multiple controller domains in Mininet, and link.

Usually, a network emulated in Mininet assumes that all of the switches in a network are connected to the same controller(s). It is possible to point different switches in the same network to different controllers, as shown in some examples here and here, but I wanted make the procedure of creating such a network less manual, and handle a bit more like the existing Topo and Mininet classes.

What I ended up with was a Domain construct that does the work of pointing different switches in the same Mininet topology to different controllers in a way that fits better with the usual Mininet APIs. A Domain defines a groupĀ of switches in a network that are connected to the same (set of) controller(s). A DomainĀ can be built up in a similar way as Topo objects and loaded into the emulated network, and requires no modifications to Mininet to be used.

(For those who come across this page and think this sounds useful, a link to the source is provided here.)

Example 1 – two-domain network.

To use this construct, multiple Domains are loaded into a single Mininet object and then interconnected, resulting in a single network where different parts are controlled by different controllers. For example, if the following topology is desired:

   domain 1    :   domain 2
               :  
     [c1]      :     [c2] 
     /  \      :     /  \
[sw11]--[sw12]---[sw21]--[sw22]
               :

The network can be seen as being built of two domains, each with two switches. Then, the following Domain defines a single one of those domains:

class TestDomain(Domain):
    """
    A tiny domain of two nodes connected together
    """
    def build(self):
    sw1 = 'sw%s1' % self.getId()
    sw2 = 'sw%s2' % self.getId()
    self.addSwitch(sw1)
    self.addSwitch(sw2)

    self.addLink(sw1, sw2)

And then, given that the TestDomain example is saved in a file called testdomain.py, the following script will construct the full network and start a CLI:

from domains import Domain
from testdomain import TestDomain
from mininet.net import Mininet
from mininet.cli import CLI
from mininet.log import setLogLevel, info

if __name__ == '__main__':
    setLogLevel('info')

    #instantiate and build two TestDomains with Domain IDs '1' and '2'
    domain1 = TestDomain(1)
    domain2 = TestDomain(2)
    domain1.addController('c1')
    domain2.addController('c2')
    domain1.build()
    domain2.build()

    # add prepared domains to the network, and initialise
    net = Mininet()
    domain1.injectInto(net)
    domain2.injectInto(net)
    net.build()

    # connect the domains together
    Domain.interConnect(domain1.getSwitches('sw12'), domain2.getSwitches('sw21'), net)

    # start()ing the Domains associates the switches in the domains
    # with the controllers that had been added to them
    domain1.start()
    domain2.start()

    # start up a CLI, as usual. net.stop() will stop the domains it knows about.
    CLI(net)
    net.stop()

For such a small multi-domain network, it might seem overkill – but domains simplify the script given more complex or larger topologies.

At the end of the day, domains are still a hack, so things like connecting two domains is still pretty clunky-looking, even though it’s functional.

Example 2 – building Domains sans child class.

Just like a Topo or Mininet object, you don’t necessarily need a child class of a Domain to build one:

# default ID for a Domain is 0
domain = Domain()
domain.addSwitch('sw1', cls=UserSwitch)
domain.addSwitch('sw2', cls=UserSwitch)
domain.addLink('sw1', 'sw2')

# use as above, sans the call to build()
domain.injectInto(net)
...

As shown above, normal Mininet API arguments can also be passed into a Domain’s methods. Some things differ slightly, like how the string names of nodes are passed to addLink(). Internally, all that these methods do is store the arguments until injectInto() is called, upon which they are passed to the Mininet object’s mid-level API and realized as Mininet network objects.

Notes on Bash arrays, strings, and whatnot

I was told that there is a special place in hell for people who use arrays and do arithmetic in shell.

Unfortunate for me, laziness is a powerful thing, and instead of learning how to use Python or some other sane scripting language, I decided to do exactly that in bash since hey, I already sort of know how to write shell scripts. Or so I thought.

The following are some notes on bash scripting based on the lessons learned, with the caveat that it’s been tested most heavily on OS X (so I may have compromised portability) and that it assumes that the reader has seen a shell script before.

Contents

The Motivation

These notes are the result of my recent attempts to write scripts for auto-generating reStructured Text (reST) files for a technical documentation project. Specifically, I wanted to see if I can generate a reST table, since it’s far more self-documenting than its HTML counterpart:

+---+---+---+---+
| A | B | C | D |
+===+===+===+===+
| 1 | 2 | 3 | 4 |
+---+---+---+---+
|   | 5   6 | 7 |
+---+-------+---+

The above creates something that looks like this (prettiness limited by how it’s rendered):

A B C D
1 2 3 4
5 6 7

Long story short, it seemed like a good idea at the time.

Arrays

Bash arrays are 0-indexed and one-dimenstional (i.e., you can’t have arrays with arrays as elements).

Instantiation/access

There are several ways to declare them. Below, we instantiate array1, array2, and array3 using the various methods:

$ # use the `declare` builtin explicitly
$ declare -a array1=(a b c)
$
$ # assign contents directly with compound assingnment
$ array2=(a b c)
$
$ # assign a value to each element explicitly
$ array3[0]=a
$ array3[1]=b
$ array3[2]=c

array1, array2, and array3 all contain three elements, ‘a’, ‘b’, and ‘c’. We can check by printing their contents:

$ printf '%s ' "${array1[@]}"; printf '\n'
a b c

"${array1[@]}" means “all non-null elements in the array”.

Of course, we can also refer to an index to print what’s there:

$ printf '%s' ${array2[0]}
a

Or pick out a sub-array out of an array. To get just the first two elements out of array3, for example:

printf '%s ' ${array3[@]:0:2}; printf '\n'
-> a b

The 1:2 can be seen as “two elements, starting from index 1”. This idiom can also be used on strings <str-iter>.

Iteration

Starting off with an example:

for i in ${array3[@]}; do
    printf '%s ' "$i"
done
printf '\n'

This translates to returns “a b c” on its own line. We can also use ${!array[@]}, which expands to the indicies of the non-null elements:

for i in ${!array3[@]}; do
    printf '%s ' "${array3[$i]}"
done
printf '\n'

This can be thought of as iterating over an array containing the numbers representing the indices of the contents of array3.

Arrays and arithmetic

When arrays or lists are involved, sooner or later you want the length. ${#array[@]} returns the number of non-null elements in a given array:

$ printf '%s\n' "${#array2[@]}"
3

We can combine this with bash arithmetic using $(()), to say, get the last element of a non-sparse array:

$ size=${#array2[@]}                     # get the number of elements
$ last=$(( ${size} - 1 ))                # calculate the last index (size-1)
$ printf '%s\n' "${array2[${last}]}"     # access the index and print
c

(I tried to break it down to make it more legible, but this should be an indication that, even before we talk about performance, bash is not the choice for these types of things)

The ‘non-null’ bit…

Note how I mention “non-null” repeatedly. This is because arrays in bash can be sparse (have unassigned indices), and it may not be immediately obvious:

$ sparse[0]=a
$ sparse[2]=b
$ sparse[4]=c
$ printf '%s ' "${sparse[@]}"; printf "\n"
a b c
$ printf '%s\n' "${#sparse[@]}"
3
$ printf '%s ' "${!sparse[@]}"; printf "\n"
0 2 4

This means that ${#array[@]} isn’t quite a good indicator of the actual length of an array in bash, so the above example for getting the last element <find_last> doesn’t work for sparse arrays. I guess one can take the last element of the array of indices, then use that to reference the actual last element.

Comparing it to another language (Ruby):

irb(main)> sparse=[]
=> []
irb(main)> sparse[0]='a'
=> "a"
irb(main)> sparse[2]='b'
=> "b"
irb(main)> sparse[4]='c'
=> "c"
irb(main)> sparse
=> ["a", nil, "b", nil, "c"]
irb(main)> sparse.size
=> 5

Strings

Some of the idioms used in arrays apply to strings, like finding its length:

$ str=test-string
$ printf '%s\n' "${#str}"
11

Or getting a substring:

$ printf '%s\n' "${str:0:4}"
test
iteration over strings

The above allows us to do things like iterate over each character in a string:

str=test
i=0
while [ $i -lt ${#str} ]; do
    printf '%d: %s\n' "$i" "${str:$i:1}"
    i=$(( $i +1 ))
done

Saving and running the above with bash <filename> returns:

0: t
1: e
2: s
3: t
String comparisons

The characters in a string have numeric values. The classic interpretation is ASCII – so the following comparison between ‘a’ and ‘b’ compare the ASCII values of the characters, decimal value ’97’ and ’98’:

$ [ 'a' \< 'b' ] && printf '%s\n' "true" || printf '%s\n' "false"
true
$ [ 'a' \> 'b' ] && printf '%s\n' "true" || printf '%s\n' "false"
false

The above uses bash‘s short-circuit && and || operators, which quit evaluation as soon as the result is known.

Note that the operaters need to be escaped (i.e. ‘\>’ not ‘>’), and that comparison will return an error if you try to use -lt, -gt, etc. since they expect integers.

The comparison doesn’t have to stop at single (or alphanumeric) characters:

$ [ '+--+' \< '----' ] && printf '%s\n' "true" || printf '%s\n' "false"
true

The above is true because ‘+’ is ASCII 43, and ‘-‘ is 45. The ‘-‘ actually brings me to:

printf(1) and special characters

I tend to use printf instead of echo because of its consistency across different platforms. If you’re printing something simple, you can invoke the string directly, without formatting:

$ printf "Hello\n"
Hello

However, printf tends to interpret strings starting with ‘-‘ as (invalid) options:

$ printf "--Hello\n"
-bash: printf: --: invalid option
printf: usage: printf [-v var] format [arguments]

Hence the reason why it’s good to go with printf 'formatting' 'strings'...:

$ printf '%s\n' "--Hello"
--Hello

Another method would have been to use printf -- "--Hello\n", to tell it that the string after ‘–‘ shouldn’t be interpreted as an option (just like for rm, for removing files that start with ‘-‘).

Passing arrays to functions

The general function looks like this (Some people omit the ‘function’ bit) :

function foo() {
   # things go here....
}

The arguments to the function basically follow the same format as the bash script itself, i.e. $@ -> argument vector (all arguments), $1->first argument, $2->second… and so on.

When an array is passed to a function, it gets expanded out into its contents, and essentially gets merged into the argument vector. This means that if you want to pass arrays to your functions, you need to do some book-keeping yourself.

A simple way to do this would be to start an array with its length, but another way that could work is by using string representations of arrays as arguments, and converting them into arrays in the function:

function arr-ifs() {
    arr1=( "$1" )

    # print your newly-made array and a newline
    printf '%s ' "${arr1[@]}"
    printf '\n'
}

Pasting that into the shell and invoking it as arr-ifs "a b c" "1 2 3" will return “a b c” (the first 3-element array).

Bash “strict” mode

Although it does get in the way at first, true to its claims, bash strict mode is a sanity saver when it comes to improving the debuggability of your shell script.