Creating networks with VIMAGE jails and epairs.

This is part of a series of notes on the experimental process of getting Mininet to run on FreeBSD.

The first step is to identify the components and commands that are required to implement the basic features. For an emulator like Mininet, this would be 1) the ability to build custom network topologies, and 2) the ability to interact with the topology by sending traffic across it, and monitoring the traffic flowing through the network.

Custom topologies

Mininet allows users to build custom network topologies by interconnecting node and link Mininet objects. Here, jails with VIMAGE replace the mount and network namespaces used to implement the nodes, and epairs replace the veth virtual Ethernet pairs implementing the links.

This link provides clear instructions for getting VIMAGE up and running for a simple topology, making it a good place to start. At the time that this post was written, the stable release (10.2) VIMAGE isn’t enabled by default, and required a custom kernel.

Since the initial (and primary) focus at this time is in building custom topologies, the jails aren’t given their own directory trees, and their paths are set to /.

Handling traffic

In addition to being able to build out topologies, Mininet also allows users to interact with their networks with tools such as ping, traceroute, and tcpdump, which require creating raw sockets from within the jails. This can be enabled by settingĀ security.jail.allow_raw_sockets to 1, or by passing allow.raw_sockets as a command to the jail utility when creating the jails.

Finally, the jails that represent network nodes (e.g. switches and routers, as opposed to end hosts) need some mechanism to move traffic. In Mininet, this would typically be an OpenFlow-programmable software switch such as Open vSwitch or the CPqD software switch. Although the former is available in the ports collection, to reduce the number of moving parts, the if_bridge network device will be used for the time being to narrow down to the core set of commands needed to bring up a topology capable of carrying traffic.

Manual topology construction

The following steps identify the steps and commands required to manually construct what Mininet calls a linear,2 topology:

|    |
h1   h2

where h1 and h2 represent hosts on the network, and s1 and s2, the network nodes (switches).

  1. Prepare the host. After enabling VIMAGE in the kernel:
    # kldload if_bridge
    # sysctl security.jail.allow_raw_sockets=1
  2. Create jails. Since allow_raw_sockets was set in the host, there is no need to pass allow.raw_sockets to jail.
    # jail -c vnet name=s1 jid=1 path=/ persist
    # jail -c vnet name=s2 jid=2 path=/ persist
    # jail -c vnet name=h1 jid=3 path=/ persist
    # jail -c vnet name=h2 jid=4 path=/ persist

    jls should now show your jails (jls -v will show you more, including the assigned names):

    # jls
       JID  IP Address      Hostname                      Path
         1  -                                             /
         2  -                                             /
         3  -                                             /
         4  -                                             /
  3. Create bridges in the ‘network node’ jails (JIDs 1,2, and 3)
    # jexec s1 ifconfig bridge1 create up
    # jexec s2 ifconfig bridge2 create up
  4. Create virtual Ethernet links (epairs) and interconnect the jails
    # ifconfig epair1 create      # s1  h1
    # ifconfig epair2 create      # s2  h2
    # ifconfig epair3 create      # s1  s2
    # ifconfig epair1a vnet s1
    # ifconfig epair1b vnet h1
    # ifconfig epair2a vnet s2
    # ifconfig epair2b vnet h2
    # ifconfig epair3a vnet s1
    # ifconfig epair3b vnet s2
  5. Add epair interfaces to each bridge and bring them up
    jexec s1 ifconfig bridge1 addm epair1a addm epair3a
    jexec s1 ifconfig epair1a up
    jexec s1 ifconfig epair3a up
    jexec s2 ifconfig bridge2 addm epair2a addm epair3b
    jexec s2 ifconfig epair2a up
    jexec s2 ifconfig epair3b up
  6. Configure IP addresses for ‘host’ jail interfaces
    # jexec h1 ifconfig epair1b up
    # jexec h2 ifconfig epair2b up

Sanity-checking the topology

It should now be possible to ping from one host to another:

# jexec h1 ping
PING ( 56 data bytes
64 bytes from icmp_seq=0 ttl=64 time=0.046 ms
--- ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.046/0.052/0.055/0.004 ms

It should also be possible to monitor the traffic passing through a network node (e.g. by running tcpdump) while the hosts are pinging one another.


Once a topology is no longer needed, it should be torn down and the virtual links and jails destroyed.

  1. Remove epairs from jails and destroy them (removing one end of an epair destroys both endpoints)
    # ifconfig epair1a -vnet s1
    # ifconfig epair2a -vnet s2
    # ifconfig epair1a destroy
    # ifconfig epair2a destroy
  2. Destroy the bridges
    jexec s1 ifconfig bridge1 destroy
    jexec s2 ifconfig bridge2 destroy
  3. Destroy jails
    jail -r s1
    jail -r s2
    jail -r h1
    jail -r h2

The idea is that the commands (and procedures) that have been identified here can be retrofitted into Mininet.

[to be continued]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s