
Vnets: Virtual Networks for Virtual Machines

Mike Wray <mike.wray@hp.com>

2005/08/25

0) Introduction
---------------

Vnets provide virtual private LANs for virtual machines.
This is done using bridging and multipoint tunneling. A virtual interface
on a vnet can only see other interfaces on the same vnet - it cannot
see the real network, and the real network cannot see it either.

Virtual interfaces on the same vnet can be on the same machine
or on different machines, they can still talk. The hosting machines
can even be on different subnets if you run vnetd to forward,
or have multicast routing enabled.


1) Installing vnet support
--------------------------

Assuming the code has been installed (make install in the parent directory),
configure xend to use 'network-vnet' instead of the default 'network' to
start up networking. This just loads the vnet module when networking starts.

In /etc/xend/xend-config.sxp:

Configure the network script:

(network-script        network-vnet)

Restart xend.

Alternatively insert the vnet module using vnet-insert,
preferably before xend starts.

2) Creating vnets
-----------------

Xend already implements commands to add/remove vnets and
bridge to them. To add a vnet use

xm vnet-create <vnet config file>

For example, if vnet97.sxp contains:

(vnet (id 97) (bridge vnet97) (vnetif vnetif97) (security none))

do

xm vnet-create vnet97.sxp

This will define a vnet with id 97 and no security. The bridge for the
vnet is called vnet97 and the virtual interface for it is vnetif97.
To add an interface on a vm to this vnet simply set its bridge to vnet97
in its configuration.

In Python:

vif="bridge=vnet97"

In sxp:

(dev (vif (mac aa:00:00:01:02:03) (bridge vnet97)))

Once configured, vnets are persistent in the xend database.
To remove a vnet use

xm vnet-delete <vnet id>

To list vnets use

xm vnet-list

To get information on a vnet id use

xm vnet-list <vnet id>

3) Troubleshooting
------------------

The vnet module should appear in 'lsmod'.
If a vnet has been configured it should appear in the output of 'xm vnet-list'.
Its bridge and interface should appear in 'ifconfig'.
It should also show in 'brctl show', with its attached interfaces.

You can 'see into' a vnet from dom0 if you put an IP address on the bridge
and configure its MAC address as a vif.
For example, if you have vnet97 with a vm with ip addr 10.0.0.12 on it,
and <mac> is the MAC address of vnet97 (use ifconfig), then

echo '(vif.add (vnet 97) (vmac <mac>))' >/proc/vnet/policy
ifconfig vnet97 10.0.0.20 up

should let you ping 10.0.0.12 via the vnet97 bridge.
This works even if the vm with vif 10.0.0.12 is on another
machine (it only works locally if you don't use vif.add).

4) Examples
-----------

Here's the full config for a vm on vnet 97, using ip addr 10.0.0.12:

(vm
 (name dom12)
 (memory '64')
 (cpu '1')
 (console '8502')
 (image
  (linux
   (kernel /boot/vmlinuz-2.6-xenU)
   (ip 10.0.0.12:1.2.3.4::::eth0:off)
   (root /dev/sda1)
   (args 'rw fastboot 4')
  )
 )
 (device (vbd (uname phy:hda2) (dev sda1) (mode w)))
 (device (vif (mac aa:00:00:11:00:12) (bridge vnet97)))
)

If you run another vm on the same vnet:

(vm
 (name dom11)
 (memory '64')
 (cpu '1')
 (console '8501')
 (image
  (linux
   (kernel /boot/vmlinuz-2.6-xenU)
   (ip 10.0.0.11:1.2.3.4::::eth0:off)
   (root /dev/sda1)
   (args 'rw fastboot 4')
  )
 )
 (device (vbd (uname phy:hda3) (dev sda1) (mode w)))
 (device (vif (mac aa:00:00:11:00:11) (bridge vnet97)))
)

the vms should be able to talk over the vnet. Check with ping.
If they are both on the same machine the connection will simply
be the vnet97 bridge, if they are on separate machines their
packets will be tunneled in etherip. They should be able to
see each other, but not the real network.


