I've decided to rebuild my KVM lab on OpenSUSE 13.2 and to try and finish documenting the process of a fully functional KVM lab on multiple VLANs.
Install KVM with yast2
I used the "Install Hypervisor and Tools" in yast2 to install KVM. I also allowed it to create the bridge (br0)
Access with virt-manager
This is the same as the steps I outlined in my previous post http://pyn00b.blogspot.co.uk/2014/01/opensuse-kvm-getting-started-notes.html - just make sure to add yourself to the libvirt group and to restart the libvirtd.service
The objective here is to have a single KVM host with a single Ethernet NIC to have multiple and routable segregated subnets. This is achieved by Bridging and VLAN tagging. We do this so that we can use the existing network infrastructure rather than using the KVM host to do any NATing or using KVM virtual networks.
Our solution is to have a single NIC, to which we create a VLAN interface, a Bridge is created from the VLAN interface and KVM attaches virtual NICs to the Bridge. The bridge is specified when creating a host and is therefore attached to the VLAN. This can be repeated multiple times for any number of VLANs. For example, a hierarchy of network devices might look like this:
Physical Nic (enp3s0) -> VLAN interface (enp3s0.69) -> Bridge (br69) -> vnet0
Physical Nic (enp3s0) -> VLAN interface (enp3s0.69) -> Bridge (br69) -> vnet1
Physical Nic (enp3s0) -> VLAN interface (enp3s0.101) -> Bridge (br101) -> vnet2
So in summary, for each new VLAN, a new VLAN interface is created with a VLAN tag, that interface is then Bridged and any number of KVM hosts attach their virtual NIC to the bridge and their traffic is VLAN tagged transparently.
Setting up VLAN interfaces Let's start by creating an virtual interface that will tag traffic for VLAN 69. We do so by creating the file /etc/sysconfig/network/ifcfg-enp3s0.69
BOOTPROTO='none' STARTMODE='auto' ETHERDEVICE='enp3s0'
Notice that by using the format ifcfg-enp3s0.VLANID sets the VLAN tag to 69 for us. You can use VLAN_ID= as an option instead if you want to. (see man ifcfg-vlan)
Also worth noting is that I have set BOOTPROTO='none' rather than 'static' and using IPADDR=''. This is somewhat in contradiction to man ifcfg which states "Do not use (none) to just skip the IP setup". However I noticed that when yast2 creates the bridge during the KVM install it sets BOOTPROTO='none'. In my testing either option had the same result. I used 'none' for consistency. The yast convention is also to create ifcfg-vlanX which I did not care for.
Setting up Bridges
We now need to created a bridge from our VLAN interface. We do so by creating /etc/sysconfig/network/ifcfg-br69
BOOTPROTO='none' BRIDGE='yes' BRIDGE_FORWARDDELAY='0' BRIDGE_PORTS='enp3s0.69' BRIDGE_STP='off' STARTMODE='auto'
Changes to sysctl
Disable any traffic over the bridges getting sent to iptables ( http://wiki.libvirt.org/page/Net.bridge.bridge-nf-call_and_sysctl.conf ) I created /etc/sysctl.d/kvm.conf and reloaded the config with sudo sysctl --system
net.ipv4.ip_forward = 0 net.bridge.bridge-nf-call-arptables = 0 net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-call-iptables = 0
I initially had issues getting wicked to work properly with the VLANs (see https://bugzilla.suse.com/show_bug.cgi?id=939580) I upgraded wicked from OBS and it was better after that although using ifup was still unreliable. systemctl restart network.service works much better for now.
It turns out wicked network manager has some nice features too. I particularly like this one as an alternative to ifconfig:
$ sudo wicked ifstatus all ... br69 up link: \#8, state up, mtu 1500 type: bridge config: compat:suse:/etc/sysconfig/network/ifcfg-br69 ... enp3s0.69 enslaved link: \#11, state up, mtu 1500, master br69 type: vlan enp3s0\[69\], hwaddr 67:89:d5:65:1e:66 config: compat:suse:/etc/sysconfig/network/ifcfg-enp3s0.69 vnet1 device-unconfigured link: \#12, state up, mtu 1500, master br69 type: tap, hwaddr fe:54:00:ac:0a:29