svētdiena, 2012. gada 1. jūlijs

OpenVZ on KVM virtual machine. Network configuration

Required scenario:

  • Build Linux machines as OpenVZ containers  (CT101, CT102, ... ) on OpenVZ host, which itself is a KVM virtual machine KvmH on KVM virtualization host HstA. 
  • Both OpenVZ host KvmH and KVM host HstA uses internal network by default, but OVZ virtual hosts should be direct connected to external net or other internal nets, which are separated by VLAN's. 
  • The KVM host HstA should be connected to switch with two ethernet cards, which are bonded for redundancy/bandwidth/NAS reasons. 

HstA network configuration

OS - Ubuntu 12.04 server

... install ifenslave, bridge-utils, vlan ...

/etc/network/interfaces: 
# Used by ifup(8) and ifdown(8). See the interfaces(5) manpage or
# /usr/share/doc/ifupdown/examples for more information.

auto lo 

iface lo inet loopback

auto bond0
iface bond0 inet manual
    post-up    ifenslave bond0 eth0 eth1
    pre-down ifenslave -d bond0 eth0 eth1
    dns-nameservers 10.0.0.1
    dns-search internal.example.com

## - br0 IntLAN A  - on default VLAN
auto br0
iface br0 inet manual
    up ifconfig  bond0 up
    up brctl addbr br0
    up brctl addif br0 bond0
    up brctl stp  br0 on
    up ifconfig br0 10.0.0.11 netmask 255.255.0.0
    up route add default gw 10.0.0.1
    down brctl delbr br0

## - br4 - on tagged ExtLAN VLAN4 -  (192.0.2.0/24)
auto vlan4
iface vlan4 inet manual
    up ifconfig vlan4 up
    vlan_raw_device bond0

auto br4
iface br4 inet manual
    up ifconfig vlan4 up
    up brctl addbr br4
    up brctl addif br4 vlan4
    up brctl stp br4 on
    up ifconfig br4 0.0.0.0 up
    down brctl delif br4 vlan4
    down brctl delbr br4

## - br6 - on tagged intLAN B VLAN6 -  (192.168.1.0/24)
auto vlan6
iface vlan6 inet manual
    up ifconfig vlan6 up
    vlan_raw_device bond0

auto br6
iface br6 inet manual
    up ifconfig vlan6 up
    up brctl addbr br6
    up brctl addif br6 vlan6
    up brctl stp br6 on
    # up ifconfig br6 192.168.1.2 netmask 255.255.255.0
    up ifconfig br6 0.0.0.0 up
    down brctl delif br6 vlan6
    down brctl delbr br6

##  - br8 - on tagged intLAN C VLAN8 (192.168.2.0/24)
auto vlan8
iface vlan8 inet manual
    up ifconfig vlan8 up
    vlan_raw_device bond0

auto br8
iface br8 inet manual
    up ifconfig vlan8 up
    up brctl addbr br8
    up brctl addif br8 vlan8
    up brctl stp br8 on
    # up ifconfig br8 192.168.2.2 netmask 255.255.255.0
    up ifconfig br8 0.0.0.0 up
    down brctl delif br8 vlan8
    down brctl delbr br8


/etc/modprobe.d/bonding.conf
alias bond0 bonding
options bonding mode=4 miimon=100

KvmH network configuration:


OS - CentOS 6.2

Virtual NIC eth0 is connected to br0 on HstA
Virtual NIC eth1 is connected to br4 on HstA

... install vzkernel vzctl vzquota bridge-utils ...
... configure/disable iptables, allow ip_forwarding via systctl ...

/etc/sysconfig/network-scripts/ifcfg-eth0:
DEVICE="eth0"
TYPE="Ethernet"
HWADDR="52:54:00:E3:AB:CD"
BOOTPROTO=none
ONBOOT="yes"
NM_CONTROLLED="no"
TYPE="Ethernet"
BRIDGE=vzbr0

/etc/sysconfig/network-scripts/ifcfg-vzbr0:
DEVICE=vzbr0
TYPE=Bridge
IPADDR=10.0.0.4
NETMASK=255.255.0.0
ONBOOT=yes
BOOTPROTO=static
NM_CONTROLLED=no
DEALAY=0

/etc/sysconfig/network-scripts/ifcfg-eth1:
DEVICE="eth1"
TYPE="Ethernet"
BOOTPROTO=none
ONBOOT="yes"
NM_CONTROLLED="no"
TYPE="Ethernet"
BRIDGE=vzbr4

/etc/sysconfig/network-scripts/ifcfg-vzbr4:
DEVICE=vzbr4
TYPE=Bridge
ONBOOT=yes
BOOTPROTO=static
NM_CONTROLLED=no
DEALAY=0

/etc/sysconfig/network: 
NETWORKING=yes
HOSTNAME=kvmh.int.test
GATEWAY=10.0.0.1

Prepare VZ to automatically add configured container veth interfaces to host bridges 

http://wiki.openvz.org/Virtual_Ethernet_device#Making_a_bridged_veth-device_persistent

Just create /etc/vz/vznet.conf containing the following. 
#!/bin/bash
EXTERNAL_SCRIPT="/usr/sbin/vznetaddbr"

Build containers:


Download OpenVZ template:

cd /vz/template/cache
wget http://download.openvz.org/template/precreated/ubuntu-12.04-x86.tar.gz

Create container:

vzctl create 101 --ostemplate ubuntu-12.04-x86

Add veth interface:

vzctl set 101 --netif_add eth0,,,,vzbr0 --save

Start container and check if veth101.0 is added to bridge vzbr0
vzctl start 101
brctl show

Go inside container 101 (vzctl enter 101), and configure eth0 as usualy ( .. /etc/network/interfaces)

Add other containers and bridges:

vzctl create 102 --ostemplate ubuntu-12.04-x86
vzctl set 102 --netif_add eth0,,,,vzbr4 --save


vzctl create 145 --ostemplate ubuntu-12.04-x86
vzctl set 145 --netif_add eth0,,,,vzbr0 --save
vzctl set 145 --netif_add eth1,,,,vzbr4 --save




plkst.