Xen Hypervisor Setup

Steps:
1) Install Ubuntu on bare hardware
2) Install Xen
3) Compile kernel for Dom0
4) Create DomU

1) You need to install ubuntu on one machine. I am proceeding to next step assuming you have machine with ubuntu installed.

2) Install Xen:
I am going to download Xen source code and going to compile it, for this purpose I need some libraries installed on machine.

Prerequisites are-

    * GCC v3.4 or later
    * GNU Make
    * GNU Binutils
    * Development install of zlib (e.g., zlib-dev)
    * Development install of Python v2.3 or later (e.g., python-dev)
    * Development install of curses (e.g., libncurses-dev)
    * Development install of openssl (e.g., openssl-dev)
    * Development install of x11 (e.g. xorg-x11-dev)
    * Development install of uuid (e.g. uuid-dev)
    * bridge-utils package (/sbin/brctl)
    * iproute package (/sbin/ip)
    * hotplug or udev
    * GNU bison and GNU flex
    * GNU gettext
    * 16-bit x86 assembler, loader and compiler (dev86 rpm or bin86 & bcc debs)
    * ACPI ASL compiler (iasl)

root@dom0$ apt-get install bcc bin86 gawk bridge-utils iproute libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif texinfo texlive-latex-base texlive-latex-recommended texlive-fonts-extra texlive-fonts-recommended pciutils-dev mercurial build-essential make gcc libc6-dev zlib1g-dev python python-dev python-twisted libncurses5-dev patch libvncserver-dev libsdl-dev libjpeg62-dev iasl libbz2-dev e2fslibs-dev git-core uuid-dev ocaml libx11-dev bison flex xz-utils ocaml-findlib gcc-multilib

Download Xen 4.3.0 source code:

root@dom0# wget http://bits.xensource.com/oss-xen/release/4.3.0/xen-4.3.0.tar.gz

Now we can extract, build and install it:

root@dom0# tar xvf xen-4.3.0.tar.gz
root@dom0# cd xen-4.3.0
root@dom0# make xen
root@dom0# make tools
root@dom0# make stubdom
root@dom0# make install-xen
root@dom0# make install-tools PYTHON_PREFIX_ARG=
root@dom0# make install-stubdom

We should have the following files in ‘/boot’ directory:

/boot/xen.gz
/boot/xen-4.gz
/boot/xen-4.3.gz
/boot/xen-4.3.0.gz

Now, in the file ‘/etc/xen/xend-config.sxp’ we enable the following option:

(xend-unix-server yes)

3) Compile kernel for Dom0 :
Download kernel from kernel org:

root@dom0# wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.7.1.tar.gz

Extract it

root@dom0# tar xvf linux-3.7.1.tar.gz

we can configure our new kernel by running:
We need to set some parameters-

root@dom0# cd linux-3.7.1

root@dom0# make menuconfig

Processor type and features →
    High memory support (64GB)
       PAE (Physical Address Extension) Support - enabled

Processor type and features →
    Allocate 2nd-level pagetables from highmem - disabled

ACPI (Advanced Configuration and Power Interface) Support - enabled

Processor type and features →
     Paravirtualized guest support [y] →
          Xen guest support – enabled

Bus oprions-
     Xen PCI frontend – enabled

Device Drivers → 
     Block Devices [*] → 
           Xen virtual block device support – enabled 
           Block-device backend driver – enabled 

     Network device support [*] → 
           Xen network device frontend driver – enabled 
           Xen backend network device – enabled

     Input device support →
           Miscellaneous devices →
                Xen virtual keyboard and mouse support – enabled

     Character devices →
           Xen Hypervisor console support – enabled

     Xen driver support →
           Xen memory balloon driver – enabled
           Scrub pages before returning them to system – enabled
           Xen /dev/xen/evtchn device Backend driver support – enabled
           Xen filesystem – enabled
           Create compatibility mount point /proc/xen – enabled
           Create xen entries under /sys/hypervisor – enabled
           userspace grant access device driver – enabled
           User-space grant reference allocator driver – enabled
           xen platform pci device driver – enabled

Below, is a list of options needed to compile Linux kernel with dom0 support:

CONFIG_ACPI_PROCFS=y 
CONFIG_XEN=y 
CONFIG_XEN_MAX_DOMAIN_MEMORY=32 
CONFIG_XEN_SAVE_RESTORE=y 
CONFIG_XEN_DOM0=y 
CONFIG_XEN_PRIVILEGED_GUEST=y 
CONFIG_XEN_PCI=y 
CONFIG_PCI_XEN=y 
CONFIG_XEN_BLKDEV_FRONTEND=y 
CONFIG_XEN_NETDEV_FRONTEND=y 
CONFIG_XEN_KBDDEV_FRONTEND=y 
CONFIG_HVC_XEN=y 
CONFIG_XEN_FBDEV_FRONTEND=y 
CONFIG_XEN_BALLOON=y 
CONFIG_XEN_SCRUB_PAGES=y 
CONFIG_XEN_DEV_EVTCHN=y 
CONFIG_XEN_GNTDEV=y 
CONFIG_XEN_BACKEND=y 
CONFIG_XEN_BLKDEV_BACKEND=y 
CONFIG_XEN_NETDEV_BACKEND=y 
CONFIG_XENFS=y 
CONFIG_XEN_COMPAT_XENFS=y 
CONFIG_XEN_XENBUS_FRONTEND=y 
CONFIG_XEN_PCIDEV_FRONTEND=y

Now, we can build and install the kernel:

user@dom0$ make
root@dom0$ make modules_install
root@dom0$ make install
root@dom0$ cd /boot
root@dom0$ mkinitramfs -o initrd.img-3.7.1 3.7.1
root@dom0$ update-grub

We can now reboot our system to the Xen enabled 3.7.1 Linux kernel. To verify that Xen is running we can do the following:

root@dom0$ cat /proc/xen/capabilities

The output must be same as the following line:

control_d

You can manually start xen daemons with-

service xencommons start

Try following commands:

root@dom0$ xl info
root@suksha:~# xl list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  1626     4     r-----    1065.2

If problem in startup:
To check for xen processes:

ps -ef | grep xen

Expected output is-

root@suksha:~/domsikernel# ps -ef | grep xen
root        31     2  0 Aug12 ?        00:00:00 [xenwatch]
root        32     2  0 Aug12 ?        00:00:00 [xenbus]
root        59     2  0 Aug12 ?        00:00:00 [xen_pciback_wor]
root      1282     1  0 Aug12 ?        00:00:03 /usr/local/sbin/oxenstored --pid-file /var/run/xenstored.pid
root      1288     1  0 Aug12 ?        00:00:08 /usr/local/sbin/xenconsoled --pid-file=/var/run/xenconsoled.pid
root      1292     1  0 Aug12 ?        00:00:04 /usr/local/lib/xen/bin/qemu-system-i386 -xen-domid 0 -xen-attach -name dom0 -nographic -M xenpv -daemonize -monitor /dev/null -serial /dev/null -parallel /dev/null -pidfile /var/run/qemu-dom0.pid
root     10421 10271  0 14:21 pts/3    00:00:00 /usr/local/lib/xen/bin/xenconsole 45 --num 0 --type pv
root     11214  8660  0 18:30 pts/4    00:00:00 grep --color=auto xen

If problem comes about library not found then do set LD_LIBRARY_PATH-

export LD_LIBRARY_PATH=/usr/local/lib/:/usr/lib/:/usr/lib64/:$LD_LIBRARY_PATH
ldconfig

additionally you need to install blktap driver, you can install it with-

apt-get install blktap-dkms

Edit /etc/modules add

blktap

So on startup blktap module will be loaded.
and you can see it by

lsmod | grep blktap

Now you have to reboot system with Xen and newly installed kernel for dom0
before that- you can increase timeout of choosing operating system on reboot.

Edit /etc/default/grub and set

GRUB_TIMEOUT=10

so now it will wait for 10 secs for your input.

Creation of xen bridge:
Edit /etc/network/interfaces

sample interface file-

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
#iface eth0 inet static

auto xenbr0
iface xenbr0 inet static
        address 172.30.30.52
        netmask 255.255.255.0
        #network 10.43.10.0
        broadcast 172.30.30.255
        gateway 172.30.30.254
        # dns-* options are implemented by the resolvconf package, if installed
        dns-nameservers 192.168.0.4
        dns-search org.com
        bridge_stp off
        bridge_ports eth0

And restart networking service

/etc/init.d/networking restart

Now its done, You have system with Xen 4.3.0 installed and Domain 0 up and running.

To create Paravirtualized VM on this Xen,
Follow my blog- https://pravinchavan.wordpress.com/2013/08/16/creating-paravirtualized-domain-on-xen/

Advertisements

8 responses to “Xen Hypervisor Setup

  1. Pingback: Creating Paravirtualized Domain on Xen | Pravinchavan's Blog·

  2. After
    make menuconfig

    I dont see these (why?):

    Processor type and features →
    High memory support (64GB)
    PAE (Physical Address Extension) Support – enabled

    Processor type and features →
    Allocate 2nd-level pagetables from highmem – disabled

    ACPI (Advanced Configuration and Power Interface) Support – enabled

    Processor type and features →
    Paravirtualized guest support [y] →
    Xen guest support – enabled

    Bus oprions-
    Xen PCI frontend – enabled

    Device Drivers →
    Block Devices [*] →
    Xen virtual block device support – enabled
    Block-device backend driver – enabled

    Network device support [*] →
    Xen network device frontend driver – enabled
    Xen backend network device – enabled

    Input device support →
    Miscellaneous devices →
    Xen virtual keyboard and mouse support – enabled

    Character devices →
    Xen Hypervisor console support – enabled

    Xen driver support →
    Xen memory balloon driver – enabled
    Scrub pages before returning them to system – enabled
    Xen /dev/xen/evtchn device Backend driver support – enabled
    Xen filesystem – enabled
    Create compatibility mount point /proc/xen – enabled
    Create xen entries under /sys/hypervisor – enabled
    userspace grant access device driver – enabled
    User-space grant reference allocator driver – enabled
    xen platform pci device driver – enabled

  3. After
    make menuconfig

    I dont see these (why?):

    Processor type and features →
    High memory support (64GB)
    PAE (Physical Address Extension) Support – enabled

    Processor type and features →
    Allocate 2nd-level pagetables from highmem – disabled

    ACPI (Advanced Configuration and Power Interface) Support – enabled

    Processor type and features →
    Paravirtualized guest support [y] →
    Xen guest support – enabled

    Bus oprions-
    Xen PCI frontend – enabled

    Device Drivers →
    Block Devices [*] →
    Xen virtual block device support – enabled
    Block-device backend driver – enabled

    Network device support [*] →
    Xen network device frontend driver – enabled
    Xen backend network device – enabled

    Input device support →
    Miscellaneous devices →
    Xen virtual keyboard and mouse support – enabled

    Character devices →
    Xen Hypervisor console support – enabled

    Xen driver support →
    Xen memory balloon driver – enabled
    Scrub pages before returning them to system – enabled
    Xen /dev/xen/evtchn device Backend driver support – enabled
    Xen filesystem – enabled
    Create compatibility mount point /proc/xen – enabled
    Create xen entries under /sys/hypervisor – enabled
    userspace grant access device driver – enabled
    User-space grant reference allocator driver – enabled
    xen platform pci device driver – enabled

  4. Pingback: Configuring Virtual TPM (vTPM) for Xen 4.3 Guest Virtual Machines. | mhsamsal·

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s