Xen PCI Passthrough with XE / XL

Xen PCI Passthrough with XE / XL

PCI passthrough allows you to give control of physical devices to guests: that is, you can use PCI passthrough to assign a PCI device (NIC, disk controller, HBA, USB controller, firewire controller, soundcard, etc) to a virtual machine guest, giving it full and direct access to the PCI device.

This has several potential uses, including the following: You can set up a Driver Domain, which can increase both security and reliability of your system.

Steps:

1) Load xen-pciback module in dom0

dom0 kernel configuration for PCI passthrough –

CONFIG_XEN_PCIDEV_FRONTEND=m
CONFIG_XEN_PCIDEV_BACKEND=m

load xen-pciback module

root@dom0:modprobe -i xen-pciback

To verify:

root@dom0:~# lsmod | grep xen

xen_pciback 52011 0

2) Create domU with xe commands-


root@dom0:~# HOST-UUID=`xe host-list --minimal`

root@dom0:~# SR-UUID=`xe sr-create content-type=user device-config:device=/dev/sda4 host-uuid=$HOST-UUID name-label="Local Storage" shared=true type=ext`

root@dom0:~# VM-UUID=`xe vm-install template=dom0\ Lucid\ Lynx\ 10.04\ \(64-bit\) sr-uuid=$SR-UUID new-name-label=my-domU`

root@dom0:~# NETWORK-UUID=`xe network-list bridge=xenbr0 --minimal`

root@dom0:~# xe vif-create mac=random device=0 network-uuid=$ NETWORK-UUID vm-uuid=$VM-UUID

root@dom0:~# xe vm-param-set other-config:debian-release=precise uuid=$VM-UUID

root@dom0:~# xe vm-param-set other-config:install-repository=http://archive.ubuntu.com/ubuntu/ uuid=$VM-UUID

root@dom0:~# xe vm-param-set other-config:disks="" uuid=$VM-UUID

root@dom0:~# xe vm-param-set memory-static-max=1024000000 uuid=$VM-UUID

root@dom0:~# xe vm-param-set memory-dynamic-max=1024000000 uuid=$VM-UUID

root@dom0:~# xe vm-param-set memory-dynamic-min=512000000 uuid=$VM-UUID

root@dom0:~# xe vm-param-set memory-static-min=512000000 uuid=$VM-UUID

root@dom0:~# xe vm-start uuid=$VM-UUID

To check vm status-

root@dom0:/# xe vm-list

state should be running

Access VM console using-

/usr/lib/xen-4.2/bin/xenconsole 

get domain-id from

root@dom0:/# xe vm-param-list uuid=$VM-UUID | grep dom-id

Then continue with OS installation.

Shutdown VM after installation completed.

root@dom0:/# xe vm-shutdown uuid=$VM-UUID

3) List lspci devices on dom0

root@dom0:/# lspci

00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)

00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09)

00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)

00:19.0 Ethernet controller: Intel Corporation 82579V Gigabit Network Connection (rev 04)

00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 04)

00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 04)

00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b4)

00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 04)

00:1f.0 ISA bridge: Intel Corporation H61 Express Chipset Family LPC Controller (rev 04)

00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller (rev 04)

00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 04)

01:00.0 PCI bridge: Integrated Technology Express, Inc. Device 8892 (rev 30)

4) Resolve linking issues with xl:

root@dom0:/# unlink /usr/sbin/xl

root@dom0:/# ln -s /usr/lib/xen-4.1/bin/xl /usr/sbin/

5) PCI passthrough

Add string “iommu=soft” in PV-args parameter of VM

To get PV-args

root@dom0:/# xe vm-param-list uuid= | grep PV-args

root@dom0:/# xe vm-param-set uuid= PV-args="root=UUID=d64ef070-24ee-418a-99f4-1ebbb345159e ro quiet iommu=soft"

For single device-

root@dom0:/# xe vm-param-set other-config:pci=0/0000:00:1b.0 uuid=

5) Start domU and check passed PCI devices with lspci

root@dom0:/# xe vm-start uuid=

Access console of domU

root@domU:/#lspci
00:01.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 04)

Issues and workaround

Issue :

1) Unable to pass through multiple pci devices with

root@dom0:/# xe vm-param-set other-config:pci=0/0000:00:1b.0,1/0000:00:1d.0,2/0000:03:00.0 uuid=11ee7266-1de8-a8fa-f786-db3dce38680c

Here 0/ 1/ 2/ is index for order in which device will appear in domU.

Workaround :

Pass pci device one at a time using XL :

# xl pci-attach  

.

Example :

For Ubuntu 13.04, add PCI device in assignable

root@dom0:~# xl pci-assignable-add 00:1b.0

root@dom0:~# xl pci-attach driver-domain 00:1b.0

For Ubuntu 12.04, hide PCI device from dom0

Modify /boot/grub/grub.cfg

module /vmlinuz-3.4.1 placeholder root=/dev/mapper/dakshinaila-root ro xen- pciback.hide=(00:1b.0) intel_iommu=on

Note : The important part here is the xen-pciback.hide parameter that identifies the PCI device to hide.

root@dom0:~# xl pci-attach driver-domain 00:1b.0
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s