Send in by Ruben Daniels

Lustre is one of the most popular upcoming open source cluster file systems out there. When you want to run Xen’s from a SAN using Lustre you need to support both in the Linux kernel. Both XEN and Lustre are near mature products. This means there is support for it. But it’s quite difficult to find the right source and to combine it with the right kernel source of each. It took me a week of trial and erroring until I found a combination that worked. Since Google wasn’t much help I wrote this article so it might help you. This installation is Ubuntu Gutsy specific. You can start out with a basic Gutsy installation. Hardy is getting Lustre support, but at the time of this writing the package doesnt match the default kernel of Hardy.

First we apt-get some utilities and Xen packages

apt-get install quilt libc6-xen libxen3.1 linux-image-xen linux-restricted-modules-xen linux-xen python-xen-3.1 xen-docs-3.1 xen-hypervisor-3.1 xen-image-2.6.19-4-server xen-ioemu-3.1 xen-tools xen-utils-3.1

Now we get the kernel from kernel.org

cd /usr/src
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.22.tar.gz
tar -zxvf linux-2.6.22.tar.gz

The best matching (and stable) lustre source I’ve found is in the Hardy Lustre source package. So we get it and extra it manually:

wget http://ubuntu2.cica.es/ubuntu/ubuntu/pool/universe/l/lustre/lustre-source_1.6.4.2-1_all.deb
dpkg -x lustre-source_1.6.4.2-1_all.deb lustre-1.6.4.2
bunzip2 lustre-1.6.4.2/usr/src/lustre.tar.bz2
tar -xvf lustre-1.6.4.2/usr/src/lustre.tar
mv lustre-1.6.4.2/usr/src/modules/lustre/* lustre-1.6.4.2
rm -R lustre-1.6.4.2/usr -R

The official XEN kernel version is 2.6.18. This is way too old for newer machines. So we’ll get the Gutsy kernel source which contains XEN patches for our 2.6.22 kernel and apply them:

apt-get source linux-source-2.6.22
cd /usr/src/linux-source-2.6.22-2.6.22
cd debian/binary-custom.d/xen/patchset
patch -p1 /usr/src/linux-2.6.22 < 001*
patch -p1 /usr/src/linux-2.6.22 < 002*
patch -p1 /usr/src/linux-2.6.22 < 003*

Now we apply the lustre patches:

cd /usr/src/linux-2.6.22
ln -s /usr/src/lustre-1.6.4.2/lustre/kernel_patches/series/2.6.22-vanilla.series series
ln -s /usr/src/lustre-1.6.4.2/lustre/kernel_patches/patches patches
quilt -av push

Make the Kernel:

make distclean
cp debian/binary-custom.d/xen/patchset/config.i386 .
make menuconfig
make
make install modules_install

If you use kernel modules you need to run the following command to create the ram fs.

mkinitramfs -o /boot/initrd.img-2.6.22 2.6.22

Add the kernel to grub’s menu.lst. You could do this manually, but let’s just use the ubuntu script.

update-grub

Reboot the machine. After reboot build Lustre:

cd /usr/src/lustre-1.6.4.2
./configure --with-linux=/usr/src/linux-2.6.22
make
make install

Reboot the machine again. You should now have a working lustre/xen kernel.