Howto Install Xen+Lustre on Ubuntu Gutsy

by Arnold Daniels on 04/21/2008

Send in by Ruben Daniels

Lustre is one of the most popular upcoming open source cluster file systems out there. When you want to run Xen’s from a SAN using Lustre you need to support both in the Linux kernel. Both XEN and Lustre are near mature products. This means there is support for it. But it’s quite difficult to find the right source and to combine it with the right kernel source of each. It took me a week of trial and erroring until I found a combination that worked. Since Google wasn’t much help I wrote this article so it might help you. This installation is Ubuntu Gutsy specific. You can start out with a basic Gutsy installation. Hardy is getting Lustre support, but at the time of this writing the package doesnt match the default kernel of Hardy.

First we apt-get some utilities and Xen packages

apt-get install quilt libc6-xen libxen3.1 linux-image-xen linux-restricted-modules-xen linux-xen python-xen-3.1 xen-docs-3.1 xen-hypervisor-3.1 xen-image-2.6.19-4-server xen-ioemu-3.1 xen-tools xen-utils-3.1

Now we get the kernel from kernel.org

cd /usr/src
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.22.tar.gz
tar -zxvf linux-2.6.22.tar.gz

The best matching (and stable) lustre source I’ve found is in the Hardy Lustre source package. So we get it and extra it manually:

wget http://ubuntu2.cica.es/ubuntu/ubuntu/pool/universe/l/lustre/lustre-source_1.6.4.2-1_all.deb
dpkg -x lustre-source_1.6.4.2-1_all.deb lustre-1.6.4.2
bunzip2 lustre-1.6.4.2/usr/src/lustre.tar.bz2
tar -xvf lustre-1.6.4.2/usr/src/lustre.tar
mv lustre-1.6.4.2/usr/src/modules/lustre/* lustre-1.6.4.2
rm -R lustre-1.6.4.2/usr -R

The official XEN kernel version is 2.6.18. This is way too old for newer machines. So we’ll get the Gutsy kernel source which contains XEN patches for our 2.6.22 kernel and apply them:

apt-get source linux-source-2.6.22
cd /usr/src/linux-source-2.6.22-2.6.22
cd debian/binary-custom.d/xen/patchset
patch -p1 /usr/src/linux-2.6.22 < 001*
patch -p1 /usr/src/linux-2.6.22 < 002*
patch -p1 /usr/src/linux-2.6.22 < 003*

Now we apply the lustre patches:

cd /usr/src/linux-2.6.22
ln -s /usr/src/lustre-1.6.4.2/lustre/kernel_patches/series/2.6.22-vanilla.series series
ln -s /usr/src/lustre-1.6.4.2/lustre/kernel_patches/patches patches
quilt -av push

Make the Kernel:

make distclean
cp debian/binary-custom.d/xen/patchset/config.i386 .
make menuconfig
make
make install modules_install

If you use kernel modules you need to run the following command to create the ram fs.

mkinitramfs -o /boot/initrd.img-2.6.22 2.6.22

Add the kernel to grub’s menu.lst. You could do this manually, but let’s just use the ubuntu script.

update-grub

Reboot the machine. After reboot build Lustre:

cd /usr/src/lustre-1.6.4.2
./configure --with-linux=/usr/src/linux-2.6.22
make
make install

Reboot the machine again. You should now have a working lustre/xen kernel.

Arnold Daniels

I've spend a big part of my life behind a computer, learning about databases (MySQL), programming (PHP) and system administration (Linux). Currently I playing with HTML5, jquery and node.js.

E-mailTwitterLinkedInGithubGittip

There are 5 comments in this article:

  1. 27 June 2008Alex says:

    Any way you could update this for hardy heron? If so that would be awesome!

    ReplyReply
  2. 27 June 2008Arnold Daniels says:

    Hi Alex,

    Unfortunately we don’t have time to go into that here.

    You might be able to find out how to do it yourself based on this tutorial. It shouldn’t be too different. If so, please post a comment and/or send me the howto and I’ll be sure to post it under your name.

    - Arnold

    ReplyReply
  3. 12 October 2008Renato Leite says:

    Hi Arnold,

    First of all, thanks for your post! It helped me a lot to figure out how to build Lustre.
    Actually, I had to do some several changes to adapt it to Ubuntu Hardy (this could help out friend Alex – previews post).

    I created a quick tutorial (and It works!) to build a new kernel and install lustre in Hardy (see the link for my blog).

    It’s still in my country language (portuguese), but I`ll create a english version.

    Thanks againg and I hope It will be helpful.
    - Renato Leite

    ReplyReply
  4. 12 October 2008Renato Leite says:

    Well… I posted an english version for my tutorial!

    “Ubuntu Hardy – How To Build Lustre and its own Kernel (english version)”

    Have FUN!
    - Renato Leite

    ReplyReply
  5. 29 April 2011xentutorial.com says:

    Howto Install Xen+Lustre on Ubuntu Gutsy…

    Lustre is one of the most popular upcoming open source cluster file systems out there. When you want to run Xen’s from a SAN using Lustre you need to support both in the Linux kernel. Both XEN and Lustre are near mature products. This means there is su…

Write a comment: