Easy Ubuntu setup with software RAID

11 December 2008

It seems that the usual way to use software RAID in Ubuntu is to navigate the menus of the Ubuntu Alternate Install CD as discussed in a HOWTO in the Ubuntu Forums and in an article on Security Viewpoints.

The following is a more satisfying approach because it is run entirely from the command-line. It is also suitable for a remote installation.

For background information on RAID, including the various types (e.g. software vs. hardware) and levels (RAID0, RAID1, RAID2… etc.) see the article on RAID in Wikipedia. You may also consult the Wikipedia page specifically dealing with Standard RAID levels.

(1) The general idea

We will install Ubuntu 8.10 (Intrepid Ibex) onto two hard drives strung together in a RAID0 array. With some tweaking, this general protocol should work for other RAID levels and other distros. In the case of RAID0, we get a virtual disk that’s roughly the sum of the two constituent drives.

For example, suppose you have two hard drives, each with exactly 500GB capacity and you format them as follows:

  • Disk 1: 0.1GB boot, 2.0GB swap, 497.9GB bulk reserved for RAID0
  • Disk 2: entire 500GB reserved for RAID0

The resulting virtual RAID0 disk will (only) be 497.9GB + 497.9GB = 995.8GB, which reflects the fact that the space contributed by each disk is limited to the size of the smallest disk (or partition). Thus, for simplicity, we’ll format both drives in the same manner even though, for RAID0, the boot and swap partitions will unused on the second hard drive (you’ll see why below). Moreover, the the symmetrical hard drive formatting will make it easier to adapt this procedure to RAID1.

The following instructions will destroy your data, so make backups before proceeding!!

(2) Partition the disks

Boot the computer into an environment that doesn’t depend on the hard drives. For the purposes of these instructions, we’ll assume that’s you’ve booted into the Ubuntu Live CD environment. (Note: If your computer is a managed server and you don’t have physical access to the machine, you’ll need to boot into a special recovery mode that has sufficient functionality to do the following.)

Be the root user:

sudo -i

Discover the device names of your hard drives by examining the output from dmesg. If your drives are SATA-connected, they will be called something like /dev/sda and /dev/sdb.

dmesg | grep sd

Or, if you’ve got IDE hard drives rather than SATA, look at:

dmesg | grep hd

Setting variables will cut down on mistakes later. Substitute your actual device names, of course. The MY_DISK1 should be a hard drive that the BIOS will look to boot from.

export MY_DISK1=/dev/sdX MY_DISK2=/dev/sdY

Now, we want to format our disks to look something like this:

  • 100 MB for /boot
  • 2 GB for swap
  • the rest for RAID

You could issue these commands manually into fdisk:

  1. Enter o to clear the partition table.
  2. Enter n, then p for primary, then 1, then hit enter for the default, then +100M for a 100 megabyte chunk.
  3. Enter n, then p for primary, then 2, then hit enter for the default, then +2G for a 2 gigabyte chunk.
  4. Enter n, then p for primary, then 3, then hit enter twice for the defaults, to use the remainder of the disk.
  5. Enter t, then 1, then 83 for a normal Linux parittion.
  6. Enter t, then 2, then 82 for a Linux swap parittion.
  7. Enter t, then 3, then fd for a Linux RAID parittion.
  8. Enter a, then 1, to set bootable flag on the first partition.
  9. Enter p to show you the partitions.
  10. Enter w to write the partition data and quit.

…but why would you want to? All those inputs can be summarized by:

export MY_FORMAT="o\nn\np\n1\n\n+100M\nn\np\n2\n\n+2G\nn\np\n3\n\n\nt\n1\n83\nt\n2\n82\nt\n3\nfd\na\n1\np\nw\n"

Then you can run the exact commands against each disk, which cuts down on typos:

echo -e $MY_FORMAT | fdisk $MY_DISK1

Check the output for errors. You should see something like:

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          14      112423+  83  Linux
/dev/sda2              15         276     2104515   82  Linux swap / Solaris
/dev/sda3             277         522     1975995   fd  Linux raid autodetect

If everything looks good, run it on your other disk:

echo -e $MY_FORMAT | fdisk $MY_DISK2

(3) Create the RAID array

Install mdadm (within your live CD environment), the excellent utility that will manage the RAID disks. (n.b. This may not be necessary if your boot environment already has mdadm.)

apt-get -y install mdadm

Create a two-part RAID0 array, resulting in the virtual device /dev/md0:

mdadm --verbose --create /dev/md0 --level=0 --raid-devices=2 ${MY_DISK1}3 ${MY_DISK2}3

This new virtual device will be the root device (mount point /). Both the boot and the swap partitions will come from disk 1.

export MY_ROOT=/dev/md0 MY_BOOT=${MY_DISK1}1 MY_SWAP=${MY_DISK1}2

(4) Format the partitions

Format the boot partition as ext2:

mkfs.ext2 $MY_BOOT

Format the swap space:

swapoff -a && mkswap $MY_SWAP

Format the root disk as ext3:

mkfs.ext3 $MY_ROOT

(5) Install Ubuntu

Mount the disks:

mount $MY_ROOT /mnt && mkdir /mnt/boot && mount $MY_BOOT /mnt/boot

We will now use debootstrap, which is a tool used in the Debian family of Linux distros (of which Ubuntu is a member) to install a base system.

We need binutils (The GNU assembler, linker and binary utilities) to run debootstrap:

apt-get -y install binutils

Consult the Debian archive to find the most recent version, then get it.

wget http://ftp.debian.org/debian/pool/main/d/debootstrap/debootstrap_1.0.10_all.deb

Install it:

dpkg -i debootstrap_1.0.10_all.deb

Choose a degree of permissiveness of source repositories. The following will enable all repositories.

export MY_SOURCES="main,restricted,universe,multiverse"

Choose a distro and an architecture

export MY_DISTRO="intrepid" MY_ARCH="i386"

Choose a Ubuntu mirror near to you (this can make a serious difference in speed).

export MY_MIRROR="http://archive.ubuntu.com/ubuntu/"

Choose a few packages to include in your base installation (n.b. these should be available within your MY_SOURCES). This sets up a minimal English-language machine with SSH, grub and, of course, mdadm. (Hint: Anything that you can apt-get, you can put in here.)

export MY_BASE="language-pack-en,language-pack-en-base,linux-image,grub,ssh,mdadm"

Perform the debootstrap-driven installation.

debootstrap --include=$MY_BASE --components=$MY_SOURCES --verbose --arch $MY_ARCH $MY_DISTRO /mnt $MY_MIRROR

(6) Configure fstab, network, and sources

Use this little piece of script to collect the UUIDs of the relevant drives.

for var in ROOT BOOT SWAP; do i=MY_$var; x=`eval echo \\$$i`; export MY_${var}_UUID=`blkid -o value -s UUID $x`; done

You can look at all the variables that you’ve set:

env | grep MY_

Auto-generate the fstab based on the UUIDs collected.

cat > /mnt/etc/fstab << EOF
# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    defaults        0       0
# entry for $MY_ROOT
UUID=$MY_ROOT_UUID /               ext3    relatime,errors=remount-ro 0       1
# entry for $MY_BOOT
UUID=$MY_BOOT_UUID /boot           ext2    relatime        0       2
# entry for $MY_SWAP
UUID=$MY_SWAP_UUID none            swap    sw              0       0
EOF

The following sets up a basic network interfaces file with a static IP address (which, of course, you’ll have to customize a little). If this setup is not appropriate for you, read Vivek Gite’s article for a detailed explanation or hunt around on your favourite search engine.

cat > /mnt/etc/network/interfaces << EOF
# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
  address 192.168.1.199
  netmask 255.255.255.0
  network 192.168.1.0
  broadcast 192.168.1.255
  gateway 192.168.1.1
EOF

The sources.list file wants space-separated sources so modify the $MY_SOURCES:

export MY_SOURCES=`echo $MY_SOURCES | sed s/,/\ /g`

Auto-generate the sources.list file:

cat > /mnt/etc/apt/sources.list << EOF
deb $MY_MIRROR intrepid $MY_SOURCES
deb-src $MY_MIRROR intrepid $MY_SOURCES
deb $MY_MIRROR intrepid-updates $MY_SOURCES
deb-src $MY_MIRROR intrepid-updates $MY_SOURCES
deb $MY_MIRROR intrepid-security $MY_SOURCES
deb-src $MY_MIRROR intrepid-security $MY_SOURCES
EOF

(7) Finish up

Set your host name:

echo my_host_name > /mnt/etc/hostname

Create a master boot record. (It won’t hurt to do so on both disks.)

grub-install --root-directory=/mnt --no-floppy $MY_DISK1
grub-install --root-directory=/mnt --no-floppy $MY_DISK2

Mount a few key things and change root into your new system:

for dir in proc dev sys; do mount --bind /$dir /mnt/$dir; done
chroot /mnt

Update your packages

apt-get update

Upgrade your packages. This might take a while, so you can defer it until later.

apt-get -y upgrade

Set at least a root password. You may also like to add another user or two.

passwd
adduser username

Update the boot loader (the -y flag will answer yes to generating /boot/grub/menu.lst).

update-grub -y

Cross your fingers and reboot:

shutdown -r now
antipode

,

---

Comment

  1. grub-install --root-directory=/mnt --no-floppy $MY_DISK1 gives: root@ubuntu:~# grub-install --root-directory=/mnt --no-floppy $MY_DISK1 Could not find device for /mnt/boot: Not found or not a block device. The same for $MY_DISK2

    — Bart · 21 December 2008 · #

  2. @Bart: It sounds like your boot partition isn’t mounted.

    Note that the first command under (5) install ubuntu is supposed to (among other things) mount the boot partition under /mnt/boot.

    mount $MY_ROOT /mnt && mkdir /mnt/boot && mount $MY_BOOT /mnt/boot
    

    If you simply issue the command,

    mount
    

    you will see what’s mounted where. Correct output will include some lines like this:

    /dev/md0 on /mnt type ext3 (rw)
    /dev/sda1 on /mnt/boot type ext2 (rw)
    

    If you’ve issued the right mount commands and grub-install is failing, please post some more details, including the output of:

    echo $MY_BOOT
    

    … just to see if that variable is being set properly.

    — antipode · 23 December 2008 · #

  3. Thanks so much for this guide.

    I was having some trouble with Ubuntu’s automated installer and creating a RAID0 array, so dropping to the command line and using your guide helped. Also, the version of mount on the Ubuntu server install CD I was using required me to specify the filesystem with the -t flag,

    i.e. ‘mount -t ext3 /dev/md0 /mnt’ and ‘mount -t ext2 /dev/sda1 /mnt/boot’

    dejitarob · 30 December 2008 · #

  (Please preview your comment before submitting it.)
Individual article
---