Grub and root on partitionable RAID1 array

To make a long story short — don’t do it, it won’t work. ­čÖé

And now the long version of the whole story. The following applies to Debian 4.0 aka “Etch,” but it’s not specific to Debian, since the principles of the whole procedure is the same on all Linux distributions.

For the first time I wanted to use a partitionable RAID1 array, because I wanted to spare myself the trouble of creating multiple partitions on both drives which would then be combined to individual RAID1 arrays. Instead I would have to create a single partitionable RAID1 array and partition it afterwards.

I had an existing non-RAID system on the first of two 20 GB hard drives. To construct a level-1 RAID you need two drives — however I only had a single spare drive. So I created the RAID1 array in degraded mode, i. e. with only a single drive and the other one missing. To do this, I created a single primary partition, extending across the whole drive, on the spare/empty drive. I then created the array as follows:

#mdadm --create /dev/md_d0 --level=1 --raid-devices=2 missing /dev/hdb1

This array now needed to be partitioned, so I invoked cfdisk on it:

#cfdisk /dev/md_d0

I then created filesystems on the partitions like follows:

#mkfs -t xfs -L "/usr" /dev/md_d0p6

Afterwards I copied the existing filesystems to the just-created filesystems on the degraded array using cp -ax /usr /tmp/raid/.

I then changed /etc/fstab to mount the filesystems from the (degraded) array instead of from /dev/hda. I also changed Grub’s menu.lst to refer to the root on the RAID1 volume (/dev/md_d0p1).

Finally, I rebooted. All was well. ­čÖé

I then partitioned the first hard drive /dev/hda the same way as described above and added the partition to the array:

#mdadm -A /dev/md_d0 /dev/hda1

After some minutes during which the degraded array on /dev/hdb1 was copied over to /dev/hda1, the array was consistent and clean.

Now for the final boot… I was holding my breath… Failure! ­čÖü

For “some” reason Grub couldn’t load stage1. I took a Knoppix 5.x CD and tried to “repair” my setup. Suddenly I realized that there wasn’t anything to be repaired — this setup simply can’t work.

And here is why:

Other than LILO, Grub can directly read filesystems. It doesn’t rely upon a list of blocks that it needs to read consecutively (as LILO does,) but you can specify pathnames for modules on an existing filesystem that Grub will then read. However, in my setup there is no filesystem to be read.

Why is this???

Well, remember that I created a partitionable array, i. e. /dev/hda1 doesn’t contain a filesystem that Grub could parse to load the kernel and initial ramdisk from. Grub would have to assemble the RAID1 array first, and then access the root filesystem in the first partition on this partitionable array. However, this functionality current doesn’t exist in Grub, and I’m not sure whether it will ever be included.

When I realized that my setup wouldn’t work, I went back to the original setup of my system. The whole procedure cost me about 3 hours — which were well spent, because I learnt how this “partitionable array” stuff works.

6 thoughts on “Grub and root on partitionable RAID1 array”

  1. Please note the following comment in the howto you pointed me to:

    Make sure the kernel has RAID-1 (and LVM if using it) built in. You cannot use an initrd and modules. The kernel will only auto-mount raid arrays if the md module is built in.

    This is why it won’t work under Debian — at least not as long as you want to stick to “doing it the Debian way.” Debian uses an initrd, and md is loaded as a module.

  2. > Make sure the kernel has RAID-1 (and LVM if using it) built in.

    I don’t see why that’s a necessary step. There’s no reason an initrd couldn’t start a partitioned md device. Yeah, dpkg-reconfigure mdadm asks you what md devices the system needs to start, so just answer /dev/md_d0. Put root=/dev/md_d0p1 on your kernel command line.

    The major downside of this is that the whole drive has to be in RAID1. I usually RAID1 my /, /usr (or however I decided to partition on that machine), and /home, and RAID0 or lvm for the rest (which I usually call something like /data, and /var/tmp).

  3. @Peter Cordes: Thinking about it again I in fact don’t see any reason either why this shouldn’t work — provided that Grub can load the initrd.

    Are you sure that this is the case?

  4. Grub 1 does not support booting from partitionable md devices, and neither does Grub 2! At the very least, as of today, the grub-probe and grub-install programs cannot yet handle /dev/md/d0 (AKA /dev/md_d0) style device names. There may be even deeper problems, but this alone has prevented me from setting up a system with partitionable md arrays today. Here’s my Debian bug report in case you’re interested: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=509960

    I then went back to creating multiple RAID component partitions and building multiple md devices from them.

  5. @Julian Mehnle: That’s exactly what I did: Create separate md devices for the no. of partitions I needed.

Leave a Reply

Your email address will not be published. Required fields are marked *