Managing RAID on Linux - Derek Vadala [55]
As I mentioned in Chapter 2, it's advisable to spread member disks across multiple I/O channels whenever possible. This not only increases array performance but, in the case of a mirror, also helps prevent array failure if an I/O channel becomes unavailable because of a disk controller failure or faulty cabling. For example, imagine that /dev/sda and /dev/sdb are member disks of /dev/md0, a RAID-1, and that they are connected to the same controller A. If controller A fails, then all of /dev/md0 becomes unavailable.
Now imagine that each disk is connected to its own controller. For instance, suppose /dev/sda is connected to controller A and /dev/sdb is connected to controller B. In this case, either controller A or B can fail without crashing /dev/md0 (see Figure 3-5), because the other controller's disk is still operational.
Figure 3-5. Using multiple controllers help prevent downtime.
Dispersing disks among multiple controllers is equally important when working with hybrid arrays. For example, take the case of a RAID-10 (mirrors combined into a stripe). Assume that we have two mirrors that each contain two member disks. /dev/md0 contains /dev/sda and /dev/sdb, and /dev/md1 contains /dev/sdc and /dev/sdd. If we placed both /dev/sda and /dev/sdb on controller A and the remaining disks on controller B, a failure of either controller would crash /dev/md2, our RAID-0 (Figure 3-6). Since a RAID-0 cannot withstand any disk failures, the entire RAID-10 would also become unavailable.
Figure 3-6. If disks are not arranged intelligently, a controller failure will crash a RAID-10.
But what if we arranged our mirrors so that /dev/sda and /dev/sdc were on controller A and /dev/sdb and /dev/sdd were on controller B? (See Figure 3-7.) In this case, the failure of a single controller would only place our mirrors into degraded mode, leaving /dev/md2 operational.
Figure 3-7. Distributing disks means you can survive controller loss.
Remember, you don't have to rearrange hardware to facilitate this arrangement. Just create an /etc/raidtab or mdadm command that reflects the physical layout of your hardware.
RAID-10 (striped mirror)
Since /etc/raidtab is parsed in order, it is essential to define arrays from the bottom up. That is, arrays that are also component disks should appear before the array that contains them. For example, a simple /etc/raidtab that defines a RAID-10 (two mirrors combined into a stripe) might look like this:
A RAID-10: Two 2-disk mirrors are combined into a RAID-0
# The first mirror
raiddev /dev/md0
raid-level 1
chunk-size 64
nr-raid-disks 2
device /dev/sdb1
raid-disk 0
device /dev/sdc1
raid-disk 1
# The second mirror
raiddev /dev/md1
raid-level 1
chunk-size 64
nr-raid-disks 2
device /dev/sdd1
raid-disk 0
device /dev/sde1
raid-disk 1
# The mirrors (/dev/md0 and /dev/md1 are
# combined into a RAID-0.
raiddev /dev/md2
raid-level 0
chunk-size 64
persistent-superblock 1
nr-raid-disks 2
device /dev/md0
raid-disk 0
device /dev/md1
raid-disk 1
Given the preceding file, run mkraid on both of the mirrors (/dev/md0 and /dev/md1) and finally on the stripe (/dev/md2). The same feat can be accomplished with the following mdadm commands:
# mdadm -C -n2 -l1 /dev/md0 /dev/sd{b,c}1
# mdadm -C -n2 -l1 /dev/md1 /dev/sd{d,e}1
# mdadm -C -n2 -l0 -c64 /dev/md2 /dev/md{0,1}
After each RAID-1 is initialized using mkraid or mdadm, it will commence resynchronization. /proc/mdstat should report two-disk mirrors at /dev/md0 and /dev/md1 and a RAID-0 at /dev/md2, which consists of /dev/md0 and /dev/md1.
# cat /proc/mdstat
Personalities : [raid0] [raid1]
read_ahead 1024 sectors
md2 : active raid0 md1[1] md0[0]
35840640 blocks 64k chunks
md1 : active raid1 sde1[1] sdd1[0]
17920384 blocks [2/2] [UU]
md0 : active raid1 sdc1[1] sdb1[0]
17920384 blocks [2/2] [UU]
unused devices: Once all three arrays are activated,