December 28, 2006

RAID

So I now have all these photos taken and miniDV video shot and collected over the past few years, it's adding up. So far about 25GB in photos and about 80gb of xvid video. I can't lose any of this, for the first time in my life there's data that I actually really really care not to lose. They sit on Yi's mac and on my workstation PC and is backed up on my Debian file server. The problem is my linux file server only has about 225GB of space and is actually composed of 3 separate old drives using LVM. It works great.... but there's a few problems, spacing is starting to run out more scarily if any 1 single drive fails, it's potentially all lost.

So what better time than use my Christmas break to convert my old 3 drive LVM setup into a RAID box.

First thing I had to decide on was hardware or software RAID... and after lots of reading online and putting much thought into it, I decided to go software RAID. The primary reason for this decision was that if a hardware RAID controller dies in a few years, I there could be potential for me be totally screwed if I cannot find a similar or exact RAID controller.

Now the next thing I had to do was decide on the size of the drives. Of course I want to make this as big as possible and it seems like Seagate 320GB Barracuda are priced quite right for me. At about $110 each, I could buy 3 for a RAID5 setup and spend about $375 for a 640GB RAID5 system... (I spent about that much for my original 120MB hardrive in 92 or so).

So next step, I backed up my 120GB or so worth of data on my current file server onto my PC workstation. This took about 10 hours over 100Mbps ethernet. Also did an "apt-get install mdadm" to get the latest RAID utility on my machine.

Next, I yanked out my 3 old drives, and replaced them with the shiny new 320GB drives. It so tempting to RAID0 them together to get close to a TB of usable space but that would make it no better than my old setup in terms of redundancy.

After some dust blowing and closing the case, it was time to boot up my box again.... ran fdisk, partitioned each of the drives with filesystem type "fd" to mark them as raid, you could leave them as the default type "83" as well but I think there's potential for someone to format a drive individually and blow up your RAID if you did that. This was unlikely to happen as it's my own machine but let's do it right.

Now time to build the drive:

mdadm --create --verbose /dev/md0 --chunk=128 --level=5 --raid-devices=3 /dev/hdb1 /dev/hdc1 /dev/hdd1

and then:

mdadm --detail --scan /dev/md0

and save the contents to /etc/mdadm/mdadm.conf so that next time mdadm will see the drives upon bootup. The array immediately started to build in the background.... this i found out eventually will take 2 days... but the cool thing is that you can start using it immediately.

So, now it's time to format my new 640GB /dev/md0

It seems that you have to becareful how you choose to format your RAID partitions, at least when using ext3... didn't look into other filesystems.

mkfs.ext3 -b 4096 -R stride=32 /dev/md0

the stride parameter needs to be your raid chunk size/block size.. which in my case I chose a chunk size of 128/4=32 - Apparently if you don't you run into some horrible performance issues. /Shrug.

This took less than hour to complete and of course then I started to copying my old files back over, which took another 15 hours, probably because the raid was still building and everything was kind of slow while this was happening.

Anyway, the end result when I run:

mdadm --detail /dev/md

/dev/md0:
Version : 00.90.03
Creation Time : Mon Dec 25 12:46:24 2006
Raid Level : raid5
Array Size : 624928256 (595.98 GiB 639.93 GB)
Device Size : 312464128 (297.99 GiB 319.96 GB)
Raid Devices : 3
Total Devices : 3
Preferred Minor : 0
Persistence : Superblock is persistent

Update Time : Thu Dec 27 21:48:35 2006
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0

Layout : left-symmetric
Chunk Size : 128K

UUID : 925b55ab:9a16cff7:7eeacd7b:931f8f9f
Events : 0.64744

Number Major Minor RaidDevice State
0 3 65 0 active sync /dev/hdb1
1 22 1 1 active sync /dev/hdc1
2 22 65 2 active sync /dev/hdd1

Now I have 640GB of space that I can play with and hopefully not worry if something goes wrong with a drive.