Do you prefer to use?
Submitted by Steve on Sat 23 Dec 2006
| Hardware RAID |
   45% | 422 votes |
| Software RAID |
28% | 270 votes |
| Optimism? |
26% | 245 votes |
| Total 937 votes |
Hardware RAID would be my first choice, as long as it isn't that _nasty_ "Host-based" RAID that you've probabaly seen on some lower end motherboards in the last while -- that really is a crock of #@$^ and just a horrible marketing gimmick to get people to pay hardware-RAID prices for thinly vield software-RAID, and cheap, nasty, inflexible software RAID too...
Rant over.
Software RAID comes next, and is still perfectly nice enough where money is tighter.
Either way, I'd still like to stick LVM over either of them -- not only for the flexible nature of it, but also because I think that the more I look into snapshots, the more I like them ;-)
Cheers, and a Merry Christmas to all!
Rant over.
Software RAID comes next, and is still perfectly nice enough where money is tighter.
Either way, I'd still like to stick LVM over either of them -- not only for the flexible nature of it, but also because I think that the more I look into snapshots, the more I like them ;-)
Cheers, and a Merry Christmas to all!
[ Parent ]
Posted by dkg (66.32.xx.xx) on Tue 26 Dec 2006 at 05:16
[ Send Message | View dkg's Scratchpad | View Weblogs ]
[ Send Message | View dkg's Scratchpad | View Weblogs ]
LVM and RAID shouldn't be seen as an either/or choice. They work very nicely in conjunction with each other.
My preferred configuration at the moment is a collection of disks strung together with mdadm (SW RAID on GNU/Linux), and then the resulting block device (e.g. /dev/md0) is used as a physical volume for lvm, which can then be carved up arbitrarily (and used for snapshotting, etc).
BTW, make sure you are using etch-era LVM (>= 2.02) if you are trying to do snapshotting, especially with more recent kernels. There are bad problems with older LVM and more recent kernels.
[ Parent ]
Posted by Anonymous (213.224.xx.xx) on Mon 25 Dec 2006 at 19:57
I have read about difficulties finding replacements for old RAID controllers, so I choose software raid as that should be easier to maintain/fix in the long run. No idea if my reasoning is correct...
[ Parent ]
Posted by dkg (66.32.xx.xx) on Tue 26 Dec 2006 at 05:09
[ Send Message | View dkg's Scratchpad | View Weblogs ]
[ Send Message | View dkg's Scratchpad | View Weblogs ]
software RAID over hardware RAID for me, for several reasons:
- closed firmware sucks
- Who writes the firmware for your HW RAID controller? Who audits it? Who even can audit it? Who fixes it when it needs an update? Do you really trust it? There are probably flaws in the free SW RAID stacks, but they can be found and fixed with a lot less hassle or sweat. For older hardware controllers, you might not even be able to get updates.
- wider userbase
- I know there are a lot of users for certain HW RAID controllers. But i doubt those numbers even come close to the userbase of the Linux or *BSD SW RAID stacks. Wider userbase means more likely peer-based support. If anyone has numbers on this, i'd be interested in reading them.
- no hardware lock-in
- On a budget? Or do you need to mix block devices to make your array the size you need? Did your disk controller crap out and you need a new one? With SW RAID, you can use whatever's available at the store (or in yer loose parts bin, in a pinch). With HW RAID, you're pretty much stuck buying from the same vendor, or doing a massive, messy migration strategy. software RAID is more flexible.
- standard monitoring/repair tools
- With HW RAID, you'll need to use hardware-specific monitoring tools. Many of these are decent, but they're often developed specifically for the hardware you've got, and don't have much outside testing or auditing. If you can get away with using standard disk monitoring tools (e.g. smartmontools), you can get familiar with them and use them widely, even on a range of different hardware.
It sucks to spend a day or two really learning IBM's ServeRAID monitoring tools, and figuring out how to hook them into a larger monitoring framework, and then turn around and be faced with compaq's disk array tools, which do similar (but not exactly the same) tasks on yer next machine. This increased administration hassle is a real cost for HW RAID.
You also really don't want to have to re-familiarize yourself with some obscure tool when you're in crisis mode. If you use the same monitoring tools on nearly all your machines, you'll be more proficient when you really need to be.
[ Parent ]
"About the only thing i think HW RAID has to offer over SW RAID at this point is speed, and modern disks and controllers are already pretty fast."
This is the key point.
You use hardware RAID to get non-volatile storage, which means you can do a committed write at a sensible speed.
Modern disks are still disks, and still spin, which means commit to disk time is some multiple of the of the physical rotation period (15000/minute on expensive modern disks, or 0.004 seconds). Typically even on modern fast disks, with modern journaled file systems, you are still talking small numbers of blocking writes to disk per second (~10's, at a push two rotations to commit (very optimistic), is 100 commits per second, which is the absolute maximum anyone is likely to get). Although I think the OSes try and do multiple commits in the same rotations using barriers or some such.
If you use software RAID you either have low performance for commit to disk, or you risk data loss by using the disks with their write caches enabled.
Commit to disk in my experience is the limiting factor on most databases, and most correctly configured email servers, and pretty much everything else I've touched recent, because all other performance limits are so stupendously high on modern hardware.
My email servers disk drives can (allegedly!) do 70MB/s throughput each, or (without the hardware RAID) deliver a dozen or so emails a second, which represents some pitiful fraction of their theoretical I/O potential (~1/5000th). With hardware RAID I can push over 500 commits per second to a RAID5 volume (sustained) which means more email delivered per second, onto less disks, which means more customers per box.
I love to use hardware RAID (so I voted hardware).
However for most cheap servers running Linux, the other comments made here predominate, and I really can't recommend it. My advice to those pondering it would be consider buying dedicated external hardware RAID that "just works", that generally isn't cheap, but it is often worth it.
Can one safely enable write caches in disk drives if one has a UPS device in use? How much riskier is this, than say a dedicated hardware RAID device. I'm guessing a lot of ISPs do this on their email, and database servers, and reality is most people aren't that concerned if one email is lost in transit (or delivered twice). Although if it takes out an enterprise database, that then demands some complex recovery procedure to convince it that it is restored to a consistent state before people can get back to work, one might be a tad more careful about things.
There was an interesting discussion on the LKML many years back with Alan Cox on these issues. I guess it raises the question, why don't disk manufacturers build in a guaranteed power capacitance to write the data in a disk drives cache, if power is lost. i.e. We should design the components individually to do the sensible thing, and then we shouldn't come unstuck when we assemble them into complicated assemblies. But I guess that would require them have capacitance which exceeds the capacity of the device sending write requests, or some sort of power supply indicator, so they could refuse new requests when the power supply is interrupted, to give themselves time to commit that which is in cache to the platters before the power runs out.
The whole argument will be obsoleted by flash style storage, which lacks the spinning latency, and so is a lot faster for these kind of transactions.
This is the key point.
You use hardware RAID to get non-volatile storage, which means you can do a committed write at a sensible speed.
Modern disks are still disks, and still spin, which means commit to disk time is some multiple of the of the physical rotation period (15000/minute on expensive modern disks, or 0.004 seconds). Typically even on modern fast disks, with modern journaled file systems, you are still talking small numbers of blocking writes to disk per second (~10's, at a push two rotations to commit (very optimistic), is 100 commits per second, which is the absolute maximum anyone is likely to get). Although I think the OSes try and do multiple commits in the same rotations using barriers or some such.
If you use software RAID you either have low performance for commit to disk, or you risk data loss by using the disks with their write caches enabled.
Commit to disk in my experience is the limiting factor on most databases, and most correctly configured email servers, and pretty much everything else I've touched recent, because all other performance limits are so stupendously high on modern hardware.
My email servers disk drives can (allegedly!) do 70MB/s throughput each, or (without the hardware RAID) deliver a dozen or so emails a second, which represents some pitiful fraction of their theoretical I/O potential (~1/5000th). With hardware RAID I can push over 500 commits per second to a RAID5 volume (sustained) which means more email delivered per second, onto less disks, which means more customers per box.
I love to use hardware RAID (so I voted hardware).
However for most cheap servers running Linux, the other comments made here predominate, and I really can't recommend it. My advice to those pondering it would be consider buying dedicated external hardware RAID that "just works", that generally isn't cheap, but it is often worth it.
Can one safely enable write caches in disk drives if one has a UPS device in use? How much riskier is this, than say a dedicated hardware RAID device. I'm guessing a lot of ISPs do this on their email, and database servers, and reality is most people aren't that concerned if one email is lost in transit (or delivered twice). Although if it takes out an enterprise database, that then demands some complex recovery procedure to convince it that it is restored to a consistent state before people can get back to work, one might be a tad more careful about things.
There was an interesting discussion on the LKML many years back with Alan Cox on these issues. I guess it raises the question, why don't disk manufacturers build in a guaranteed power capacitance to write the data in a disk drives cache, if power is lost. i.e. We should design the components individually to do the sensible thing, and then we shouldn't come unstuck when we assemble them into complicated assemblies. But I guess that would require them have capacitance which exceeds the capacity of the device sending write requests, or some sort of power supply indicator, so they could refuse new requests when the power supply is interrupted, to give themselves time to commit that which is in cache to the platters before the power runs out.
The whole argument will be obsoleted by flash style storage, which lacks the spinning latency, and so is a lot faster for these kind of transactions.
[ Parent ]
Nice Comment!
As for the idea of additional capacitance on HW RAID Controllers, there are many nowadays that come-with/have-available battery-backup-units.
I know that some/most 3Ware controllers have BBU's, and this allows for the retention of data in the write-cache that can be flushed to disk when the power comes back up.
Is that what you had in mind, or were you thinking of something extra?
Cheers.
As for the idea of additional capacitance on HW RAID Controllers, there are many nowadays that come-with/have-available battery-backup-units.
I know that some/most 3Ware controllers have BBU's, and this allows for the retention of data in the write-cache that can be flushed to disk when the power comes back up.
Is that what you had in mind, or were you thinking of something extra?
Cheers.
[ Parent ]
I was thinking in disk drives, not in hardware raid controllers.
The point is most disks have a write cache, but you can't safely use it, and assume data committed to that cache is safely preserved in the event of a power failure.
The battery backup, and similar technologies in hardware RAID devices is precisely what makes them attractive. But it should be hard to put that in every disk drive if manufacturers determined it was something they could sell.
The point is most disks have a write cache, but you can't safely use it, and assume data committed to that cache is safely preserved in the event of a power failure.
The battery backup, and similar technologies in hardware RAID devices is precisely what makes them attractive. But it should be hard to put that in every disk drive if manufacturers determined it was something they could sell.
[ Parent ]
After using Hardware RAID at work for a long time, I ended to prefer Software raid.
I found it more reliable, more flexible and easier to use.
Hardware RAID is often a pita. (NEVER ! use Promise raid... never)
I found it more reliable, more flexible and easier to use.
Hardware RAID is often a pita. (NEVER ! use Promise raid... never)
[ Parent ]
Whatever the merits or otherwise of the hardware v software choice, I think not using RAID on anything with user data on is just silly.
Currently 40GB of IDE disk space will set you back about a tenth the price of a very cheap PC. I have two desktop PCs, and a laptop at home, one has RAID, the other mounts "/home" from the one with RAID, the laptop similarly picks up data from the RAID storage. Every server at work has RAID1 software mirrored disk unless it has something fancier, or is just one of several identical machines in a cluster (so redundant by other means).
Sure there are other ways to get data redundancy, but everyone should be doing something by now. The Sarge installer was the first Debian one to make the RAID install trivial.
My bigger pain is I need to get the tape drive working (again) with recent kernel versions.
As Bill Hassell at HP signature puts it.....
"There are two types of computer users in the world...those that have lost data, and those that are going to." (blh, circa 1972)
Currently 40GB of IDE disk space will set you back about a tenth the price of a very cheap PC. I have two desktop PCs, and a laptop at home, one has RAID, the other mounts "/home" from the one with RAID, the laptop similarly picks up data from the RAID storage. Every server at work has RAID1 software mirrored disk unless it has something fancier, or is just one of several identical machines in a cluster (so redundant by other means).
Sure there are other ways to get data redundancy, but everyone should be doing something by now. The Sarge installer was the first Debian one to make the RAID install trivial.
My bigger pain is I need to get the tape drive working (again) with recent kernel versions.
As Bill Hassell at HP signature puts it.....
"There are two types of computer users in the world...those that have lost data, and those that are going to." (blh, circa 1972)
[ Parent ]
Posted by Anonymous (84.133.xx.xx) on Fri 29 Dec 2006 at 06:03
I voted "optimism" but in fact I use un-RAID-ed drives mirrored regularily via rsync on neighbour systems with "decrementals" (keeping old file automatism of rsync) and am pretty happy with that strategy.
[ Parent ]
Posted by Anonymous (194.176.xx.xx) on Fri 29 Dec 2006 at 16:04
Optimism?
Steve, you're not implying that RAID is backup, are you???
Steve, you're not implying that RAID is backup, are you???
[ Parent ]
Posted by Steve (62.30.xx.xx) on Sat 30 Dec 2006 at 10:08
[ Send Message | View Steve's Scratchpad | View Weblogs ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
[ Parent ]
At work we use HP/Compaq kit which has RAID1 "mirroring" built in. The Linux boxes just see one logical-drive and as long as the kernel knows what to do (2.6 kernels can't on some of the older units...!) it just works.
The fancy IBM System P AIX kit uses a small IBM disk array, that's mostly running hardware RAID5 and other RAID combinations. The resultant disks-arrays are then presented to the hardware as single logical disks.
At home I'm not using RAID, just regular backups. When I get a new home server, I'll probably install software RAID+LVM.
--
"It's Not Magic, It's Work"
Adam
The fancy IBM System P AIX kit uses a small IBM disk array, that's mostly running hardware RAID5 and other RAID combinations. The resultant disks-arrays are then presented to the hardware as single logical disks.
At home I'm not using RAID, just regular backups. When I get a new home server, I'll probably install software RAID+LVM.
--
"It's Not Magic, It's Work"
Adam
[ Parent ]