[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: ZFS Advice
- To: Matt Simerson <matt_(_at_)_corp_(_dot_)_spry_(_dot_)_com>
- Subject: Re: ZFS Advice
- From: Jeremy Chadwick <koitsu_(_at_)_FreeBSD_(_dot_)_org>
- Date: Wed, 6 Aug 2008 02:22:26 -0700
- Cc: freebsd-fs_(_at_)_freebsd_(_dot_)_org
On Wed, Aug 06, 2008 at 12:32:43AM -0700, Matt Simerson wrote:
> Then I tested ZFS with RAIDZ in various configs (raidz, raidz2, 4,6, and
> 8 disk arrays) on FreeBSD. When using raidz and FreeBSD, the difference
> in performance of the controllers is much smaller. It's bad with the
> Areca controller and worse with the Marvell. My overall impression is
> that ZFS performance under FreeBSD is poor.
Performance has been significantly improved with a patch from pjd@
provided about a week ago. I have not tested it personally, but there
have been a couple reports so far that the performance improvement is
significant. I recommend you read the *entire thread*, and yes, it is
very long. Subject is "ZFS patches".
It continues into August, with some people using mail clients that don't
properly utilise mail reference IDs, so their replies are scattered.
Again, look for "ZFS patches".
> I say this because I also tested one of the systems with OpenSolaris on
> the Marvell card (OpenSolaris doesn't support the Areca). Read
> performance with ZFS and RAIDZ on Solaris was not just 2-3 but 10-12x
> faster on Solaris. OpenSolaris write performance was about 50% faster
> than FreeBSD on the Areca controller and 100% faster than FreeBSD on the
> The only way I could get decent performance out of FreeBSD and ZFS was
> to use the Areca as a RAID controller and then ZFS stripe the data
> across the two RAID arrays. I haven't tried it but I'm willing to bet
> that if I used UFS and geom_stripe to do the same thing, I'd get better
> performance with UFS. If you are looking for performance, then raidz and
> ZFS is not where you want to be looking.
Do you have any actual numbers showing performance differential? If so,
how did you obtain them under FreeBSD? Also, what tuning parameters did
you use on FreeBSD (specifically kernel/loader.conf stuff)?
It seems that using "zpool iostat" is only useful if one wishes to see
the amount of I/O that ends up hitting the physical disks in the pool;
if the data is cached in memory, "zpool iostat" won't show any I/O.
I can get performance data from gstat(8), but that won't tell me how ZFS
itself is actually performing, only at what rate the kernel read/write
data from the physical disks.
On my system (a 3-disk raidz spool, using WDC WD5000AAKS disks, SATA300,
on an Intel ICH7 controller), I can get about 70MB/sec from each disk
when reading, and somewhere around 55-60MB/sec when writing. But again,
this is for actual disk I/O and isn't testing ZFS performance.
> As far as workload with prefetch: under my workloads (heavy network &
> file system I/O) prefetch=almost instant crash and burn. As soon as I
> put any heavy load on it, it hangs (as I've described previously on this
I assume by "hangs" you mean the system becomes unresponsive while disk
reads/writes are being performed, then recovers, then stalls again,
recovers, rinse lather repeat? If so -- yes, that's the exact behaviour
others and myself have reported. Disabling prefetch makes the system
much more usable during heavy I/O.
> 3ware controllers = cheap and you get what you pay for. At my last job
> we had thousands of 3ware cards deployed because they were so
> inexpensive and RAID = RAID, right? Well, they were the controllers
> most likely to result in catastrophic data loss for our clients. Maybe
> it's because the interface is confusing the NOC technicians, maybe it's
> because their recovery tools suck, or because when the controller fails
> it hoses the disks in interesting ways. For various reasons, our luck at
> recovering failed RAID arrays on 3ware cards was poor.
This story is pretty much the norm. Once in a while you'll find someone
praising 3ware controllers, but I often wonder just what kind of
workload and failure testing they've done prior to their praise. A
friend of at Rackable told me horror stories involving 3ware
controllers. I'm thankful 3ware cares about FreeBSD (most vendors do
not), but with a history of firmware/BIOS bugs and the sensitive
nature of their cards, I choose to stay away from them.
I've heard nothing but praise when it comes to Areca controllers.
All that said -- have you actually performed a hard failure with an
Areca controller on FreeBSD (using both UFS and ZFS)? Assuming you have
hot-swap enclosures/carriers, what happens if you yank a disk on the
Areca controller? How does FreeBSD behave in this case?
| Jeremy Chadwick jdc at parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP: 4BD6C0CB |
freebsd-fs_(_at_)_freebsd_(_dot_)_org mailing list
To unsubscribe, send any mail to "freebsd-fs-unsubscribe_(_at_)_freebsd_(_dot_)_org"
Visit your host, monkey.org