Re: Disk I/O (fwd)

[Christopher Reid Palmer <reid@pconline.com>]

More fun.

--
Christopher Reid Palmer : www.innerfireworks.com

---------- Forwarded message ----------
Date: Sat, 26 Feb 2000 11:50:15 -0600
From: Randy Maas <randym@acm.org>
To: Christopher Reid Palmer <reid@pconline.com>
Subject: Re: Disk I/O

Forward at will.

At 08:16 PM 2/25/00 -0600, Chris Palmer wrote:
>Block seems to be pretty good, everything else sucks. I'm sure I could be
>getting more out of that disk if I had a wide SCSI adaptor; sticking it on
>a narrow bus is sadness. I'd have to reboot to check and be sure, but I
>think it's set for asynchronous transfers on that SCSI ID.

Its more than just that.  PCI is a shared bus; it is pretty inconceivable
you will ever get to the 36MB/s (theoretically) possible on that bus.
Inorder to do so, you would need to eliminate /all/ contention (interrupts)
and get an adapter that can do long scatter/gather lists in PCI burst mode.
 And you need to make sure the adapter is high priority on the bus.  And
there is a minimal number of PCI bridges between it and the primary store
(DRAM).  

Do you have an AGP motherboard?  Is the AGP on a PCI brdige, or a separate
channel?  If it is effectively on the PCI bus, it could screw peak transfer
rate.

>Netscape (thanks for the link Carl!) and checking my email. It's useless
>to measure the performance of a machine sitting still, because machines
>don't sit still.

Its also stupid to measure the performance of a machines subsystem when
you're pre-empting it.  You need to characterize the nodes, their
interrconnects, and then the full system load behaviour.  Otherwise
everything if for naught, and senseless griping.

>What worries me is the huge CPU eating rates. The Commentary doc that
>comes with Bonnie suggest that's a fault of Unix filesystems.

A combination of lazy writes, resource management & allocation overheads,
context switching, event handling (interrupts), etc.  Raw partitions make
benchmarks look pretty.  Thats about it.  How long of scatter/gather lists
can be queued to the adapter?

>Both disks performed in parallel not too much less well than when the 
>benchmark is run on each disk in series. That's the advantage of SCSI.

It also tells you something about what the data rate level is to the disk
/relative/ to the interconnects.  The interconnects -- shared PCI bus,
shared SCSI bus, shared SCSI adapter, etc -- can sustain a much higher than
the target can, apparently.  It also hints as to the prioritization on the
SCSI bus.

(also note: Adapters and disk tends to have /read ahead caches/, but no
write caches.  This will change what appears to be fast and what doesn't.
You can often configure portions of the drivers /read ahead/.  But tuning
for a benchmark will hurt system usability.  Chris, I wrote a very long set
of articles about this a year ago; you should re-read those if you want a
way of benchmarking your system for tuning it.)

>Note the weirdness in sequential block inputs for /dev/sdb when run in
>series compared to parallel--it's over *twice as fast* when run in
>parallel! This must be some kind of anomaly.

Why? Prioritization to a specific disk by the driver, adapter, or scsi bus
is normal, even if it is undesirable.


>I'd have to run the whole
>series of tests many times and take an average. Perhaps a shell script is
>in my future.

Yes.  You need to and toss out the outliers before taking the average.

Randy