NT and Linux benchmarking

[Christopher Palmer <reid@pconline.com>]

On Fri, 20 Aug 1999, Clayton T. Fandre wrote:

> But I guess everything can be written to be biased. Does anyone know of
> some accurate comparisons between Linux/NT?

But what constitues an 'accurate comparison'?

Benchmarks are basically worthless (more below). All you can do is try two
systems in your own specific environment, and compare them. Performance is
very closely tied to the task, the people doing the task, the place the
task is done, and other factors. The gear and the software are important,
but no more so than the extra-computational factors.

Among the many factors impinging on the performance of a computer system,
the user's competence is primary -- for that reason alone it's hard to
interpret real-world accounts as being anything more than anecdotal.

Unfortunately, that leads us back to benchmarking. :P


> I'd like to put them up on the web. (And no, Mindcraft is NOT accurate.)

Actually, the Mindcraft benchmarks probably were accurate. But whether
anyone competent would misconfigure their Linux system in the real world
is an open question (as is how many misconfigured NT systems there are out
there). (I'm referring to the Mindcraft thing.)


> Now that I think of it, that would make a great project for the TCLUG.
> Why don't we do our own benchmark comparisons. Anyone willing to lead up
> this effort?

Benchmarks have meaning only in tightly specified contexts. Example:
Computer system running a given application compared to the same system
with more memory running the same application. 'Yep, Apache goes faster if
you have more memory.'

Comparing Linux to NT, even on the same hardware, is impossible -- there
are just too many variables. Apart from the fact that they can't run the
same applications (NT runs an experimental version of Apache only, for
example, iirc), how many of us are exactly as competent with NT as we are
Linux?

What do you hope to show with benchmarks?


A better tactic, I think, is to make highly specified, *qualitative*
judgments. E.g.:

"I can run a Web server on Linux and get uptimes of n days... serving p
pages per hour... requiring h hours of administration per month...
spending d dollars for the software and hardware..."

"I can run x, y and z applications on Linux... I can configure Linux in a
variety of ways (muLinux to Beowulf)... I can use Linux to speed up an old
computer and save money..."

What these hypothetical reports show is that there are reasons to choose
Linux, in specific contexts or to acheive specific goals. Show that Linux
is an option, and how it can help people, but don't try to compare apples
to oranges because the differences are too variable to mean anything much.

That said, I would love to see such things on the Web site, and am willing
to help.


-- Chris

  Christopher Reid Palmer : http://www.innerFireWorks.com/