TCLUG Archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Searching revisted



Any open source solutions to indexing your web-site into MySQL? Right now I
use htdig and it works fine, but it is not scaling very well.

I have a web archive of a mailing list that spans 4 years. The 43,572 raw html
take up 302 Mb of disk. The htdig database is 304 Mb and it takes ~1 hour to
index the site on a PIII 550 with 256Mb of RAM.

And the archive is only growing.

I'd like to move the index database to MySQL, for speed, disk space and the
ability to have multiple webserver hitting the db for accessing the indexed
web pages.

Anything like this?

-- 
Bob Tanner <tanner@real-time.com>       | Phone : (612)943-8700
http://www.real-time.com                | Fax   : (612)943-8500
Key fingerprint =  6C E9 51 4F D5 3E 4C 66 62 A9 10 E5 35 85 39 D9