Joshua, I saw that you and Chris had been kicking this around on IRC, and it seemed to me then that it was a great idea. Now that I've seen it run (on a database that I know), I am really impressed. I think you've found something that would really be good for Koha. Stephen Joshua Ferraro said:
Hi everyone,
I've spent the last couple of days playing around with Indexdata's Zebra (http://www.indexdata.dk/zebra/). Zebra is another example of a textual database engine.
Zebra can natively index MARC records, so I started by dumping out NPL's records (which took about 20 minutes).
Indexing 150K records took just under 3 minutes and all the searches I've done using Zebra's integrated Z39.50 Server have taken less than a second. I haven't had a chance to write a wrapper yet to integrate the search into our existing code but you can try it out at the LOC's generic Z39.50 gateway:
This link will allow you to search NPL's 150K database (you could also use any Z39.50 client (like Yaz) to do the search. The server's IP is 66.213.78.76:9999 and the db name is VOYAGER (because that's the only way I could get the LOC's gateway to connect).
Zebra is very high performance (it's in use for systems with tens of millions of records). It supports incremental database updates on live systems, exact boolean search expressions, relevance-ranked free-text queries (not sure about stemming).
Of the searching ideas we've come up with so far this one is by far the most impressive. There are still some other textual database engines out there (lucene, clucene, xapian, namzu, to name a few), so if anyone would like to try these out and commit some sample indexing/searching scripts I'd be happy to try them out.
Coments, questions? -- Joshua Ferraro VENDOR SERVICES FOR OPEN-SOURCE SOFTWARE President, Technology migration, training, maintenance, support LibLime Koha ILS, Mambo Intranet, DiscrimiNet Filter jmf@liblime.com | Full Demos at http://liblime.com | 1(888)KohaILS
-- Stephen Hedges Skemotah Solutions, USA www.skemotah.com -- shedges@skemotah.com