On Wed, May 11, 2005 at 05:09:18PM -0700, Joshua Ferraro wrote:
Koha's SQL is pretty good at handling the database _about_ the books, though I think we could optimize it to work even faster -- but as far as handling the database of bibliographic records, SQL is quite limited in terms of scalability and flexability. Lucene or other textual database engines have lots of nice feature built in, things like boolian searching, relevance ranking, full-text searching, etc. Sorry to reply to my own post but I thought I should clarify something.
I don't mean to say that we should scrap Koha's MARC tables, etc. What I am trying to say is that for search purposes, a textual database does a much better job than SQL does. Based on what I've learned so far, a textual database would allow us to create extremely useful indexes of marc records (indexes that we might even be able to create directly from the database -- I'm not sure if it would require an export to a markup language first). These indexes would allow us to perform very powerful query operations on our records that would have far too much overhead in SQL (the tables get too big too fast, and boolian operations, relevance ranking, spellchecking, etc. are pretty complex in SQL). Also, textual indexes are FAST! Additionally, adding new record formats to search in the catalog (like EAD) would be much simpler if our search API were independent of one kind of record format (MARC). -- Joshua Ferraro VENDOR SERVICES FOR OPEN-SOURCE SOFTWARE CEO, LIBLIME migration, training, maintenance, support jmf@liblime.com Koha ILS, Mambo Intranet, DiscrimiNet Filter 1(740)707-7654 TRY OUR FULL DEMOS AT http://liblime.com