[Koha-devel] koha 2.4 tools & requirements

[Joshua Ferraro]

On Wed, May 11, 2005 at 05:09:18PM -0700, Joshua Ferraro wrote:
> Koha's SQL is pretty good at handling the database _about_ the books,
> though I think we could optimize it to work even faster -- but as
> far as handling the database of bibliographic records, SQL is
> quite limited in terms of scalability and flexability. Lucene or
> other textual database engines have lots of nice feature built in,
> things like boolian searching, relevance ranking, full-text 
> searching, etc.
Sorry to reply to my own post but I thought I should clarify something.

I don't mean to say that we should scrap Koha's MARC tables, etc. What
I am trying to say is that for search purposes, a textual database does
a much better job than SQL does. Based on what I've learned so far, a           textual database would allow us to create extremely useful indexes of
marc records (indexes that we might even be able to create directly
from the database -- I'm not sure if it would require an export to
a markup language first). These indexes would allow us to perform
very powerful query operations on our records that would have far
too much overhead in SQL (the tables get too big too fast, and
boolian operations, relevance ranking, spellchecking, etc. are
pretty complex in SQL). Also, textual indexes are FAST!

Additionally, adding new record formats to search in the catalog
(like EAD) would be much simpler if our search API were independent
of one kind of record format (MARC).

-- 
Joshua Ferraro               VENDOR SERVICES FOR OPEN-SOURCE SOFTWARE
CEO, LIBLIME                migration, training, maintenance, support
jmf at liblime.com          Koha ILS, Mambo Intranet, DiscrimiNet Filter
1(740)707-7654               TRY OUR FULL DEMOS AT http://liblime.com