[koha-Infos] Fwd: koha/zebra

[Joshua Ferraro]

Hi everyone,

I've been 'slowly' following the koha-infos 'Zebra' dicussion with
my limited French reading skills and I'd like to respond now. It seems
that concerns have been raised in two areas:

1. Zebra will make installation of Koha too complex

2. Zebra development will distract from other 'requested' technology
   needs.

Before I respond to those two concerns let me begin by addresing why
I began looking for another searching alternative. While it's true
that Koha's current RDBMS-based searching works well for small 
collections, it does not do a good job for collections greater than
(say) 20K items. More than just speed is at issue here: even if 
the results were returned fast they do not have many features that
Koha should have (relevence ranking, sorting by any field, stemming,
etc.). These features are not as important for finding items when
the number of items in the collection is small -- but they are
expected for a larger collection. Adding Zebra as a textual
database engine would solve these problems.

Why use a textual database? I'll tell you. There are different types
of databases (for example, factual and bibliographic/textual). RDBMS
deal very well with factual databases and textual database managers
with textual databases.

With a library system we have two rather distinct and different 
databases (really!). One is a database you received from the item
records (even if not MARC), and is a textual database, searchable 
by attribute (title, author, keyword, subject, etc.). It is a
database about books, not of books. It is searchable by keyword or
other book metattribute, and returns 'bidids' (and viceversa).

The other database is a factual database, a database about the
movement and status of items (and borrowers, etc.).
Understanding these differences is critical for understanding why
we've been looking at a new way to manage the textual information.

Why should Koha aim at larger collections? (I know many are worried
about this as well). My answer is simple: if larger libraries begin
using Koha we will have more resources (money) to develop new 
functions and developers working on Koha can survive. It is security
for Koha users (and developers) if a large institution begins using Koha.

Now, the question for us developers is: how can we make Koha appealing
to both small and large libraries? Small libraries have two concerns
I mention above.

To the first: Zebra installation is easy (./configure, make, make install).
It's the configuration that's complex. So our goal with Koha 2.4 is to
ease the configuration of Zebra and document very well how to set it
up. Really, since Zebra deals with MARC records in textual form it's 
actually less complex than Koha's current MARC RDBMS (which is very
complex!). And since Zebra supports CQL, searching on the collection
will be easier too (adding nice syntax that patron's expect these
days).

Second: one major reason for using free software is that each user
has the power to customize the software (or pay a developer to do so).
So if a library wants a feature developed they should sponsor that
development (programmers need to eat :-)). I don't see how Zebra
development will detract from sponsored feature development. In fact,
I would like to see many more programmers added to the Koha project
but that will not happen until more resources are contributed. Koha
development does not work like proprietary software development -- 
instead of one company only working on features for the 'middle of 
the market', Koha features emerge directly from the sponsership that
libraries provide. So what I mean to say is "if you need something fixed,
pay a developer to fix it" :-).

That said, I want to emphasize that I do want Koha to be a viable
solution for all types of libraries -- so whatever route we take to
fix searching _must_ take into account everyone in the community -- small
libraries included!

Sincerely,
-- 
Joshua Ferraro               VENDOR SERVICES FOR OPEN-SOURCE SOFTWARE
President, Technology       migration, training, maintenance, support
LibLime                  Koha ILS, Mambo Intranet, DiscrimiNet Filter
jmf at liblime.com |  Full Demos at http://liblime.com  |  1(888)KohaILS