[Koha-devel] zebraop commented [for joshua 1st]

[Joshua Ferraro]

On Wed, Dec 06, 2006 at 12:07:13PM +0100, Paul POULAIN wrote:
> Joshua, you told me yesterday that the commenting of zebraop (*) in 
> dev_week was not a bug, but due to a problem with _zconn in Koha (that 
> looses the connection we don't know why) + it seems that updates are 
> long on large databases.
> 
> Thus, for adding/updating biblios/items, NPL uses update_items script, 
> running once a day (and thus having a zebra DB uptodate only once a day, 
> which can be considered as a shame, even if not blocking)
> 
> SAN-OP will go live in january, 3rd, and will do large test (library 
> closed to public, everybody working/testing koha) on dec, 14th, to I 
> thought it could be a good idea to investigate the problem during those 
> tests.
> 
> Joshua told me that NPL uses zebra 1.x, and SAN-OP will use zebra 2.0, 
> so maybe it could solve the problem.
> 
> Here is my question : how do we know that the update/insert went wrong ? 
> an internal server error ? an error in the log ? nothing, just being 
> unable to find the added/modified biblio/item ?
The errors I've seen in the past occur when a zebraop is executed (an
add, edit, or delete operation on the zebra db over ZOOM). So just to
clarify what I've tested, before NPL went live, I did some simple
testing with zebraop running for all circ operations (to update statuses
of items) and cataloging (add, edit, delete). Given that circ is so
frequent (many times per second), I discovered that Zebra could not keep
up (very slow), and so created update_items.pl, which I supposed would run
every hour or less to update the item statuses.

After NPL went live, we had some problems with the index crashing
frequently. The crash wasn't too much trouble as searching still worked
... just edits/adds/deletes didn't. So my solution was to comment out
zebraop alltogether and only use update_items.pl to update the index.

Even update_items.pl would fail occassionally, especially if run
frequently during the day (once per hour for instance), so now it runs
only once per day :/.

> Did the problem occur once a week or once every hour ? did they notice 
> it occurs more on heavy load or any particular situation ?
It's almost impossible to keep track of when it happens, especially with
a 7 branch library that does over half a million circ annually ... so
unfortunately I can't say for sure what triggers an index crash. I do
know that ID has fixed several bugs in Zebra 1.x based on reports I've
sent with log files, etc., and that Zebra hasn't crashed in over two
weeks. I plan to try hourly runs of update_item.pl again soon ... I will
report my findings when I do.

Hope that helps.

Cheers,

-- 
Joshua Ferraro                       SUPPORT FOR OPEN-SOURCE SOFTWARE
President, Technology       migration, training, maintenance, support
LibLime                                Featuring Koha Open-Source ILS
jmf at liblime.com |Full Demos at http://liblime.com/koha |1(888)KohaILS