On Wed, Dec 06, 2006 at 12:07:13PM +0100, Paul POULAIN wrote:
Joshua, you told me yesterday that the commenting of zebraop (*) in dev_week was not a bug, but due to a problem with _zconn in Koha (that looses the connection we don't know why) + it seems that updates are long on large databases.
Thus, for adding/updating biblios/items, NPL uses update_items script, running once a day (and thus having a zebra DB uptodate only once a day, which can be considered as a shame, even if not blocking)
SAN-OP will go live in january, 3rd, and will do large test (library closed to public, everybody working/testing koha) on dec, 14th, to I thought it could be a good idea to investigate the problem during those tests.
Joshua told me that NPL uses zebra 1.x, and SAN-OP will use zebra 2.0, so maybe it could solve the problem.
Here is my question : how do we know that the update/insert went wrong ? an internal server error ? an error in the log ? nothing, just being unable to find the added/modified biblio/item ? The errors I've seen in the past occur when a zebraop is executed (an add, edit, or delete operation on the zebra db over ZOOM). So just to clarify what I've tested, before NPL went live, I did some simple testing with zebraop running for all circ operations (to update statuses of items) and cataloging (add, edit, delete). Given that circ is so frequent (many times per second), I discovered that Zebra could not keep up (very slow), and so created update_items.pl, which I supposed would run every hour or less to update the item statuses.
After NPL went live, we had some problems with the index crashing frequently. The crash wasn't too much trouble as searching still worked ... just edits/adds/deletes didn't. So my solution was to comment out zebraop alltogether and only use update_items.pl to update the index. Even update_items.pl would fail occassionally, especially if run frequently during the day (once per hour for instance), so now it runs only once per day :/.
Did the problem occur once a week or once every hour ? did they notice it occurs more on heavy load or any particular situation ? It's almost impossible to keep track of when it happens, especially with a 7 branch library that does over half a million circ annually ... so unfortunately I can't say for sure what triggers an index crash. I do know that ID has fixed several bugs in Zebra 1.x based on reports I've sent with log files, etc., and that Zebra hasn't crashed in over two weeks. I plan to try hourly runs of update_item.pl again soon ... I will report my findings when I do.
Hope that helps. Cheers, -- Joshua Ferraro SUPPORT FOR OPEN-SOURCE SOFTWARE President, Technology migration, training, maintenance, support LibLime Featuring Koha Open-Source ILS jmf@liblime.com |Full Demos at http://liblime.com/koha |1(888)KohaILS