[Koha-zebra] Updating zebradb

[Sebastian Hammer]

Tümer,

Is there some sequence of updates/commits that will reliably cause the 
crash that you've seen?

If we could come up with a Perl script, say, which performed a certain 
sequence of updates and commits and caused the problem to reappear, it 
would be a lot easier to trace. Zebra should take care of locking 
between commits and updates, but there might be a condition that has 
been missed which leads to the corrupted database. Whatever it is, it 
sounds like a bug and it's something we're keen to have fixed.

I strongly suspect that the challenge here is that you have multiple 
processes (i.e. circulation desks) doing concurrent updates..  many 
applications I have run into would have updates funnelled through one 
queue...  but we're pretty keen to isolate this problem.

--Seb

Tümer Garip wrote:

>Hi,
>
>This is the continuation of building zebradb thread.
>
>As suggested by Sebastian we moved to zebra 1.4 but our problems have
>not gone away.
>Version 1.4 not being a release version we moved back to 1.3.34 to do
>further tests.
>
>Updating zebradb with just one process is not a problem. i.e a script
>that reads 2000 random records modifies a flag and updates the zebradb
>works OK. If you do not have shadow files search respone slows down
>drastically but if you update with shadow files and do a single commit
>everything looks fine.
>
>But that is not a real life situation and it real life everything is
>still hectic.
>In real life you have about 20 searches going on and about 10-20 update
>requests in queue. In those situations zebra can not handle things. If
>you use shadow files we had a crashed db. I dont know the mechanics
>behind zebra but when you have:
>
>Update
> |
> |
> |		Update
> |		 |
>Commit	 |
>		 |
>		 |
>		commit
>
>Processes like this going on is it safe? Is each commit knows what to
>commit? I can not comment on this but just can say that it does not
>work.
>
>So we changed file handling not to use shadow files. Now we get lots of
>TIMEOUT errors from updates and then zebraserver crashes. Well at least
>the zebradb stayed intact. 
>
>We tried all sorts of other things (like looping the process until it
>updates) but all in vain.
>
>I can definitely say that we have not managed to get zebra updates
>coming from multiple clients work whatever we tried. Either ZEBRA does
>not like this sort of updating or we have a long way of debugging. 
>
>A little bit disappointed but not lost hope. We'll keep on trying and
>reporting.
>
>Regards
>Tumer
>NEU Grand Library
>CYPRUS 
>
>
>
>_______________________________________________
>Koha-zebra mailing list
>Koha-zebra at nongnu.org
>http://lists.nongnu.org/mailman/listinfo/koha-zebra
>
>  
>

-- 
Sebastian Hammer, Index Data
quinn at indexdata.com   www.indexdata.com
Ph: (603) 209-6853