[Koha-devel] (sort of) Refactoring rebuild_zebra

[Tomas Cohen Arazi]

On Thu, Aug 26, 2010 at 7:45 AM, MJ Ray <mjr at phonecoop.coop> wrote:
> Tomas Cohen Arazi wrote:
>> On Tue, Aug 24, 2010 at 2:02 PM, MJ Ray <mjr at phonecoop.coop> wrote:
>> > we figure out one size that fits all?  Maybe have one daemon
>> > running and then different ways for a cron job or librarian
>> > interface task to signal (some SIG perhaps?) that it needs to act?
>>
>> I've just send a rewrite of zebraqueue daemon that does run (sort of
>> refactoring of) rebuild_zebra as one would do with cron. I think is
>> the right way to handle this. Avoid using cron, and configure the
>> frequency of the update via systempreferences.
>
> As you know, I've made a few comments on that and having a daemon
> around does open the door to realtime reindexing, but I don't think
> that rewrite does that yet.  I don't think it would be difficult to
> allow the librarian interface to wake it up sooner than the update
> frequency set in systempreferences, would it?

That's exactly what I explained in the IRC. The new zebraqueue-daemon
script is just a workaround (running that script too often seems worse
than triggering it instantly on updates). As I said, this could be
acheived directly from the librarian interface and that's why I'm
proposing a library for rebuild_zebra tasks. My main concern is about
atomicity and possible race conditions. We need to atomice:

1) Read zebraqueue table and make a list of records that need updating.
2) Generate the files for that records that will be passed to zebraidx
3) Run zebraidx on those files
4) Mark those records in zebraqueue as done.

We need to avoid spawning a new update until this is finished,
concurrently being able to add new rows to zebraqueue table for
concurrent cataloguing.

> This has been discussed over on the koha list and one case where
> realtime reindexing would really help is when authorities are being
> added at cataloguing-time.  At the moment, some librarians are
> changing their workflow to suit Koha, which seems backwards to me.

I agree with that as my librarians express that on an hourly basis :-D

To+