[Koha-devel] Search Engine Changes : let's get some solr

[LAURENT Henri-Damien]

Le 11/11/2010 22:58, Robin Sheat a écrit :
> MJ Ray schreef op do 11-11-2010 om 13:09 [+0000]:
>>>    4. during index process of a file, if you have a problem in your
>>> data, zebraidx just fails silently…
>> Example?
> 
> When Zebra has a record that it chokes on, it will sometimes segfault.
> This is a problem in itself, a second part of that problem (and this is
> one we can fix) is that rebuild_zebra doesn't notice this. It should
> really start screaming about it.
> 
> When zebra fails (segfaulting, or just not liking some data) it does
> things like refuse to process anything else from that point on, and it
> can be quite a time consuming process to track down exactly what record
> it is that's causing the issue.
> 
>>> And this is NOT secure.
>> What security data does zebra leak in this failure case? 
> 
> There's more than one definition of the word security. It's not secure
> in the same sense that a wheel on a car might not be secure. When it
> comes loose and goes flying, things don't work so well. (Although in the
> case of Zebra, you may not notice for a while.)
> 
Well, librarians consider the search engine to be more compared to the
reactor of a plane rather than the wheel of a car.
But I hope that MJ got the idea.
In fact, the only way to get some information from zebrasrv is to catch
the warn " previous transaction doesn't reach commit" in the logs from
zebrasrv...
And if you are indexing 10000 records, and you have one record which
cause that error, your whole bunch of records is not indexed for want of
maybe only one record (maybe more maybe less, who knows ? zebra stops at
the very first without telling you which it is and without indexing the
part that worked.)....
So sometimes, in zebraqueue, things are marked as indexed... while they
are not.... You may think it is ok for a good while and realize it is not.
You are on a plane up to 10000 feet in the sky, you never know when your
engine will blow out or stop. No warnings (from the crontab), no way to
know that it is working.
If it stops, hold on to your stick.

-- 
Henri-Damien LAURENT