[Koha-devel] Replace Catmandu indexing code with pure perl and eventually drop Catmandu as a Koha dependency

[David Gustafsson]

Hi all!
I have been working on replacing Catmandu depandant indexing code with a
simpler and faster Koha-specific one using the Search::Elasticsearch
package (which Catmandu uses internally):
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=19893

Some of the benefits would be:

1) Increased indexing performance (about twice as fast, six times as fast
if comparing time spent in update_index()), due to more efficient
json-conversion and fewer Elasticsearch requests.
2) With Catmandu indexing speed decreases as more mappings are added, with
the alternative algorithm indexing is kept more or less constant no matter
how many mappings you add.
3) Neglectable indexing start-up time. Especially noticeable when indexing
a single document. For example we have an issue with the book drop machine,
each return taking a couple of seconds because of the Catmandu start-up
overhead (or when saving biblios in staff client).
4) More transparent code and less complexity compared with Catmandu
(admittedly partly subjective statement) should lead to improved
maintainability and increased stability.
5) No need for new developers to learn the Fix language
6) Closer to the metal so easier to perform even more Koha-specific
optimizations and customizations which might not be feasible with Catmandu
in tthe way

The proposed patch only addresses the indexing logic but the remaining
Catmandu-dependant code (mainly for searching) should be pretty trivial to
replace with Search::Elasticsearch implementation which can be done as a
next step.

Would be wonderful if this could be raised for discussion at the next
developers meeting.

Best regards
David Gustafsson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20180517/d6926282/attachment.html>