[Koha-devel] Replace Catmandu indexing code with pure perl and eventually drop Catmandu as a Koha dependency

[David Gustafsson]

“the book drop machine” = machine where you return books. It does not
matter if using Plack or not, Catmandu has some kind of compile step or
similar that has a startup time of a couple of seconds. So everytime one
returned a book (and the biblio was updated and indexed) there was a delay
of a couple of seconds, if returning multiple books is a major issue.

Best Regards
David

2018-05-18 2:33 GMT+02:00 David Cook <dcook at prosentient.com.au>:

> I don’t do anything with Elastic or Catmandu at the moment, so I won’t
> comment about that.
>
>
>
> But you mention the overhead of Catmandu start-up. Can you speak more to
> that? What’s “the book drop machine”? Why isn’t Catmandu running in a
> persistent process?*
>
> *I say as someone who still uses Koha using CGI rather than Plack…
>
>
>
> David Cook
>
> Systems Librarian
>
> Prosentient Systems
>
> 72/330 Wattle St
> <https://maps.google.com/?q=72/330+Wattle+St+Ultimo,+NSW&entry=gmail&source=g>
>
> Ultimo, NSW
> <https://maps.google.com/?q=72/330+Wattle+St+Ultimo,+NSW&entry=gmail&source=g>
> 2007
>
> Australia
>
>
>
> Office: 02 9212 0899
>
> Direct: 02 8005 0595
>
>
>
> *From:* koha-devel-bounces at lists.koha-community.org [mailto:
> koha-devel-bounces at lists.koha-community.org] *On Behalf Of *David
> Gustafsson
> *Sent:* Thursday, 17 May 2018 11:57 PM
> *To:* Koha-devel at lists.koha-community.org
> *Subject:* [Koha-devel] Replace Catmandu indexing code with pure perl and
> eventually drop Catmandu as a Koha dependency
>
>
>
> Hi all!
>
> I have been working on replacing Catmandu depandant indexing code with a
> simpler and faster Koha-specific one using the Search::Elasticsearch
> package (which Catmandu uses internally): https://bugs.koha-community.
> org/bugzilla3/show_bug.cgi?id=19893
>
>
>
> Some of the benefits would be:
>
>
>
> 1) Increased indexing performance (about twice as fast, six times as fast
> if comparing time spent in update_index()), due to more efficient
> json-conversion and fewer Elasticsearch requests.
>
> 2) With Catmandu indexing speed decreases as more mappings are added, with
> the alternative algorithm indexing is kept more or less constant no matter
> how many mappings you add.
>
> 3) Neglectable indexing start-up time. Especially noticeable when indexing
> a single document. For example we have an issue with the book drop machine,
> each return taking a couple of seconds because of the Catmandu start-up
> overhead (or when saving biblios in staff client).
>
> 4) More transparent code and less complexity compared with Catmandu
> (admittedly partly subjective statement) should lead to improved
> maintainability and increased stability.
>
> 5) No need for new developers to learn the Fix language
>
> 6) Closer to the metal so easier to perform even more Koha-specific
> optimizations and customizations which might not be feasible with Catmandu
> in tthe way
>
>
>
> The proposed patch only addresses the indexing logic but the remaining
> Catmandu-dependant code (mainly for searching) should be pretty trivial to
> replace with Search::Elasticsearch implementation which can be done as a
> next step.
>
>
>
> Would be wonderful if this could be raised for discussion at the next
> developers meeting.
>
>
>
> Best regards
>
> David Gustafsson
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20180518/f964d541/attachment.html>