[Koha-bugs] [Bug 21872] Elasticsearch indexing faster by making it multi-threaded

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21872

--- Comment #22 from David Cook <dcook at prosentient.com.au> ---
(In reply to David Gustafsson from comment #19)
> Isn't PEO event-loop based and thus runs in a single thread? If so it would
> not help at all in speeding up the indexing process (except for perhaps
> committing to Elasticsearch in parallel since that does not run in perl).

POE does work off an event loop, so it does run in a single process/single
thread, but that's where POE::Wheel::Run becomes relevant. That module forks
child processes and uses pipes for bilateral communication between the parent
and children. The children do the parallel processing and the parent manages
the job/task queue for distributing work to the children. 

It could be used for Elasticsearch or Zebra really. The current rebuild scripts
are written in Perl but the Zebra one is just a wrapper around command line
tools.

-- 
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.