[Koha-bugs] [Bug 21872] Elasticsearch indexing faster by making it multi-threaded

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21872

--- Comment #27 from David Cook <dcook at prosentient.com.au> ---
(In reply to Ere Maijala from comment #25)
> For good indexing performance you need to send records to Elasticsearch in
> batches. The current default is to collect 5000 records and then commit the
> batch to ES. If we have a lot of workers that only process one record at a
> time, we also need IPC to collect the records in the main process to be able
> to update in batches.
> 

It's fairly trivial to have workers process batches rather than single records,
and IPC really isn't that hard either. 

> All that's of course possible, but I'm not sure there's any real benefit
> from the way more complex mechanism compared to the slice version.

I'm just providing an alternative suggestion. You're the one doing the real
work, so if you want to go with the slice version, then that sounds good to me.

-- 
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.