[Koha-bugs] [Bug 35086] Koha::SearchEngine::Elasticsearch::Indexer->update_index needs to commit in batches

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35086

--- Comment #2 from Nick Clemens <nick at bywatersolutions.com> ---
Created attachment 160278
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=160278&action=edit
Bug 35086: Add chunk_size option to elasticsearch configuration

Whne performing batch operations we can send a large numebr of records for
reindexing at once.
Currently this can create requetss that are too large for Elasticsearch to
process. We need
to break these requests into chunks/

This patch adds a chunk_size configuration to the elasticsearch stanza in
koha-conf.xml

If blank we default to 5000.

To test:
0 - Have Koha using Elasticsearch
1 - Create and download a report of all barcodes:
    SELECT barcode FROM items
2 - Batch modify these items
3 - Note a single ESindexing job is created
4 - Create and download a report of all authority ids:
    SELECT auth_header.authid FROM auth_header
5 - Setup a marc modification template, and batch modify all the authorities
6 - Again note a single ES backgorund job is created
7 - Apply patch
8 - Repeat the modifications above - you still get a single job
9 - Edit koha-conf.xml and add <chunk_size>250</chunk_size> to elasticsearch
stanza
10 - Repeat modifications - you now get several background ES jobs
11 - prove -v t/db_dependent/Koha/SearchEngine/Elasticsearch/Indexer.t

-- 
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.