[Koha-bugs] [Bug 35086] Koha::SearchEngine::Elasticsearch::Indexer->update_index needs to commit in batches
bugzilla-daemon at bugs.koha-community.org
bugzilla-daemon at bugs.koha-community.org
Thu Jan 11 15:38:29 CET 2024
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35086
Jonathan Druart <jonathan.druart+koha at gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Attachment #160560|0 |1
is obsolete| |
Attachment #160561|0 |1
is obsolete| |
Attachment #160562|0 |1
is obsolete| |
Attachment #160572|0 |1
is obsolete| |
--- Comment #18 from Jonathan Druart <jonathan.druart+koha at gmail.com> ---
Created attachment 160853
-->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=160853&action=edit
Bug 35086: Add chunk_size option to elasticsearch configuration
Whne performing batch operations we can send a large numebr of records for
reindexing at once.
Currently this can create requetss that are too large for Elasticsearch to
process. We need
to break these requests into chunks/
This patch adds a chunk_size configuration to the elasticsearch stanza in
koha-conf.xml
If blank we default to 5000.
To test:
0 - Have Koha using Elasticsearch
1 - Create and download a report of all barcodes:
SELECT barcode FROM items
2 - Batch modify these items
3 - Note a single ESindexing job is created
4 - Create and download a report of all authority ids:
SELECT auth_header.authid FROM auth_header
5 - Setup a marc modification template, and batch modify all the authorities
6 - Again note a single ES backgorund job is created
7 - Apply patch
8 - Repeat the modifications above - you still get a single job
9 - Edit koha-conf.xml and add <chunk_size>250</chunk_size> to elasticsearch
stanza
10 - Repeat modifications - you now get several background ES jobs
11 - prove -v t/db_dependent/Koha/SearchEngine/Elasticsearch/Indexer.t
Signed-off-by: David Nind <david at davidnind.com>
Signed-off-by: Jonathan Druart <jonathan.druart at bugs.koha-community.org>
--
You are receiving this mail because:
You are watching all bug changes.
More information about the Koha-bugs
mailing list