[Koha-bugs] [Bug 35345] Pass custom SQL to rebuild_elasticsearch.pl

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35345

--- Comment #6 from Thomas Klausner <domm at plix.at> ---
Created attachment 159044
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=159044&action=edit
Bug 35345: Add --where option to rebuild_elasticsearch.pl

Sometimes we need to only re-index a subset of our bibliographic data or
authorities. Currently this is only possible by enumerating all id (-bn or
-ai), which does not work well when indexing eg 100.000 items of a 2.000.000
DB. Re-indexing everything is also overkill.

This patch adds an `--where` flag to misc/search_tools/rebuild_elasticsearch.pl
which can take arbitrary SQL (that of course has to match the respective
tables) and adds it as an additional param to the resultset to index

To test, start koha-testing-docker with ElasticSearch enabled, for example via
`ktd --es7 up

Before applying the patch, rebuild_elasticsearch will index all data:

Biblios:
$ misc/search_tools/rebuild_elasticsearch.pl -b -v
[12387] Checking state of biblios index
[12387] Indexing biblios
[12387] Committing final records...
[12387] Total 435 records indexed
(there might be a waring regarding a broken biblio, which can be ignored)

Auth:
$ misc/search_tools/rebuild_elasticsearch.pl -a -v
[12546] Checking state of authorities index
[12546] Indexing authorities
[12546] 1000 records processed
[12546] Committing final records...
[12546] Total 1706 records indexed

Now apply the patch

Biblio, limit by range of biblioid:
$ misc/search_tools/rebuild_elasticsearch.pl -b -v --where "biblionumber
between 100 and 150"
[12765] Checking state of biblios index
[12765] Indexing biblios
[12765] Committing final records...
[12765] Total 50 records indexed

Note that only 50 records where indexed (instead of the whole set of 435
records)

Auth, limit by authtypecode:
$ misc/search_tools/rebuild_elasticsearch.pl -a -v --where "authtypecode =
'GEOGR_NAME'"
[12848] Checking state of authorities index
[12848] Indexing authorities
[12848] Committing final records...
[12848] Total 142 records indexed

Again, only 142 have been indexed.

Note that this patch also includes perltidied versions of the affected files
(because the qa script complained)

Sponsored-by: Steiermärkische Landesbibliothek

-- 
You are receiving this mail because:
You are the assignee for the bug.
You are watching all bug changes.