https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Zebraqueue should not be |Zebraqueue should not be |added to |added to when only | |Elasticsearch is used --- Comment #1 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- So I think maybe a new preference or option? -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=26024 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #3 from David Cook <dcook@prosentient.com.au> --- (In reply to Fridolin SOMERS from comment #2)

(In reply to Katrin Fischer from comment #1)

...

So I think maybe a new preference or option?

Or a new value to SearchEngine : ES alone, ES + Zebra, Zebra alone

That's probably more trouble than it's worth? I reckon a new system preference makes more sense? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 --- Comment #5 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- (In reply to Katrin Fischer from comment #4)

I think a new option to the existing or a new pref would both work and be about the same amount of trouble :)

Thinking about it, actual pref "searchengine" is alread used. A new pref whould be less disturbing. Maybe EnableZebraQueue ? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |didier.gautheron@biblibre.c | |om --- Comment #7 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- *** Bug 25710 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 Liz Rea <wizzyrea@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |wizzyrea@gmail.com --- Comment #9 from Liz Rea <wizzyrea@gmail.com> --- let's make it a kconf flag, that way if you need it for whatever reason, a fallback is available -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 --- Comment #11 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- *** Bug 37658 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 Peter Vashchuk <stalkernoid@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |stalkernoid@gmail.com Assignee|koha-bugs@lists.koha-commun |stalkernoid@gmail.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 --- Comment #13 from Peter Vashchuk <stalkernoid@gmail.com> --- Created attachment 174494 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=174494&action=edit Bug 21820: Don't add records to zebraqueue table when using Elasticsearch Right now the index_records function in SearchEngine/Elasticsearch calls the index_records function from SearchEngine/Zebra, causing every items that we index for Elasticsearch to be also added to the zebraqueue table even when only Elasticsearch is in use. So this patch removes that call to prevent that from happening. To reproduce: 1. While using Elasticsearch, edit any biblio record to activate the indexer. 2. Check zebraqueue table and confirm that new record was added. 3. Apply the patch. 4. Edit a biblio record again to trigger the indexer. 5. Check that it was not added to the zebraqueue table after you applied the patch. 6. Check that the biblio was indexed correctly: go to admin->jobs and verify that a new "Update Elasticsearch index" job has been added and completed, additionally use Elasticsearch to find the biblio record you edited to confirm that it works and nothing was broken. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA --- Comment #15 from David Cook <dcook@prosentient.com.au> --- This is on the right track, but it needs to have a configurable option so that people can continue to use both Elasticsearch and Zebra. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 --- Comment #17 from David Cook <dcook@prosentient.com.au> --- (In reply to David Cook from comment #16)

It appears that there is at least 1 part of Koha that is hard-coded to use Zebra even when Elasticsearch is in use.

Please see bug 39351 - "SuggestionEngine plugin AuthorityFile is hard-coded to use Zebra".

Of course, there's nothing like turning off Zebra and only using Elasticsearch to flush out these kinds of problems. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 --- Comment #18 from Fridolin Somers <fridolin.somers@biblibre.com> --- (In reply to David Cook from comment #15)

This is on the right track, but it needs to have a configurable option so that people can continue to use both Elasticsearch and Zebra.

So syspref 'SearchEngine' with 3 values ? ES alone, ES + Zebra, Zebra alone -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |In Discussion CC| |domm@plix.at --- Comment #20 from Thomas Klausner <domm@plix.at> --- At the hackfest we discussed that some libraries need the Zebra-Queue, even though they are using Elasticsearch. So completely removing the zebra indexing (as done in Peters patch from 2025-11-14) is not an option. So we need to solve this via a Syspref. But adding a third option to `SearchEngine` is probably a lot of work (as there are a lot of checks like "if (SearchEngine == Zebra) {} else {}". So all of those would need to be touched. And we don't know if any external / local tools also do checks like this. So we think it's best to add a new SysPref like `ElasticsearchWithZebraQueue`. If it is set, keep the old behavior and add each doc to zebra after doing the ES index. If it is not set, don't do that. For Updates/migration, this value needs to be set to true (so we don't break the old behavior), but for new installations it should the false (so only libraries that really need that feature have to turn it on). Does this make sense to you? If yes, I will prepare a patch etc. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 Janusz Kaczmarek <januszop@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |januszop@gmail.com --- Comment #21 from Janusz Kaczmarek <januszop@gmail.com> --- (In reply to Thomas Klausner from comment #20)

At the hackfest we discussed that some libraries need the Zebra-Queue, even though they are using Elasticsearch. So completely removing the zebra indexing (as done in Peters patch from 2025-11-14) is not an option.

So we need to solve this via a Syspref.

But adding a third option to `SearchEngine` is probably a lot of work (as there are a lot of checks like "if (SearchEngine == Zebra) {} else {}". So all of those would need to be touched. And we don't know if any external / local tools also do checks like this.

So we think it's best to add a new SysPref like `ElasticsearchWithZebraQueue`. If it is set, keep the old behavior and add each doc to zebra after doing the ES index. If it is not set, don't do that.

For Updates/migration, this value needs to be set to true (so we don't break the old behavior), but for new installations it should the false (so only libraries that really need that feature have to turn it on).

Does this make sense to you?

If yes, I will prepare a patch etc.

And how about not starting zebra and zebra-indexer in case of Elasticsearch only? Now both are started by default (the indexer can be disabled in /etc/default/koha-common, but I'm not sure if we can rely on this). -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #174494|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Needs Signoff Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|Small patch |Trivial patch Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 --- Comment #24 from Fridolin Somers <fridolin.somers@biblibre.com> --- Created attachment 196306 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=196306&action=edit Bug 21820: Add ElasticsearchEnableZebraQueue syspref to control Zebra indexing when using Elasticsearch Previously, Koha always added items to the Zebra queue when indexing via Elasticsearch. This adds a new system preference ElasticsearchEnableZebraQueue that allows disabling this behavior. Unit Test: * run the unit test t/db_dependent/Koha/SearchEngine/Elasticsearch/Indexer.t * there should be a new subtest like this: 1..5 ok 1 - We start with an empty zebra queue ok 2 - new entry in zebraqueue for auth ok 3 - new entry in zebraqueue for biblio ok 4 - no new entry in zebraqueue for auth ok 5 - no new entry in zebraqueue for biblio ok 5 - syspref ElasticsearchEnableZebraQueue Manual Test Plan: * Start ktd with Elasticsearch enabled * As there is no interface to the zebraqueue, you need to connect to the DB and use SQL directly * ktd --dbshell * `select count(*) from zebraqueue;` * remember that number * Edit any biblio or auth * Check the zebraqueue again via SQL * There should be one more entry * Now apply the patch * Then got to Sysprefs and set ElasticsearchEnableZebraQueue to `Disable` * Edit another biblio or auth * has to be a different one, because Koha will prevent the same ID to be added again to zebraqueue * Again check zebraqueue via SQL * There should be *no* new entry QA Note: the qa tool reports [FAIL] Koha/SearchEngine/Elasticsearch/Indexer.pm FAIL pod_coverage sub _get_record has no pod coverage. But this patch does not touch this code, so I guess this fail is caused by a stricter qa tool Signed-off-by: Fridolin Somers <fridolin.somers@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 --- Comment #25 from Thomas Klausner <domm@plix.at> --- Created attachment 196448 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=196448&action=edit Bug 21820: Add ElasticsearchEnableZebraQueue syspref to control Zebra indexing when using Elasticsearch Previously, Koha always added items to the Zebra queue when indexing via Elasticsearch. This adds a new system preference ElasticsearchEnableZebraQueue that allows disabling this behavior. Unit Test: * run the unit test t/db_dependent/Koha/SearchEngine/Elasticsearch/Indexer.t * there should be a new subtest like this: 1..5 ok 1 - We start with an empty zebra queue ok 2 - new entry in zebraqueue for auth ok 3 - new entry in zebraqueue for biblio ok 4 - no new entry in zebraqueue for auth ok 5 - no new entry in zebraqueue for biblio ok 5 - syspref ElasticsearchEnableZebraQueue Manual Test Plan: * Start ktd with Elasticsearch enabled * As there is no interface to the zebraqueue, you need to connect to the DB and use SQL directly * ktd --dbshell * `select count(*) from zebraqueue;` * remember that number * Edit any biblio or auth * Check the zebraqueue again via SQL * There should be one more entry * Now apply the patch * Then got to Sysprefs and set ElasticsearchEnableZebraQueue to `Disable` * Edit another biblio or auth * has to be a different one, because Koha will prevent the same ID to be added again to zebraqueue * Again check zebraqueue via SQL * There should be *no* new entry QA Note: the qa tool reports [FAIL] Koha/SearchEngine/Elasticsearch/Indexer.pm FAIL pod_coverage sub _get_record has no pod coverage. But this patch does not touch this code, so I guess this fail is caused by a stricter qa tool -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 Paul Derscheid <paul.derscheid@lmscloud.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #196448|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 Paul Derscheid <paul.derscheid@lmscloud.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |paul.derscheid@lmscloud.de QA Contact| |paul.derscheid@lmscloud.de -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 --- Comment #28 from Paul Derscheid <paul.derscheid@lmscloud.de> --- One question, is the divergence regarding the default value between the db update and sysprefs.sql intentional? So keeping it on for upgraded instances but off by default on new ones? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 --- Comment #30 from Paul Derscheid <paul.derscheid@lmscloud.de> --- Makes sense, thanks :) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 --- Comment #31 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Nice work everyone! Pushed to main for 26.05 -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |additional_work_needed CC| |jonathan.druart@gmail.com -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21820 Jacob O'Mara <jacob.omara@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to main |Needs documenting --- Comment #34 from Jacob O'Mara <jacob.omara@openfifth.co.uk> --- This will not be backported to 25.11.x due to it being deemed an enhancement or a dependancy not being met -- You are receiving this mail because: You are watching all bug changes.