[Bug 33019] New: Records not indexed in Elasticsearch ES when first catalogued
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Bug ID: 33019 Summary: Records not indexed in Elasticsearch ES when first catalogued Change sponsored?: --- Product: Koha Version: 22.11 Hardware: All OS: All Status: NEW Severity: blocker Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: aleisha@catalyst.net.nz We have a Koha instance where records are not being indexed into Elasticsearch when they are first catalogued. They can be accessed if you go to the catalogue detail page directly, but cannot be searched. However if you edit the record and save, it is indexed successfully. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Aleisha Amohia <aleisha@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Change sponsored?|--- |Sponsored Assignee|koha-bugs@lists.koha-commun |aleisha@catalyst.net.nz |ity.org | Status|NEW |ASSIGNED --- Comment #1 from Aleisha Amohia <aleisha@catalyst.net.nz> --- We've come up with a hacky fix for this because I can't think of any other reason indexing isn't working, but I am really happy to be advised about this. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Aleisha Amohia <aleisha@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 --- Comment #2 from Aleisha Amohia <aleisha@catalyst.net.nz> --- Created attachment 146999 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=146999&action=edit Bug 33019: Make sure ES records are indexed the first time This patch ensures records are indexed when they are created. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 --- Comment #3 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I have no idea if it's related yet, but there was also bug 33008 filed a little while ago. Could it be a similar issue? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=33008 CC| |fridolin.somers@biblibre.co | |m -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |koha@trust-box.at --- Comment #4 from Mark Hofstetter <koha@trust-box.at> --- Hi Aleisha, is the hack adding an item? if so it seems that it is related to https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33008 cheers -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 --- Comment #5 from Aleisha Amohia <aleisha@catalyst.net.nz> --- Hi all, Yes adding an item does cause the record to be indexed, but this Koha instance does not use items (they are a cataloguing agency). Also it had nothing to do with Zebra being started or stopped, I tried cataloguing with Zebra on and with Zebra off and it had no effect. Aleisha -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Mark Hofstetter <koha@trust-box.at> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=33243 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 --- Comment #6 from Mark Hofstetter <koha@trust-box.at> --- Hi, I hope https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33243 helps, you may use it in a cronjob eg every 5 minutes rebuild_elasticsearch.pl -v -b --minutes 5 if you find in helpful and it works for you, please sign it off thx Mark -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #146999|0 |1 is obsolete| | --- Comment #7 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 148444 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=148444&action=edit Bug 33019: Make sure ES records are indexed the first time This patch ensures records are indexed when they are created. Previously, we were launching the indexer inside of of a transaction. This meant that the job was being enqueued, but not being found by the worker, becaue it was not yet in the DB This patch skips record indexing in the transaction, and moves the indexing command after To test: 1 - Make sure you are using ES, and the es indexer is running 2 - tail -f /var/log/koha/kohadev/*.log 3 - Create a new record 4 - Note error in es-indexer-output.log like: [2023/03/21 12:22:36] [WARN] No job found for id=157 main:: /kohadevbox/koha/misc/workers/es_indexer_daemon.pl (129) 5 - Apply patch 6 - Create another record 7 There should be no error 8 - Search for the record and confirm it can be found 9 - View the background jobs in admin, confirm the most recent job has completed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nick@bywatersolutions.com --- Comment #8 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Aleisha Amohia from comment #1)
We've come up with a hacky fix for this because I can't think of any other reason indexing isn't working, but I am really happy to be advised about this.
I amended the patch, and added some comments, could you test Aleisha? It turns out the issue was that we were inside a transaction -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #148444|0 |1 is obsolete| | --- Comment #9 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 148447 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=148447&action=edit Bug 33019: Make sure ES records are indexed the first time This patch ensures records are indexed when they are created. Previously, we were launching the indexer inside of of a transaction. This meant that the job was being enqueued, but not being found by the worker, becaue it was not yet in the DB This patch skips record indexing in the transaction, and moves the indexing command after To test: 1 - Make sure you are using ES, and the es indexer is running 2 - tail -f /var/log/koha/kohadev/*.log 3 - Create a new record 4 - Note error in es-indexer-output.log like: [2023/03/21 12:22:36] [WARN] No job found for id=157 main:: /kohadevbox/koha/misc/workers/es_indexer_daemon.pl (129) 5 - Apply patch 6 - Create another record 7 There should be no error 8 - Search for the record and confirm it can be found 9 - View the background jobs in admin, confirm the most recent job has completed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=33309 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Aleisha Amohia <aleisha@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Aleisha Amohia <aleisha@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #148447|0 |1 is obsolete| | --- Comment #10 from Aleisha Amohia <aleisha@catalyst.net.nz> --- Created attachment 148572 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=148572&action=edit Bug 33019: Make sure ES records are indexed the first time This patch ensures records are indexed when they are created. Previously, we were launching the indexer inside of of a transaction. This meant that the job was being enqueued, but not being found by the worker, becaue it was not yet in the DB This patch skips record indexing in the transaction, and moves the indexing command after To test: 1 - Make sure you are using ES, and the es indexer is running 2 - tail -f /var/log/koha/kohadev/*.log 3 - Create a new record 4 - Note error in es-indexer-output.log like: [2023/03/21 12:22:36] [WARN] No job found for id=157 main:: /kohadevbox/koha/misc/workers/es_indexer_daemon.pl (129) 5 - Apply patch 6 - Create another record 7 There should be no error 8 - Search for the record and confirm it can be found 9 - View the background jobs in admin, confirm the most recent job has completed Signed-off-by: Aleisha Amohia <aleishaamohia@hotmail.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #148572|0 |1 is obsolete| | --- Comment #11 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 148591 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=148591&action=edit Bug 33019: Make sure ES records are indexed the first time This patch ensures records are indexed when they are created. Previously, we were launching the indexer inside of of a transaction. This meant that the job was being enqueued, but not being found by the worker, becaue it was not yet in the DB This patch skips record indexing in the transaction, and moves the indexing command after To test: 1 - Make sure you are using ES, and the es indexer is running 2 - tail -f /var/log/koha/kohadev/*.log 3 - Create a new record 4 - Note error in es-indexer-output.log like: [2023/03/21 12:22:36] [WARN] No job found for id=157 main:: /kohadevbox/koha/misc/workers/es_indexer_daemon.pl (129) 5 - Apply patch 6 - Create another record 7 There should be no error 8 - Search for the record and confirm it can be found 9 - View the background jobs in admin, confirm the most recent job has completed Signed-off-by: Aleisha Amohia <aleishaamohia@hotmail.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #148591|0 |1 is obsolete| | --- Comment #12 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 148592 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=148592&action=edit Bug 33019: Make sure ES records are indexed the first time This patch ensures records are indexed when they are created. Previously, we were launching the indexer inside of of a transaction. This meant that the job was being enqueued, but not being found by the worker, becaue it was not yet in the DB This patch skips record indexing in the transaction, and moves the indexing command after To test: 1 - Make sure you are using ES, and the es indexer is running 2 - tail -f /var/log/koha/kohadev/*.log 3 - Create a new record 4 - Note error in es-indexer-output.log like: [2023/03/21 12:22:36] [WARN] No job found for id=157 main:: /kohadevbox/koha/misc/workers/es_indexer_daemon.pl (129) 5 - Apply patch 6 - Create another record 7 There should be no error 8 - Search for the record and confirm it can be found 9 - View the background jobs in admin, confirm the most recent job has completed Signed-off-by: Aleisha Amohia <aleishaamohia@hotmail.com> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart+koha@gmail. | |com Assignee|aleisha@catalyst.net.nz |nick@bywatersolutions.com Version|22.11 |unspecified Depends on| |30822 Keywords| |rel_22_11_candidate --- Comment #13 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- I guess that's coming from bug 30822. Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30822 [Bug 30822] BatchCommit does not deal with indexation correctly -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |23.05.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 --- Comment #14 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 23.05. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |Pushed to stable Version(s)|23.05.00 |23.05.00,22.11.04 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 --- Comment #15 from Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com> --- Nice work everyone! Pushed to stable for 22.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Matt Blenkinsop <matt.blenkinsop@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|23.05.00,22.11.04 |23.05.00,22.11.05 released in| | CC| |matt.blenkinsop@ptfs-europe | |.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |RESOLVED Resolution|--- |FIXED CC| |lucas@bywatersolutions.com --- Comment #16 from Lucas Gass <lucas@bywatersolutions.com> --- Missing dependencies for 22.05.x, no backport. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33019 Aleisha Amohia <aleisha@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=35156 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org