[Bug 35815] New: [Elasticsearch] Fix indexing of bibliographic records when items are deleted
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35815 Bug ID: 35815 Summary: [Elasticsearch] Fix indexing of bibliographic records when items are deleted Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Tools Assignee: koha-bugs@lists.koha-community.org Reporter: aleisha@catalyst.net.nz QA Contact: testopia@bugs.koha-community.org Bibliographic records do not always get reindexed in Elasticsearch when items are batch deleted. This means that search results show an inaccurate number of remaining items. To reproduce: 1. Spin up a Koha instance in your testing environment which uses Elasticsearch 2. Create a new bibliographic record and add three items to it 3. Do a search for your record that will return multiple results, so that you are shown the search results. Confirm that the search results shows three items are attached to the record. 4. Go to the record and select the checkbox for an item. 5. Click the link to Delete selected items. Confirm this again on the next page. 6. Confirm the job is enqueued then completed. It should say your item has been deleted. 7. Do a search for your record again so that you see it in the search results. Notice the number of items has not been updated - it still says three --> BUG -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35815 Aleisha Amohia <aleisha@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Change sponsored?|--- |Sponsored Assignee|koha-bugs@lists.koha-commun |aleisha@catalyst.net.nz |ity.org | Status|NEW |ASSIGNED -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35815 Aleisha Amohia <aleisha@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35815 --- Comment #1 from Aleisha Amohia <aleisha@catalyst.net.nz> --- Created attachment 161052 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=161052&action=edit Bug 35815: Ensure ES biblio index is updated when items are batch deleted This patch moves the Elasticsearch indexing outside of the transaction. This ensures that bibliographic records are reindexed when items are batch deleted. To test: 1. Spin up a Koha instance in your testing environment which uses Elasticsearch 2. Create a new bibliographic record and add three items to it 3. Do a search for your record that will return multiple results, so that you are shown the search results. Confirm that the search results shows three items are attached to the record. 4. Go to the record and select the checkbox for an item. 5. Click the link to Delete selected items. Confirm this again on the next page. 6. Confirm the job is enqueued then completed. It should say your item has been deleted. 7. Do a search for your record again so that you see it in the search results. Notice the number of items has not been updated - it still says three --> BUG 8. Apply the patch and restart services 9. Go to the record (there will be two items now) and select the checkbox for one item. 10. Click the link to Delete selected items. Confirm this again on the next page. 11. Confirm the job is enqueued then completed. It should say your item has been deleted. 12. Do a search for your record again so that you see it in the search results. Confirm the number of items HAS been updated. It should now show one item attached to the record. Sponsored-by: Toi Ohomai Institute of Technology -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35815 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #2 from David Nind <david@davidnind.com> --- I had a go at testing on the current master using koha-testing-docker (KTD). I couldn't replicate the issue using Elasticsearch 7 and 8, and Open Search 1 and 2. Happy to try with another Koha version, if you think that will make a difference. Testing steps: 1. As per the test plan, added a record with title "DITA best practices" - minimal record with book as the item type. 2. Search was for "best" - 9 results with the sample data + new record. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35815 --- Comment #3 from Aleisha Amohia <aleisha@catalyst.net.nz> --- (In reply to David Nind from comment #2)
I had a go at testing on the current master using koha-testing-docker (KTD).
I couldn't replicate the issue using Elasticsearch 7 and 8, and Open Search 1 and 2.
That's a good point - it was definitely happening with ES 6, I'm not sure if it happened with ES 7. We might no longer be supporting ES 6 and since this patch was written we have actually upgraded to ES 7 locally anyway so it might not be needed... however I believe it is consistent with other ES indexing fixes I've seen recently in Koha. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35815 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=35092 --- Comment #4 from David Nind <david@davidnind.com> --- I've tested using Koha 22.11 and ES6, and I couldn't replicate the issue. Adding bug 35092 in case those working on the background job/worker issues have any comments. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35815 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |Sandbox -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35815 Michaela Sieber <michaela.sieber@kit.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |emmi.takkinen@koha-suomi.fi | |, michaela.sieber@kit.edu --- Comment #5 from Michaela Sieber <michaela.sieber@kit.edu> --- There was a discussion on the devel mailinglist Wed, 31 Jan 2024 : "We've had some reports that biblios which have been deleted via "Merge records" aren't always deleted from index and are shown in the search result list. When one opens such a result, Koha of course says there is no such a record in database." -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35815 --- Comment #6 from Emmi Takkinen <emmi.takkinen@koha-suomi.fi> --- This has been problem for us for a while (using Koha 22.11 and Elasticsearch 6), mostly with combining biblios. But this occurs randomly and is hard to reproduce so I'm unable to provide any specific test case. We decided to wait and see if problem continues in new versions of Koha and Elasticsearch. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35815 Clemens Tubach <clemens.tubach@kit.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |clemens.tubach@kit.edu --- Comment #7 from Clemens Tubach <clemens.tubach@kit.edu> --- We also have this problem on production 23.05.04 using elastic 7. The background job 'batch_item_record_deletion' was finished successfully and items are deleted. But the related background job 'update_elastic_index' stays in status 'new', so the item is visible in the search result. We couldn't reproduce it on current master with KTD either. We hope that the problem will be solved with improvements in bug 35819 and related. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35815 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@ptfs-europe | |.com Status|Needs Signoff |In Discussion -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org