[Bug 31532] New: Zebra search results incorrect because of Bug 15187
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 Bug ID: 31532 Summary: Zebra search results incorrect because of Bug 15187 Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching - Zebra Assignee: koha-bugs@lists.koha-community.org Reporter: dcook@prosentient.com.au In most parts of Koha, we use the MARCXML from the MySQL database, but in the search results we use the MARCXML from Zebra. While Bug 15187 allows us to index 880 according to its linkages, it means that the 880 is replaced and additional linked fields are created, which makes for some incorrect Zebra search results in Koha... -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |15187 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15187 [Bug 15187] Adding 880 Fields to index-list in order to Increase Search for ALL non-latin Scripts -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |dcook@prosentient.com.au |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 --- Comment #1 from David Cook <dcook@prosentient.com.au> --- The solution from https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15187#c14 works well, especially for package installs. To overcome the issues from bug 15187, you can reindex the records at your leisure. If you don't overcome it immediately, everything will keep working. But when you next index a record, it'll have the best indexing done (in regards to the 880 handling). I'm implementing it locally, and I'll provide a patch here soon. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |15187 Depends on|15187 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15187 [Bug 15187] Adding 880 Fields to index-list in order to Increase Search for ALL non-latin Scripts -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 --- Comment #2 from David Cook <dcook@prosentient.com.au> --- Created attachment 140331 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=140331&action=edit Bug 31532: Add preprocessor to Zebra indexing pipeline to index 880 as linked field This patch adds a preprocessor XSLT to the Zebra indexing pipeline, so that 880 fields get indexed as the fields they're linked to. For example, a "880 $6 245" field would be indexed as a "245" field. However, because the preprocessor only occurs in the indexing part of the pipeline, it does not affect the retrieval of MARCXML from Zebra. That MARCXML will be the same MARCXML that was sent to Zebra from Koha. Test plan: 0. Revert bug 15187, and apply patch for 31532 1. cp ./etc/zebradb/biblios/etc/dom-config.xml /etc/koha/zebradb/biblios/etc/dom-config.xml 2a. cp etc/zebradb/marc_defs/marc21/biblios/preprocess_marcxml.xsl /etc/koha/zebradb/marc_defs/marc21/biblios/. 2b. cp etc/zebradb/marc_defs/normarc/biblios/preprocess_marcxml.xsl /etc/koha/zebradb/marc_defs/normarc/biblios/. 2c. cp etc/zebradb/marc_defs/unimarc/biblios/preprocess_marcxml.xsl /etc/koha/zebradb/marc_defs/unimarc/biblios/. 3. koha-rebuild-zebra -b -f -v kohadev 4. Note that in search results the 880$6245$a data appears before the 245$a data 5. Note that you can do a title index search on the 880$6245$a data -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 --- Comment #3 from David Cook <dcook@prosentient.com.au> --- If you need 880 data to test with: 1. Go to http://localhost:8081/cgi-bin/koha/admin/marc_subfields_structure.pl?op=add_form&tagfield=880&frameworkcode= 2. Go to "a" and "6" subfields, and add "Editor" for "Visibility" 3. Go to http://localhost:8081/cgi-bin/koha/cataloguing/addbiblio.pl?biblionumber=29&searchid=scs_1662620103145 4. Change framework to "Default framework" 5. In the "880" field, add "245-01" for the $6 6. In the "880" field, add "THISISATEST" for the $a (Using English for the 880$a makes it a bit easier for testing using CHR indexing.) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |normal -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 --- Comment #4 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- The revert bit means this can only be tested on a local installation - I'll give it a go. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 --- Comment #5 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- * I verified the problem is only visible when using Zebra. * Found a search for my title returned all records on ktd = no ICU * Tried activating ICU: https://wiki.koha-community.org/wiki/ICU_chains_configuration * Noticed reindex failed: https://wiki.koha-community.org/wiki/ICU_chains_configuration * Killed the record with SQL * Reindexed again... Output is clean. * Search now brings back the exported record. * Why do we make this so hard? * Now revert: git revert 85690a14f8 git revert e6947b7408 git revert 058d411643 * Apply patch * Copy updated Zebra files so they are used: cp -i ./etc/zebradb/biblios/etc/dom-config.xml /etc/koha/zebradb/biblios/etc/dom-config.xml cp -i ./etc/zebradb/marc_defs/marc21/biblios/preprocess_marcxml.xsl /etc/koha/zebradb/marc_defs/marc21/biblios/ cp -i ./etc/zebradb/marc_defs/unimarc/biblios/preprocess_marcxml.xsl /etc/koha/zebradb/marc_defs/unimarc/biblios/ I've left out the change for NORMARC, as it's unsupported (see bug 18984) * Restarted Zebra server: sudo koha-zebra --restart -v kohadev 3. sudo koha-rebuild-zebra -b -f -v kohadev 4. Note that in search results the 880$6245$a data appears before the 245$a data 5. Note that you can do a title index search on the 880$6245$a data * Verify title words are not found in author index and vice versa :) All good, signing off! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #140331|0 |1 is obsolete| | --- Comment #6 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 140760 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=140760&action=edit Bug 31532: Add preprocessor to Zebra indexing pipeline to index 880 as linked field This patch adds a preprocessor XSLT to the Zebra indexing pipeline, so that 880 fields get indexed as the fields they're linked to. For example, a "880 $6 245" field would be indexed as a "245" field. However, because the preprocessor only occurs in the indexing part of the pipeline, it does not affect the retrieval of MARCXML from Zebra. That MARCXML will be the same MARCXML that was sent to Zebra from Koha. Test plan: 0. Revert bug 15187, and apply patch for 31532 1. cp ./etc/zebradb/biblios/etc/dom-config.xml /etc/koha/zebradb/biblios/etc/dom-config.xml 2a. cp etc/zebradb/marc_defs/marc21/biblios/preprocess_marcxml.xsl /etc/koha/zebradb/marc_defs/marc21/biblios/. 2b. cp etc/zebradb/marc_defs/normarc/biblios/preprocess_marcxml.xsl /etc/koha/zebradb/marc_defs/normarc/biblios/. 2c. cp etc/zebradb/marc_defs/unimarc/biblios/preprocess_marcxml.xsl /etc/koha/zebradb/marc_defs/unimarc/biblios/. 3. koha-rebuild-zebra -b -f -v kohadev 4. Note that in search results the 880$6245$a data appears before the 245$a data 5. Note that you can do a title index search on the 880$6245$a data Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Zebra search results |Zebra search results |incorrect because of Bug |including 880 with original |15187 |script incorrect because of | |Bug 15187 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #140760|0 |1 is obsolete| | --- Comment #7 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 141337 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=141337&action=edit Bug 31532: Add preprocessor to Zebra indexing pipeline to index 880 as linked field This patch adds a preprocessor XSLT to the Zebra indexing pipeline, so that 880 fields get indexed as the fields they're linked to. For example, a "880 $6 245" field would be indexed as a "245" field. However, because the preprocessor only occurs in the indexing part of the pipeline, it does not affect the retrieval of MARCXML from Zebra. That MARCXML will be the same MARCXML that was sent to Zebra from Koha. Test plan: 0. Revert bug 15187, and apply patch for 31532 1. cp ./etc/zebradb/biblios/etc/dom-config.xml /etc/koha/zebradb/biblios/etc/dom-config.xml 2a. cp etc/zebradb/marc_defs/marc21/biblios/preprocess_marcxml.xsl /etc/koha/zebradb/marc_defs/marc21/biblios/. 2b. cp etc/zebradb/marc_defs/normarc/biblios/preprocess_marcxml.xsl /etc/koha/zebradb/marc_defs/normarc/biblios/. 2c. cp etc/zebradb/marc_defs/unimarc/biblios/preprocess_marcxml.xsl /etc/koha/zebradb/marc_defs/unimarc/biblios/. 3. koha-rebuild-zebra -b -f -v kohadev 4. Note that in search results the 880$6245$a data appears before the 245$a data 5. Note that you can do a title index search on the 880$6245$a data Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@ptfs-europe | |.com --- Comment #8 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Works well.. RM/RMaints remember to do the revert step in your branches as you roll this out. Passing QA. Note: I'm wondering now if this also affect Elastics as we effectively copied what we're doing there for bug 15187.. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com Keywords| |release-notes-needed --- Comment #9 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Please, add a comment about the required steps after upgrade for end users, so we add it to a technical section on the release notes. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |22.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 --- Comment #10 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 22.11. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|22.11.00 |22.11.00, 22.05.08 released in| | CC| |lucas@bywatersolutions.com --- Comment #11 from Lucas Gass <lucas@bywatersolutions.com> --- Backported to 22.05.x for upcoming 22.05.08 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 Arthur Suzuki <arthur.suzuki@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |arthur.suzuki@biblibre.com Version(s)|22.11.00, 22.05.08 |22.11.00, 22.05.08, released in| |21.11.15 Status|Pushed to master |Pushed to stable --- Comment #12 from Arthur Suzuki <arthur.suzuki@biblibre.com> --- applied to 21.11 for 21.11.15 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 Arthur Suzuki <arthur.suzuki@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |Pushed to oldstable -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 wainuiwitikapark@catalyst.net.nz changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |wainuiwitikapark@catalyst.n | |et.nz --- Comment #13 from wainuiwitikapark@catalyst.net.nz --- Not backported to 21.05.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 --- Comment #14 from David Cook <dcook@prosentient.com.au> --- (In reply to Lucas Gass from comment #11)
Backported to 22.05.x for upcoming 22.05.08
It looks like bug 15187 wasn't reverted. I think that needs to be done too. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31532 Lisette Scheer <lisette@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|release-notes-needed | -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org