[Bug 21286] New: Advanced search for Corporate-name creates Zebra errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 Bug ID: 21286 Summary: Advanced search for Corporate-name creates Zebra errors Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Searching - Zebra Assignee: koha-bugs@lists.koha-community.org Reporter: dcook@prosentient.com.au If you're in the advanced search in the staff client, and if you're using the QueryWeightFields system preference, and if you're searching with "Corporate-name", you'll generate Zebra errors. There are a few things contributing to this problem. One is that Corporate-name is only indexed into the "word" register and not the "phrase" register. This becomes a problem with QueryWeightFields, because of the last conditional block in C4::Search::_build_weighted_query() which rewrites the search query to include $index,ext,r1, which forces a search of the "phrase" register, which causes Zebra to explode (when it doesn't use the 14=1 special attribute). -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 --- Comment #1 from David Cook <dcook@prosentient.com.au> --- For example: 11:46:06-30/08 zebrasrv(44) [request] Search biblios ERROR 114 1 1+0 RPN @attrset Bib-1 @and @or @or @attr 1=Corporate-name @attr 4=1 @attr 6=3 @attr 9=32 @attr 2=102 corruption @attr 1=Corporate-name @attr 4=1 @attr 9=26 @attr 2=102 corruption @attr 1=Corporate-name @attr 4=6 @attr 9=20 @attr 2=102 corruption @attr 1=homebranch CPL If you try that same search in yaz-client, you'll get something like: Z> f @attrset Bib-1 @and @or @or @attr 1=Corporate-name @attr 4=1 @attr 6=3 @attr 9=32 @attr 2=102 corruption @attr 1=Corporate-name @attr 4=1 @attr 9=26 @attr 2=102 corruption @attr 1=Corporate-name @attr 4=6 @attr 9=20 @attr 2=102 corruption @attr 1=homebranch CPL Sent searchRequest. Received SearchResponse. Search was a bloomin' failure. Number of hits: 0, setno 1 Result Set Status: none records returned: 0 Diagnostic message(s) from database: [114] Unsupported Use attribute -- v2 addinfo 'Corporate-name' Elapsed: 0.006073 You can get some insight from this wiki entry: https://wiki.koha-community.org/wiki/Troubleshooting_Zebra#What_is_an_Unsupp... (I wrote this years ago and it needs work but it's mostly on point.) So it's an issue with the index not belonging in a particular register. As per my earlier comment and based on the search query, it's the "phrase" register it's missing from. The interesting thing is that if we remove the @attr 6=3 from the above query, the query returns successfully. That's because the "phrase" register is only used for complete fields (ie for queries that have a @attr 6=3 attribute). You can see this in Table 5.13 at https://software.indexdata.com/zebra/doc/querymodel-zebra.html. Mind blown. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 --- Comment #2 from David Cook <dcook@prosentient.com.au> --- Now the question is do we put Corporate-name in the phrase register in the Zebra indexing XSLT... ...or do we re-think C4::Search::_build_weighted_query() because it's probably causing this exact same problem in lots of other contexts. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 --- Comment #3 from David Cook <dcook@prosentient.com.au> --- Or do we not care because we're just trying to move forward with Elasticsearch? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 --- Comment #4 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- (In reply to David Cook from comment #2)
Now the question is do we put Corporate-name in the phrase register in the Zebra indexing XSLT...
...or do we re-think C4::Search::_build_weighted_query() because it's probably causing this exact same problem in lots of other contexts.
Indeed, its a tricky error. We have the same issue in UNIMARC. I think we may change etc/zebradb/xsl/koha-indexdefs-to-zebra.xsl to always add phrase indexing on a words-list index (like Corporate-name:w) -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 --- Comment #5 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 121326 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=121326&action=edit Bug 21286: Add Corporate-name as phrase to zebra indexes When using Zebra for searching, Koha performs a number of searches in order to improve relevancy. This means that even for 'wordlist' search, we perform a phrase search. When selecting 'Corporate-name' as an index, this expansion of the search causes errors and fails the search We can fix this for 'Corporate-name' searches by adding a phrase index To test: 1 - Edit koha-conf.xml and uncomment the zebra debug line and add 'request' to the list 2 - Restart all 3 - tail -f /var/log/koha/kohadev/zebra-output.log 4 - Edit a record to add a 110 field e.g. 'House plants' 5 - Enable syspref IntranetCatalogSearchPulldown 6 - Search for 'Corporate name' and term 'House plants' 7 - No results 8 - View the log, see 'ERROR' and full search terms listed 9 - Apply patch 10 - copy the zebra files to the production instance: cp etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml cp etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl 11 - restart all 12 - rebuild: sudo koha-rebuild-zebra -v -f kohadev 13 - Repeat search 14 - Success! -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | CC| |nick@bywatersolutions.com -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #6 from David Nind <david@davidnind.com> --- I attempted to test this, but I couldn't recreate the error(steps 4-8) - I could find the record with the corporate name in it (I added a unique name not used anywhere else). The QueryWeightFields system preference is enabled. I changed the dropdown list to 'Corporate name' and also tried with 'Corporate name as a phrase'. Is there something else I need to do to try and recreate the issue? Here is the Zebra log entry for the request: 01:28:46-30/05 zebrasrv(54) [request] Search biblios OK 1 1 1+0 RPN @attrset Bib-1 @attr 1=Corporate-name @attr 4=6 @attr 5=1 @attr 2=102 "xanadu mftc" 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(54) [request] Present OK - 1 1+1 01:28:46-30/05 zebrasrv(55) [request] Auth idPass kohauser - 01:28:46-30/05 zebrasrv(55) [request] Init OK - ID:81 Name:ZOOM-C/YAZ Version:5.27.1 872b6f92a024bb53bc1c11dfeccd47f065f98238 01:28:46-30/05 zebrasrv(55) [request] Search biblios OK 0 1 1+0 RPN @attrset Bib-1 @attr 1=1033 "Programming Perl /" Testing notes (koha-testing-docker): - Enabled Zebra logging: . edit /etc/koha/sites/kohadev/koha-conf.xml . find zebra_loglevels setting . uncomment and add request - Added "xanadu mftc" as an authority record and linked to a record -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 --- Comment #7 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to David Nind from comment #6)
I attempted to test this, but I couldn't recreate the error(steps 4-8) - I could find the record with the corporate name in it (I added a unique name not used anywhere else).
Try disabling the syspref 'QueryAutoTruncate' - that seems to be a key to triggering this -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 --- Comment #8 from David Nind <david@davidnind.com> ---
Try disabling the syspref 'QueryAutoTruncate' - that seems to be a key to triggering this
Thanks Nick! That worked, sign off on the way.. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #121326|0 |1 is obsolete| | --- Comment #9 from David Nind <david@davidnind.com> --- Created attachment 121554 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=121554&action=edit Bug 21286: Add Corporate-name as phrase to zebra indexes When using Zebra for searching, Koha performs a number of searches in order to improve relevancy. This means that even for 'wordlist' search, we perform a phrase search. When selecting 'Corporate-name' as an index, this expansion of the search causes errors and fails the search We can fix this for 'Corporate-name' searches by adding a phrase index To test: 1 - Edit koha-conf.xml and uncomment the zebra debug line and add 'request' to the list 2 - Restart all 3 - tail -f /var/log/koha/kohadev/zebra-output.log 4 - Edit a record to add a 110 field e.g. 'House plants' 5 - Enable syspref IntranetCatalogSearchPulldown 6 - Search for 'Corporate name' and term 'House plants' 7 - No results 8 - View the log, see 'ERROR' and full search terms listed 9 - Apply patch 10 - copy the zebra files to the production instance: cp etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml cp etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl 11 - restart all 12 - rebuild: sudo koha-rebuild-zebra -v -f kohadev 13 - Repeat search 14 - Success! Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |katrin.fischer@bsz-bw.de -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #121554|0 |1 is obsolete| | --- Comment #10 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 121567 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=121567&action=edit Bug 21286: Add Corporate-name as phrase to zebra indexes When using Zebra for searching, Koha performs a number of searches in order to improve relevancy. This means that even for 'wordlist' search, we perform a phrase search. When selecting 'Corporate-name' as an index, this expansion of the search causes errors and fails the search We can fix this for 'Corporate-name' searches by adding a phrase index To test: 1 - Edit koha-conf.xml and uncomment the zebra debug line and add 'request' to the list 2 - Restart all 3 - tail -f /var/log/koha/kohadev/zebra-output.log 4 - Edit a record to add a 110 field e.g. 'House plants' 5 - Enable syspref IntranetCatalogSearchPulldown 6 - Search for 'Corporate name' and term 'House plants' 7 - No results 8 - View the log, see 'ERROR' and full search terms listed 9 - Apply patch 10 - copy the zebra files to the production instance: cp etc/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-koha-indexdefs.xml cp etc/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl /etc/koha/zebradb/marc_defs/marc21/biblios/biblio-zebra-indexdefs.xsl 11 - restart all 12 - rebuild: sudo koha-rebuild-zebra -v -f kohadev 13 - Repeat search 14 - Success! Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |21.11.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 --- Comment #11 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Pushed to master for 21.11, thanks to everybody involved! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This fixes the advanced release notes| |search in the staff | |interface so that searching | |using the 'Corporate name' | |index now works correctly | |when the QueryAutoTruncate | |system preference is not | |enabled. Before this a | |search (using Zebra) for a | |name such as 'House plants' | |would not return any | |results and generate error | |messages in the log files. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |Pushed to stable Version(s)|21.11.00 |21.11.00,21.05.02 released in| | CC| |kyle@bywatersolutions.com --- Comment #12 from Kyle M Hall <kyle@bywatersolutions.com> --- Pushed to 21.05.x for 21.05.02 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 --- Comment #13 from Fridolin Somers <fridolin.somers@biblibre.com> --- Knowing that, we may add in koha-indexdefs-to-zebra.xsl the phrase form if missing -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|21.11.00,21.05.02 |21.11.00,21.05.02,20.11.08 released in| | Status|Pushed to stable |Pushed to oldstable --- Comment #14 from Fridolin Somers <fridolin.somers@biblibre.com> --- Pushed to 20.11.x for 20.11.08 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=21286 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|Pushed to oldstable |RESOLVED CC| |victor@tuxayo.net --- Comment #15 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Not backported to oldoldstable (20.05.x). Feel free to ask if it's needed. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org