[Bug 25864] New: Case sensitivity breaks searching of some fields in ES5
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 Bug ID: 25864 Summary: Case sensitivity breaks searching of some fields in ES5 Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: major Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: nick@bywatersolutions.com In Elasticsearch.pm: 1161 # Force lower case on indexed field names for case insensitive 1162 # field name searches 1163 lc($search_field->name), But in QueryBuilder.pm 1172 push @search_fields, [ 1173 $search_field->name, Now that we pass 'fields' for all generic searches this breaks things in ES5 To recreate: 1 - Have Koha using ES5 and Elasticsearch as search engine 2 - Enable DumpTemplateVarsIntranet and DumpSearchQueryTemplate 3 - Do a search in authorities using 'Search entire record' (abduction if using sample db) 4 - Note no results 5 - View the page source and find 'search_query' 6 - Note the uppercased fields 7 - curl 'es:9200/koha_kohadev_authorities/data/417?pretty' 8 - Note all fields lower-cased -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 --- Comment #1 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 106246 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106246&action=edit Bug 25864: Unit test adjustment -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 --- Comment #2 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 106247 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106247&action=edit Bug 25864 - Lower case fields for searching To recreate: 1 - Have Koha using ES5 and Elasticsearch as search engine 2 - Enable DumpTemplateVarsIntranet and DumpSearchQueryTemplate 3 - Do a search in authorities using 'Search entire record' (abduction if using sample db) 4 - Note no results 5 - View the page source and find 'search_query' 6 - Note the uppercased fields 7 - curl 'es:9200/koha_kohadev_authorities/data/417?pretty' 8 - Note all fields lower-cased 9 - Apply patch 10 - Repeat search 11 - It works! -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |alex.arnaud@biblibre.com, | |andrew@bywatersolutions.com | |, frederic@tamil.fr, | |jonathan.druart@bugs.koha-c | |ommunity.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 Heather <heather_hernandez@nps.gov> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |heather_hernandez@nps.gov -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 --- Comment #3 from Heather <heather_hernandez@nps.gov> --- Hi! I was unable to recreate the problem in my sandbox. In my sandbox (which Nick made sure satisfied step 1--thank you, Nick!) I followed step 2, changing those two sysprefs from "Don't" to "Do," but then when trying step 3 with 'Search entire record' selected for the search term: fiction I do get results. I also get authority record results with all four authority record search options when searching: rowling Am I doing something wrong to recreate the problem? Thanks! Cheerio! h2 (In reply to Nick Clemens from comment #2)
Created attachment 106247 [details] [review] Bug 25864 - Lower case fields for searching
To recreate: 1 - Have Koha using ES5 and Elasticsearch as search engine 2 - Enable DumpTemplateVarsIntranet and DumpSearchQueryTemplate 3 - Do a search in authorities using 'Search entire record' (abduction if using sample db) 4 - Note no results 5 - View the page source and find 'search_query' 6 - Note the uppercased fields 7 - curl 'es:9200/koha_kohadev_authorities/data/417?pretty' 8 - Note all fields lower-cased 9 - Apply patch 10 - Repeat search 11 - It works!
-- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 --- Comment #4 from Heather <heather_hernandez@nps.gov> --- Replying to myself to note that my confusion is cleared up--was able to follow test plan through step 6. Still working on it... (In reply to Heather from comment #3)
Hi!
I was unable to recreate the problem in my sandbox. In my sandbox (which Nick made sure satisfied step 1--thank you, Nick!) I followed step 2, changing those two sysprefs from "Don't" to "Do," but then when trying step 3 with 'Search entire record' selected for the search term: fiction I do get results. I also get authority record results with all four authority record search options when searching: rowling
Am I doing something wrong to recreate the problem?
Thanks! Cheerio! h2
(In reply to Nick Clemens from comment #2)
Created attachment 106247 [details] [review] [review] Bug 25864 - Lower case fields for searching
To recreate: 1 - Have Koha using ES5 and Elasticsearch as search engine 2 - Enable DumpTemplateVarsIntranet and DumpSearchQueryTemplate 3 - Do a search in authorities using 'Search entire record' (abduction if using sample db) 4 - Note no results 5 - View the page source and find 'search_query' 6 - Note the uppercased fields 7 - curl 'es:9200/koha_kohadev_authorities/data/417?pretty' 8 - Note all fields lower-cased 9 - Apply patch 10 - Repeat search 11 - It works!
-- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 ByWater Sandboxes <bws.sandboxes@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #106247|0 |1 is obsolete| | --- Comment #5 from ByWater Sandboxes <bws.sandboxes@gmail.com> --- Created attachment 106345 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106345&action=edit Bug 25864 - Lower case fields for searching To recreate: 1 - Have Koha using ES5 and Elasticsearch as search engine 2 - Enable DumpTemplateVarsIntranet and DumpSearchQueryTemplate 3 - Do a search in authorities using 'Search entire record' (abduction if using sample db) 4 - Note no results 5 - View the page source and find 'search_query' 6 - Note the uppercased fields 7 - curl 'es:9200/koha_kohadev_authorities/data/417?pretty' 8 - Note all fields lower-cased 9 - Apply patch 10 - Repeat search 11 - It works! Signed-off-by: Heather Hernandez <heather_hernandez@nps.gov> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 Heather <heather_hernandez@nps.gov> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off --- Comment #6 from Heather <heather_hernandez@nps.gov> --- Works well! I can't wait to get this fix in our catalog! Thank you, Nick, and everyone!!! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |alex.arnaud@biblibre.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #106246|0 |1 is obsolete| | --- Comment #7 from Alex Arnaud <alex.arnaud@biblibre.com> --- Created attachment 106491 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106491&action=edit Bug 25864: Unit test adjustment Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #106345|0 |1 is obsolete| | --- Comment #8 from Alex Arnaud <alex.arnaud@biblibre.com> --- Created attachment 106492 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106492&action=edit Bug 25864 - Lower case fields for searching To recreate: 1 - Have Koha using ES5 and Elasticsearch as search engine 2 - Enable DumpTemplateVarsIntranet and DumpSearchQueryTemplate 3 - Do a search in authorities using 'Search entire record' (abduction if using sample db) 4 - Note no results 5 - View the page source and find 'search_query' 6 - Note the uppercased fields 7 - curl 'es:9200/koha_kohadev_authorities/data/417?pretty' 8 - Note all fields lower-cased 9 - Apply patch 10 - Repeat search 11 - It works! Signed-off-by: Heather Hernandez <heather_hernandez@nps.gov> Signed-off-by: Alex Arnaud <alex.arnaud@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 koha-US bug tracker <bugzilla@koha-us.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bugzilla@koha-us.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |20.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 --- Comment #9 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Pushed to master for 20.11, thanks to everybody involved! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com Version(s)|20.11.00 |20.11.00. 20.05.02 released in| | Status|Pushed to master |Pushed to stable --- Comment #10 from Lucas Gass <lucas@bywatersolutions.com> --- backported to 20.05.x for 20.05.02 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 Aleisha Amohia <aleisha@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |aleisha@catalyst.net.nz Version(s)|20.11.00. 20.05.02 |20.11.00. 20.05.02, released in| |19.11.08 Status|Pushed to stable |Pushed to oldstable --- Comment #11 from Aleisha Amohia <aleisha@catalyst.net.nz> --- backported to 19.11.x for 19.11.08 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |victor@tuxayo.net --- Comment #12 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Does look to be affecting an existing feature of 19.05.x, won't backport. (given how things looked when trying to backport) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 --- Comment #13 from Victor Grousset/tuxayo <victor@tuxayo.net> --- *Doesn't look -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |20589 --- Comment #14 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Victor Grousset/tuxayo from comment #12)
Does look to be affecting an existing feature of 19.05.x, won't backport.
(given how things looked when trying to backport)
Yes, I added dependency for 20589 - so not needed for 19.05 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20589 [Bug 20589] Add field boosting and use elastic query fields parameter instead of deprecated _all -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25864 koha-US bug tracker <bugzilla@koha-us.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC|bugzilla@koha-us.org | -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org