[Bug 31023] New: Cannot create new GENRE/FORM authorities when QueryRegexEscapeOptions set to 'Unescape escaped'
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 Bug ID: 31023 Summary: Cannot create new GENRE/FORM authorities when QueryRegexEscapeOptions set to 'Unescape escaped' Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: nick@bywatersolutions.com Bug 24286 and bug 24264 added some escaping for the forward slash in this authtype, however, when we unescape escaped characters for regex, it appears we break the escaping in FindDuplicateAuthority To test: 0 - Have Koha using Elasticsearch 1 - Set QueryRegexEscapeOptions to 'unescape escaped' 2 - Attempt to add a new 'GENRE/FORM' authority record 3 - On save you get a 500 error Unable to understand your search query, please rephrase and try again. at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/Search.pm line 102 [2022/06/23 10:07:02] [WARN] [Request] ** [http://es:9200]-[400] [query_shard_exception] Failed to parse query [(authtype:GENRE/FORM* AND heading:"ddddd")], with: {"index":"koha_kohadev_authorities","index_uuid":"I00T6jgPTKynx5kKc8heNQ"}, called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/Search.pm line 95. With vars: {'request' => {'body' => {'query' => {'query_string' => {'query' => '(authtype:GENRE/FORM* AND heading:"ddddd")','analyze_wildcard' => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' ),'default_operator' => 'AND','fuzziness' => 'auto','type' => 'cross_fields','lenient' => $VAR1->{'request'}{'body'}{'query'}{'query_string'}{'analyze_wildcard'},'fields' => ['lc-card-number','local-number','record-source','personal-name','meeting-name','personal-name-see-also-from','see-from','authtype','meeting-name-see-also-from','match-heading-see-from','heading-use-main-or-added-entry','subject-heading-thesaurus','match','heading-use-series-added-entry','kind-of-record','see-also-from','heading','corporate-name-see-also-from','descriptive-cataloging-rules','meeting-name-see-from','corporate-name-see-from','heading-use-subject-added-entry','personal-name-see-from','personal-name-heading','meeting-name-heading','heading-main']}},'size' => 1,'aggregations' => {'su-geo' => {'terms' => {'size' => '20','field' => 'su-geo__facet'}},'location' => {'terms' => {'size' => '20','field' => 'location__facet'}},'author' => {'terms' => {'field' => 'author__facet','size' => '20'}},'ln' => {'terms' => {'field' => 'ln__facet','size' => '20'}},'holdingbranch' => {'terms' => {'size' => '20','field' => 'holdingbranch__facet'}},'subject' => {'terms' => {'field' => 'subject__facet','size' => '20'}},'itype' => {'terms' => {'field' => 'itype__facet','size' => '20'}},'title-series' => {'terms' => {'field' => 'title-series__facet','size' => '20'}},'ccode' => {'terms' => {'field' => 'ccode__facet','size' => '20'}}},'from' => 0},'mime_type' => 'application/json','serialize' => 'std','method' => 'GET','ignore' => [],'path' => '/koha_kohadev_authorities/_search','qs' => {'track_total_hits' => 'true'}},'status_code' => 400,'body' => {'error' => {'reason' => 'all shards failed','phase' => 'query','type' => 'search_phase_execution_exception','grouped' => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' ),'failed_shards' => [{'node' => '8aathbKCRVmwFPakemaNEA','index' => 'koha_kohadev_authorities','reason' => {'reason' => 'Failed to parse query [(authtype:GENRE/FORM* AND heading:"ddddd")]','index' => 'koha_kohadev_authorities','type' => 'query_shard_exception','caused_by' => {'type' => 'parse_exception','caused_by' => {'type' => 'token_mgr_error','reason' => 'Lexical error at line 1, column 43. Encountered: <EOF> after : "/FORM* AND heading:\\"ddddd\\")"'},'reason' => 'Cannot parse \'(authtype:GENRE/FORM* AND heading:"ddddd")\': Lexical error at line 1, column 43. Encountered: <EOF> after : "/FORM* AND heading:\\"ddddd\\")"'},'index_uuid' => 'I00T6jgPTKynx5kKc8heNQ'},'shard' => 0}],'root_cause' => [{'index' => 'koha_kohadev_authorities','reason' => 'Failed to parse query [(authtype:GENRE/FORM* AND heading:"ddddd")]','index_uuid' => 'I00T6jgPTKynx5kKc8heNQ','type' => 'query_shard_exception'}]},'status' => 400}} -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 --- Comment #1 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 137631 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137631&action=edit Bug 31023: Unit tests -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 --- Comment #2 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 137632 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137632&action=edit Bug 31023: Quote authtypecode in FindDuplicateAuthorities To test: 0 - Have Koha using Elasticsearch 1 - Set QueryRegexEscapeOptions to 'unescape escaped' 2 - Attempt to add a new 'GENRE/FORM' authority record 3 - On save you get a 500 error Unable to understand your search query, please rephrase and try again. at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/Search.pm line 102 4 - Apply patch, restart all 5 - Attempt to add a new 'GENRE/FORM' authority record 6 - Success! -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #137631|0 |1 is obsolete| | --- Comment #3 from David Nind <david@davidnind.com> --- Created attachment 137662 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137662&action=edit Bug 31023: Unit tests Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #137632|0 |1 is obsolete| | --- Comment #4 from David Nind <david@davidnind.com> --- Created attachment 137663 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137663&action=edit Bug 31023: Quote authtypecode in FindDuplicateAuthorities To test: 0 - Have Koha using Elasticsearch 1 - Set QueryRegexEscapeOptions to 'unescape escaped' 2 - Attempt to add a new 'GENRE/FORM' authority record 3 - On save you get a 500 error Unable to understand your search query, please rephrase and try again. at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/Search.pm line 102 4 - Apply patch, restart all 5 - Attempt to add a new 'GENRE/FORM' authority record 6 - Success! Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | --- Comment #5 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Nick, please don't forget to assign yourself! -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #137662|0 |1 is obsolete| | --- Comment #6 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 139852 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=139852&action=edit Bug 31023: Unit tests Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #137663|0 |1 is obsolete| | --- Comment #7 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 139853 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=139853&action=edit Bug 31023: Quote authtypecode in FindDuplicateAuthorities To test: 0 - Have Koha using Elasticsearch 1 - Set QueryRegexEscapeOptions to 'unescape escaped' 2 - Attempt to add a new 'GENRE/FORM' authority record 3 - On save you get a 500 error Unable to understand your search query, please rephrase and try again. at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/Search.pm line 102 4 - Apply patch, restart all 5 - Attempt to add a new 'GENRE/FORM' authority record 6 - Success! Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com Status|Passed QA |In Discussion --- Comment #8 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Hi, Nick. Do you think you could use Koha::SearchEngine::Zebra::QueryBuilder::clean_search_term on the authtypecode? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Passed QA --- Comment #9 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Tomás Cohen Arazi from comment #8)
Hi, Nick. Do you think you could use Koha::SearchEngine::Zebra::QueryBuilder::clean_search_term on the authtypecode?
I think that might be overkill on this one - everythign already works in Zebra - in ES, it only doesn't work under a special case where we are unescaping values as specified by the pref. Ideally, I think we would just remove the slash from the type, GENRE_FORM, or something. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |22.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 --- Comment #10 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 22.11. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com Version(s)|22.11.00 |22.11.00, 22.05.07 released in| | --- Comment #11 from Lucas Gass <lucas@bywatersolutions.com> --- Backported to 22.05.x for upcoming 22.05.07 release -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |Pushed to stable -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 Arthur Suzuki <arthur.suzuki@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |Pushed to oldstable Version(s)|22.11.00, 22.05.07 |22.11.00, 22.05.07, released in| |21.11.14 CC| |arthur.suzuki@biblibre.com --- Comment #12 from Arthur Suzuki <arthur.suzuki@biblibre.com> --- applied to 21.11.x for 21.11.14 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31023 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |victor@tuxayo.net Resolution|--- |FIXED Status|Pushed to oldstable |RESOLVED --- Comment #13 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Not backported to oldoldstable (21.05.x). Feel free to ask if it's needed. Nothing to document it seems, marking resolved. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org