[Bug 40340] New: Backslashes next to a subfield ($) breaks staging
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 Bug ID: 40340 Summary: Backslashes next to a subfield ($) breaks staging Change sponsored?: --- Product: Koha Version: unspecified Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Cataloging Assignee: koha-bugs@lists.koha-community.org Reporter: enica@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org CC: m.de.rooy@rijksmuseum.nl Created attachment 183951 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183951&action=edit Test MARC file If you have a backslash next to a subfield in the data, the staging will fail if you are using the tag as a record matching rule. Most commonly, matching on 245$a (title or title-cover). For instance, in the attached file, there is a backslash right before $h =245 00$aKnights of the black earth \$h[Book] If you have 245 as matching rule, the import batch will fail. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 Enica Davis <enica@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |minor -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 Enica Davis <enica@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Backslashes next to a |Backslashes next to a |subfield ($) breaks staging |subfield and double-quotes | |can break stage_file.pl -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 --- Comment #1 from Enica Davis <enica@bywatersolutions.com> --- Additionally, if you include double quotes in 245$a, stage_file.pl will time out. See the test MARC file #2 in the attachment. Furthermore, if the tag you are matching on contains something like {dollar}, which is commonly found in 020$c, stage_file will time out. The error: [query_shard_exception] Failed to parse query [(title:""---And Ladies of the Club" /")], with: {"index":"koha_utstatelibdivtest_biblios","index_uuid":"_AD3tsfnTpC9t4mncAArSw"}, called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at /usr/share/koha/lib/Koha/SearchEngine/Elasticsearch/Search.pm line 101. With vars: {'body' => {'error' => {'phase' => 'query','grouped' => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' ),'failed_shards' => [{'node' => 'Q3tYO6wsTXKbYv20ptCAKw','index' => 'koha_utstatelibdivtest_biblios','shard' => 0,'reason' => {'reason' => 'Failed to parse query [(title:""---And Ladies of the Club" /")]','index_uuid' => '_AD3tsfnTpC9t4mncAArSw','type' => 'query_shard_exception','caused_by' => {'caused_by' => {'reason' => 'Encountered " "-" "- "" at line 1, column 10. Was expecting one of: <BAREOPER> ... "(" ... "*" ... <QUOTED> ... <TERM> ... <PREFIXTERM> ... <WILDTERM> ... <REGEXPTERM> ... "[" ... "{" ... <NUMBER> ... <TERM> ... "*" ... ','type' => 'parse_exception'},'reason' => 'Cannot parse \'(title:""---And Ladies of the Club" /")\': Encountered " "-" "- "" at line 1, column 10. Was expecting one of: <BAREOPER> ... "(" ... "*" ... <QUOTED> ... <TERM> ... <PREFIXTERM> ... <WILDTERM> ... <REGEXPTERM> ... "[" ... "{" ... <NUMBER> ... <TERM> ... "*" ... -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 --- Comment #2 from Enica Davis <enica@bywatersolutions.com> --- Created attachment 193147 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=193147&action=edit Test MARC file #2 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com --- Comment #3 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- In Matcher.pm ( _get_match_keys ) can the trailing backslash just be removed? my $norms = $component->{'norms'}; my $key = $string; $key =~ {some regex to strip the blackslash} -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 Lisette Scheer <lisette@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lisette@bywatersolutions.co | |m --- Comment #4 from Lisette Scheer <lisette@bywatersolutions.com> --- (In reply to Lucas Gass (lukeg) from comment #3)
In Matcher.pm ( _get_match_keys ) can the trailing backslash just be removed?
my $norms = $component->{'norms'}; my $key = $string; $key =~ {some regex to strip the blackslash}
In the log, it's specifically having trouble with the ---- so that normalization won't fix everything. ','type' => 'parse_exception'},'reason' => 'Cannot parse \'(title:""---And Ladies of the Club" /")\': Encountered " "-" "- "" at line 1, column 10. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Trivial patch Status|NEW |Needs Signoff Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 --- Comment #5 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Created attachment 196549 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=196549&action=edit Bug 40340: Strip trailing backslashes from searches for ES This patch adds trailing backslashes to clean_search_terms To test: 1 - Have Koha using ES 2 - Search for: title:Knights of the black earth \ 3 - Search error 4 - Search for: title:"Knights of the black earth \" 5 - Search error 6 - Apply patch, restart_all 7 - Repeat searches 8 - No errors! 9 - Download 'Test MARC file' on this bug 10 - Stage and import with no matchign or checking for items 11 - Go to Administration -> Record matching rules 12 - Create a new rule 'Title' / threshold: 1000 / search index: title / score: 1000 / tag: 245 / subfield: a 13 - Stage the file again, use the matchign rule above 14 - Confirm file stages and matches 15 - Import and confirm no issues -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 --- Comment #6 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- This one is ready to test - perhaps a second bug for the second title with quotes and dashes, just to keep things discrete? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 --- Comment #7 from Enica Davis <enica@bywatersolutions.com> --- Created attachment 196589 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=196589&action=edit Test#2.mrc -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #8 from David Nind <david@davidnind.com> --- For steps 2 to 5 (no patch applied, no record imported), I don't get any errors when searching (using either a normal search in the staff interface or using advanced search with 'Title' selected for the keyword). I just get "No results found". -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 --- Comment #9 from Lisette Scheer <lisette@bywatersolutions.com> --- (In reply to David Nind from comment #8)
For steps 2 to 5 (no patch applied, no record imported), I don't get any errors when searching (using either a normal search in the staff interface or using advanced search with 'Title' selected for the keyword).
I just get "No results found".
David do you know if you were testing in es7 or es8 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 --- Comment #10 from Lisette Scheer <lisette@bywatersolutions.com> --- I also didn't get an error when searching for the title. I'm guessing the record needs to already be in the catalog for that to happen? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 --- Comment #11 from Lisette Scheer <lisette@bywatersolutions.com> --- Also even without matching, the record failed to import on my ktd after the patch was applied. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 Lisette Scheer <lisette@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |42533 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42533 [Bug 42533] Double quotes can break stage_file.pl -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 Lisette Scheer <lisette@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40340 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #12 from David Cook <dcook@prosentient.com.au> --- To me it sounds like this just isn't being turned into JSON properly. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org