[Bug 28484] New: Elasticsearch fails to parse query if colon or exclamation point is in 245$a
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Bug ID: 28484 Summary: Elasticsearch fails to parse query if colon or exclamation point is in 245$a Change sponsored?: --- Product: Koha Version: 20.11 Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Cataloging Assignee: koha-bugs@lists.koha-community.org Reporter: b.pastern4k@gmail.com QA Contact: testopia@bugs.koha-community.org CC: m.de.rooy@rijksmuseum.nl We have Elasticsearch 6.8.15 and the following error happens when trying to enter colon or exclamation point into 245$a as the last symbol [query_shard_exception] Failed to parse query [(host-item:osteuropa:)], with: {"index_uuid": ....} called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at /usr/share/koha/lib/Koha/SearchEngine/Elasticsearch/Search.pm line 96 'Failed to parse query [(host-item:osteuropa:)]'}],'failed_shards' => [{'shard' => 0,'node' => 'node','reason' => {'type' => 'query_shard_exception','caused_by' => {'caused_by' => {'type' => 'parse_exception','reason' => 'Encountered " ":" ": "" at line 1, column 20. Error does not happen when entering either of these symbols in 245$b. Are these symbols not properly escaped in that subfield? -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |normal -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Component|Cataloging |Searching - Elasticsearch QA Contact|testopia@bugs.koha-communit | |y.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 John Yorio <jyorio@equinoxinitiative.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jyorio@equinoxinitiative.or | |g --- Comment #1 from John Yorio <jyorio@equinoxinitiative.org> --- I'm just confirming this error occurs. Staff saw Internal Server Error when trying to view a bib record. Looking in the log files, I saw the "[query_shard_exception] Failed to parse query [(host-item: ...." message as per Bohdan's original post. Editing the bib record by directly bringing it up with addbiblio.pl?biblionumber= worked and I removed the colon at the end of the 245a. The record could then be viewed. Adding the colon back caused the error again. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 --- Comment #2 from John Yorio <jyorio@equinoxinitiative.org> --- (In reply to John Yorio from comment #1)
I'm just confirming this error occurs.
Staff saw Internal Server Error when trying to view a bib record.
Looking in the log files, I saw the "[query_shard_exception] Failed to parse query [(host-item: ...." message as per Bohdan's original post.
Editing the bib record by directly bringing it up with addbiblio.pl?biblionumber= worked and I removed the colon at the end of the 245a. The record could then be viewed. Adding the colon back caused the error again.
I did some further testing. In my testing of a 245$a ending in a colon, if the colon is preceded by a space, the error does not occur. So, something like "My Title :" works fine, but "My Title:" produces the error. Exclamation points failed regardless. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 --- Comment #3 from Alex Arnaud <alex.arnaud@biblibre.com> --- Created attachment 123045 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=123045&action=edit Bug 28484: ES clean title in biblio detail Test plan: - Create a biblio with title like "osteuropa:" or "osteuropa!" - Go the this biblio detail pages (cgi-bin/koha/catalogue/detail.pl) => Error - Apply bug 28316 and this one - test again -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |alex.arnaud@biblibre.com Depends on| |28316 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28316 [Bug 28316] Fix ES crashes related to various punctuation characters -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff --- Comment #4 from Alex Arnaud <alex.arnaud@biblibre.com> --- Bug 28316 add some escapings for specials characters like exclamation mark or colon. But this works only for searches. In biblio detail, the query is built in the script, so the cleaning doesn't apply. The proposed patch clean the biblio title before the call to simple_search_compat. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Alex Buckley <alexbuckley@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Alex Buckley <alexbuckley@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #123045|0 |1 is obsolete| | --- Comment #5 from Alex Buckley <alexbuckley@catalyst.net.nz> --- Created attachment 123607 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=123607&action=edit Bug 28484: ES clean title in biblio detail Test plan: - Create a biblio with title like "osteuropa:" or "osteuropa!" - Go the this biblio detail pages (cgi-bin/koha/catalogue/detail.pl) => Error - Apply bug 28316 and this one - test again Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz> -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Alex Buckley <alexbuckley@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |alexbuckley@catalyst.net.nz --- Comment #6 from Alex Buckley <alexbuckley@catalyst.net.nz> --- Followed test plan, patch worked as described in kohadevbox running Elasticsearch. So signed off. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #123607|0 |1 is obsolete| | --- Comment #7 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 124379 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=124379&action=edit Bug 28484: ES clean title in biblio detail Test plan: - Create a biblio with title like "osteuropa:" or "osteuropa!" - Go the this biblio detail pages (cgi-bin/koha/catalogue/detail.pl) => Error - Apply bug 28316 and this one - test again Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@ptfs-europe | |.com Status|Signed Off |Passed QA --- Comment #8 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Trivial patch after the work carried out by bug 28316.. passing QA -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |martin.renvoize@ptfs-europe | |.com -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |hagud@orex.es --- Comment #9 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- *** Bug 28326 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |11175 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11175 [Bug 11175] Show the parent record's component parts in the detailed views -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |26314 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26314 [Bug 26314] "Volumes: show volumes" showing regardless of whether there are volumes linked to the record -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=28682 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version|20.11 |unspecified CC| |jonathan.druart+koha@gmail. | |com Assignee|koha-bugs@lists.koha-commun |alex.arnaud@biblibre.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 --- Comment #10 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- I don't recreate the work on master, can you provide more details please. I don't really see the point of the new method this patch adds: sub clean_search_term { my ( $self, $term ) = @_; return $term; } ... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 --- Comment #11 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- (In reply to Jonathan Druart from comment #10)
I don't really see the point of the new method this patch adds:
sub clean_search_term { my ( $self, $term ) = @_;
return $term; }
...
Never mind, it's for Zebra. Still cannot recreate the problem however. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Joonas Kylmälä <joonas.kylmala@iki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |joonas.kylmala@iki.fi --- Comment #12 from Joonas Kylmälä <joonas.kylmala@iki.fi> --- (In reply to Jonathan Druart from comment #11)
(In reply to Jonathan Druart from comment #10)
I don't really see the point of the new method this patch adds:
sub clean_search_term { my ( $self, $term ) = @_;
return $term; }
...
Never mind, it's for Zebra.
Still cannot recreate the problem however.
hmm, seems like the dependency here, bug 28316, does the cleaning already so this seems to just call the cleaning function another time. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 --- Comment #13 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- I don't recreate *on master*: - Create a biblio with title like "osteuropa:" or "osteuropa!" - Go the this biblio detail pages (cgi-bin/koha/catalogue/detail.pl) => Error There is no error for me. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |BLOCKED --- Comment #14 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Bug 28316 is waiting for SO, can we get your attention there please? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|BLOCKED |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 --- Comment #15 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Switching back to PQA as the blocker has been pushed. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 --- Comment #16 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- (In reply to Jonathan Druart from comment #13)
I don't recreate *on master*:
- Create a biblio with title like "osteuropa:" or "osteuropa!" - Go the this biblio detail pages (cgi-bin/koha/catalogue/detail.pl) => Error
There is no error for me.
For the record, the error appears on the *OPAC* detail page. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Failed QA --- Comment #17 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- And I am still getting the error after applying this patch: Unable to understand your search query, please rephrase and try again. at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/Search.pm line 100 In logs: ','type' => 'parse_exception'},'reason' => 'Failed to parse query [(host-item:(osteuropa!))]','index_uuid' => '-iKfKTkGSwCClT_0bcKLdQ','type' => 'query_shard_exception'},'node' => 'PCWsz1DTR6ybJJ3d0wR2YA'}],'t ype' => 'search_phase_execution_exception','reason' => 'all shards failed'}}} -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 --- Comment #18 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- There is also a QA script failure FAIL Koha/SearchEngine/Zebra/QueryBuilder.pm FAIL pod *** WARNING: =head2 without preceding higher level in file Koha/SearchEngine/Zebra/QueryBuilder.pm -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #124379|0 |1 is obsolete| | --- Comment #19 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 125433 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=125433&action=edit Bug 28484: ES clean title in biblio detail Test plan: - Create a biblio with title like "osteuropa:" or "osteuropa!" - Go the this biblio detail pages (cgi-bin/koha/catalogue/detail.pl) => Error - Apply bug 28316 and this one - test again Signed-off-by: Alex Buckley <alexbuckley@catalyst.net.nz> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 --- Comment #20 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 125434 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=125434&action=edit Bug 28484: (QA follow-up) Add fix on OPAC Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 --- Comment #21 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 125435 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=125435&action=edit Bug 28484: (QA follow-up) POD fix Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Eric Phetteplace <ephetteplace@cca.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ephetteplace@cca.edu -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Elasticsearch fails to |Elasticsearch fails to |parse query if colon or |parse query if exclamation |exclamation point is in |point is in 245$a |245$a | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |21.11.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 --- Comment #22 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Pushed to master for 21.11, thanks to everybody involved! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28484 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |Pushed to stable CC| |kyle@bywatersolutions.com Version(s)|21.11.00 |21.11.00,21.05.05 released in| | --- Comment #23 from Kyle M Hall <kyle@bywatersolutions.com> --- Pushed to 21.05.x for 21.05.05 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org