[Bug 28326] New: If ElasticSearch enable is not possible to edit or save records with ' ES reserved charset' like []
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 Bug ID: 28326 Summary: If ElasticSearch enable is not possible to edit or save records with ' ES reserved charset' like [] Change sponsored?: --- Product: Koha Version: unspecified Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: hagud@orex.es With ElasticSearch enabled it is not possible to create or edit any record with special charset in the title like [] [2021/05/12 06:29:49] [WARN] [Request] ** [hxxxxxx]-[400] [query_shard_exception] Failed to parse query [(host-item:[Romanza appassionata])], with: {"index_uuid":"VlZbj1MfQtWjWgdlnkSUXA","index":"koha_cxxx_biblios"}, called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at /usr/share/koha/lib/Koha/SearchEngine/Elasticsearch/Search.pm line 96. With vars: {'status_code' => 400,'body' => {'error' => {'grouped' => bless( do{\(my $o = 1)}, 'JSON::PP::Boolean' ),'root_cause' => [{'reason' => 'Failed to parse query [(host-item:[Romanza appassionata])]','index_uuid' => 'VlZbj1MfQtWjWgdlnkSUXA','index' => 'koha_xxxx_biblios','type' => 'query_shard_exception'}],'reason' => 'all shards failed','failed_shards' => [{'reason' => {'index_uuid' => 'VlZbj1MfQtWjWgdlnkSUXA','caused_by' => {'type' => 'parse_exception','caused_by' => {'type' => 'parse_exception','reason' => 'Encountered " <RANGE_GOOP> "appassionata "" at line 1, column 20. Was expecting: "TO" ... '},'reason' => 'Cannot parse \'(host-item:[Romanza appassionata])\': Encountered " <RANGE_GOOP> "appassionata "" at line 1, column 20. Was expecting: "TO" ... -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 Lucy Vaux-Harvey <lucy.vaux-harvey@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucy.vaux-harvey@ptfs-europ | |e.com --- Comment #1 from Lucy Vaux-Harvey <lucy.vaux-harvey@ptfs-europe.com> --- This is causing problems for one of our libraries, it would be great to see a fix for this. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |26313 CC| |nick@bywatersolutions.com Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26313 [Bug 26313] "Show analytics" and "Show volumes" links don't work with Elasticsearch and UseControlNumber -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com Severity|enhancement |major Assignee|koha-bugs@lists.koha-commun |tomascohen@gmail.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|tomascohen@gmail.com |nick@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 --- Comment #2 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 120971 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=120971&action=edit Bug 28326: Escape special characters in title when searching for analytics Bug 26313 introduced a search for analytics, which is great, but it started sending titles to ES. This, unfrotunately, caused problems where the title contained reserved ES characters. As these searches should always be literl title searches, we should escape all the ES characters before the search. We should not make this a standard, however, as then it prevents using the characters for advanced ES searching To test: 1 - Have ES setup and running and swtich SearchEngine syspref to 'Elasticsearch' 2 - Edit a record and add some reserved ES characters to the title https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-qu... e.g. De [Crasi] Aristophea. 3 - You get an error on the detail page loading 4 - Apply patch 5 - Reload 6 - The page successfully loads -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=28316 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #120971|0 |1 is obsolete| | --- Comment #3 from David Nind <david@davidnind.com> --- Created attachment 121000 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=121000&action=edit Bug 28326: Escape special characters in title when searching for analytics Bug 26313 introduced a search for analytics, which is great, but it started sending titles to ES. This, unfrotunately, caused problems where the title contained reserved ES characters. As these searches should always be literl title searches, we should escape all the ES characters before the search. We should not make this a standard, however, as then it prevents using the characters for advanced ES searching To test: 1 - Have ES setup and running and swtich SearchEngine syspref to 'Elasticsearch' 2 - Edit a record and add some reserved ES characters to the title https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-qu... e.g. De [Crasi] Aristophea. 3 - You get an error on the detail page loading 4 - Apply patch 5 - Reload 6 - The page successfully loads Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #121000|0 |1 is obsolete| | --- Comment #4 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 121035 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=121035&action=edit Bug 28326: Escape special characters in title when searching for analytics Bug 26313 introduced a search for analytics, which is great, but it started sending titles to ES. This, unfrotunately, caused problems where the title contained reserved ES characters. As these searches should always be literl title searches, we should escape all the ES characters before the search. We should not make this a standard, however, as then it prevents using the characters for advanced ES searching To test: 1 - Have ES setup and running and swtich SearchEngine syspref to 'Elasticsearch' 2 - Edit a record and add some reserved ES characters to the title https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-qu... e.g. De [Crasi] Aristophea. 3 - You get an error on the detail page loading 4 - Apply patch 5 - Reload 6 - The page successfully loads Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@ptfs-europe | |.com Status|Signed Off |Passed QA --- Comment #5 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- This gets us out of a bind, and works perfectly. We should probably, in the future, move this into the QueryBuilder module, but I think that needs more thought and can be saved for the future. Passing QA. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart+koha@gmail. | |com Status|Passed QA |Failed QA --- Comment #6 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- We need this code moved to a method and covered by tests, for re-usability. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #121035|0 |1 is obsolete| | --- Comment #7 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 121436 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=121436&action=edit Bug 28326: Add escape_query function to deal with reserved characters Bug 26313 introduced a search for analytics, which is great, but it started sending titles to ES. This, unfortunately, caused problems where the title contained reserved ES characters. As these searches should always be literal title searches, we should escape all the ES characters before the search. We should not make this a standard, however, as then it prevents using the characters for advanced ES searching, but we still add a function for future reuse To test: 1 - Have ES setup and running and swtich SearchEngine syspref to 'Elasticsearch' 2 - Edit a record and add some reserved ES characters to the title https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-qu... e.g. De [Crasi] Aristophea. 3 - You get an error on the detail page loading 4 - Apply patch 5 - Reload 6 - The page successfully loads 7 - prove -v t/db_dependent/Koha/SearchEngine/Search.t -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA CC| |victor@tuxayo.net --- Comment #8 from Victor Grousset/tuxayo <victor@tuxayo.net> --- The following name cause the crash despite the patch tag:{alpha TO omega} -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 --- Comment #9 from Victor Grousset/tuxayo <victor@tuxayo.net> --- More tests. The following name cause the crash despite the patch. == Tested strings, work == quikc~ brwn~ foks~ name:/joh?n(ath[oa]n)/ count:[10 TO *] age:(+>=10 +<20) quick^2 fox The reserved characters are: + - = && || > < ! ( ) { } [ ] ^ " ~ * ? : \ / == Crash == "fox quick"~5 tag:{alpha TO omega} Nmap network scanning : <> >< -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 Janet McGowan <janet.mcgowan@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |janet.mcgowan@ptfs-europe.c | |om -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |RESOLVED Resolution|--- |DUPLICATE --- Comment #10 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- I think this is now superseded by bug 28316 *** This bug has been marked as a duplicate of bug 28316 *** -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 --- Comment #11 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- *** This bug has been marked as a duplicate of bug 28484 *** -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28326 --- Comment #12 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Sorry.. bug 28484 which uses the work from bug 28316 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org