[Bug 18318] New: Unicode support for Elasticsearch
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Bug ID: 18318 Summary: Unicode support for Elasticsearch Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching Assignee: gmcharlt@gmail.com Reporter: nick@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org We need to offer unicode support in Elasticsearch https://www.elastic.co/guide/en/elasticsearch/guide/current/case-folding.htm... https://www.elastic.co/guide/en/elasticsearch/guide/current/icu-plugin.html Currently a search for: pokemon Will not return results for: pokémon That is just an example :-) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |claire_gravely@hotmail.com, | |fridolin.somers@biblibre.co | |m, | |jonathan.druart@bugs.koha-c | |ommunity.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 --- Comment #1 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 61982 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=61982&action=edit Bug 18318 - Unicode support for Elasticsearch You must install the icu plugin for elasticsearch https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu.h... Once installed, apply this patch Reindex your data, deleting the existing indexes perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d Find (or add) some titles with accented characters Verify that a search for the exact character or the unaccented version works -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|gmcharlt@gmail.com |nick@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Séverine Queune <severine.queune@bulac.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |severine.queune@bulac.fr -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Blou <philippe.blouin@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |philippe.blouin@inlibro.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |major Summary|Unicode support for |Elasticsearch: Cannot |Elasticsearch |search non-ASCII strings -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Elasticsearch: Cannot |Unicode support for |search non-ASCII strings |Elasticsearch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Component|Searching |Searching - Elasticsearch QA Contact|testopia@bugs.koha-communit | |y.org | Summary|Unicode support for |Wrong unicode tokenization |Elasticsearch | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 --- Comment #2 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Just a note, the ICU analysis plugin has been incorporated into KohaDevBox so in order to test this patch you can provision a new box as usual: KOHA_ELASTICSEARCH=1 vagrant up Regards -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #61982|0 |1 is obsolete| | --- Comment #3 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 67344 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=67344&action=edit Bug 18318: Unicode support for Elasticsearch You must install the icu plugin for elasticsearch https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu.h... Once installed, apply this patch Reindex your data, deleting the existing indexes perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d Find (or add) some titles with accented characters Verify that a search for the exact character or the unaccented version works Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |julian.maurice@biblibre.com QA Contact|tomascohen@gmail.com |julian.maurice@biblibre.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Failed QA --- Comment #4 from Julian Maurice <julian.maurice@biblibre.com> --- The patch does what it says, but it adds a dependency (ICU plugin) without mentioning it anywhere (AFAIK). I don't know what is the best place for this (https://wiki.koha-community.org/wiki/Elasticsearch maybe ?), but it must be written somewhere. Marking as Failed QA for now. I will pass QA as soon as there is some kind of documentation for this. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Signed Off --- Comment #5 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to Julian Maurice from comment #4)
The patch does what it says, but it adds a dependency (ICU plugin) without mentioning it anywhere (AFAIK). I don't know what is the best place for this (https://wiki.koha-community.org/wiki/Elasticsearch maybe ?), but it must be written somewhere.
Marking as Failed QA for now. I will pass QA as soon as there is some kind of documentation for this.
The commit message mentions itbut definitely worth documenting in a more publicly available place. I've adjusted the docs (https://wiki.koha-community.org/wiki/Elasticsearch). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 --- Comment #6 from Julian Maurice <julian.maurice@biblibre.com> --- (In reply to Tomás Cohen Arazi from comment #5)
I've adjusted the docs (https://wiki.koha-community.org/wiki/Elasticsearch). Thanks!
-- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #67344|0 |1 is obsolete| | --- Comment #7 from Julian Maurice <julian.maurice@biblibre.com> --- Created attachment 67449 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=67449&action=edit Bug 18318: Unicode support for Elasticsearch You must install the icu plugin for elasticsearch https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu.h... Once installed, apply this patch Reindex your data, deleting the existing indexes perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d Find (or add) some titles with accented characters Verify that a search for the exact character or the unaccented version works Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Julian Maurice <julian.maurice@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master --- Comment #8 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Pushed to master for 17.11, thanks to everybody involved! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 --- Comment #9 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- ICU support, awesome :D, thanks all. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |Pushed to Stable --- Comment #10 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- Pushed to 17.05.x, will be in 17.05.05. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18318 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de Resolution|--- |FIXED Status|Pushed to Stable |RESOLVED --- Comment #11 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Patch does not apply cleanly. Closing as 16.11.x is missing quite some patches for Elasticsearch now. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org