[Bug 18189] New: Elasticsearch sorting broken
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 Bug ID: 18189 Summary: Elasticsearch sorting broken Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching Assignee: gmcharlt@gmail.com Reporter: nick@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org Current setting in elasticsearch mapping use 'text' type for sort fields, this seems like extra overhead. If the fields are only used for sorting we can index them as 'keyword' (exact phrase) and this has the side effect of fixing sorting -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 --- Comment #1 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 60780 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=60780&action=edit Bug 18189 - Elasticsearch sorting broken To test: 1 - Have ES enabled and biblios and authorities indexed 2 - Perform a catalog search 3 - Try to sort results by author, it will fail 4 - Perform an authority search with sorting enabled, it will fail 5 - Apply patch 6 - Reindex, deleting current indexes perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d -v 7 - Repeat 3&4, they should succeed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|gmcharlt@gmail.com |nick@bywatersolutions.com --- Comment #2 from Nick Clemens <nick@bywatersolutions.com> --- Note this won't fix all sorting options unless they are defined as sortable in the config. acqdate fails out of the box - set acqdate as sortable - reindex - it should now work I suppose we either don't allow altering sortability for some fields, or the dropdown must populate based on what is defined as sortable. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@bugs.koha-c | |ommunity.org Depends on| |17255 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17255 [Bug 17255] Upgrade Elastic Search code to work with version 5.1 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 Srdjan Jankovic <srdjan@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |srdjan@catalyst.net.nz --- Comment #3 from Srdjan Jankovic <srdjan@catalyst.net.nz> --- I'm getting: Fielddata is disabled on text fields by default. Set fielddata=true on [author.phrase] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Is something wrong with my es config? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 --- Comment #4 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Srdjan Jankovic from comment #3)
I'm getting: Fielddata is disabled on text fields by default. Set fielddata=true on [author.phrase] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory.
Is something wrong with my es config?
Is this before or after the patch? I cannot seem to replicate the issue. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 --- Comment #5 from Srdjan Jankovic <srdjan@catalyst.net.nz> --- (In reply to Nick Clemens from comment #4)
(In reply to Srdjan Jankovic from comment #3)
I'm getting: Fielddata is disabled on text fields by default. Set fielddata=true on [author.phrase] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory.
Is something wrong with my es config?
Is this before or after the patch? I cannot seem to replicate the issue.
Both. What was the error that you were trying to get rid of? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 --- Comment #6 from Nick Clemens <nick@bywatersolutions.com> --- Before this patch passing sort options to Elastic gives an error much like the one you mentioned. If you are getting this before and after the patch I would suspect it is an issue with modules/versions, and or outdated mappings. Using admin/searchengine/elasticsearch/mappings.pl?op=reset&i_know_what_i_am_doing=1 should get your mappings up to date, then delete and rebuild indexes, and you should be good I would also check versions against kohadevbox elastic setup: https://github.com/digibib/kohadevbox/blob/master/roles/kohadevbox/tasks/ela... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mtj@kohaaloha.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 --- Comment #7 from Srdjan Jankovic <srdjan@catalyst.net.nz> --- Hm, done the reset thing, and now getting: No mapping found for [author__sort.phrase] in order to sort on, with: {"index":"koha_biblios","index_uuid":"fehRdZHCRlGe1FxHYiKbMA"} -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 Srdjan Jankovic <srdjan@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 Srdjan Jankovic <srdjan@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #60780|0 |1 is obsolete| | --- Comment #8 from Srdjan Jankovic <srdjan@catalyst.net.nz> --- Created attachment 61494 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=61494&action=edit [SIGNED-OFF] Bug 18189 - Elasticsearch sorting broken To test: 1 - Have ES enabled and biblios and authorities indexed 2 - Perform a catalog search 3 - Try to sort results by author, it will fail 4 - Perform an authority search with sorting enabled, it will fail 5 - Apply patch 6 - Reindex, deleting current indexes perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d -v 7 - Repeat 3&4, they should succeed Signed-off-by: Srdjan <srdjan@catalyst.net.nz> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 --- Comment #9 from Srdjan Jankovic <srdjan@catalyst.net.nz> --- Sorry, my fault, omitted -d when reindexing. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 J Schmidt <jschmidt@switchinc.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jschmidt@switchinc.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 --- Comment #10 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- What about integers and dates? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 --- Comment #11 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Does only work with author, our other fields are not marked as sortable by default (?!?) Moreover "Ó Cadhain, Máirtín" is sorted after "Yerby, Frank Garvin", terrible... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #61494|0 |1 is obsolete| | --- Comment #12 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 61689 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=61689&action=edit Bug 18189 - Elasticsearch sorting broken To test: 1 - Have ES enabled and biblios and authorities indexed 2 - Perform a catalog search 3 - Try to sort results by author, it will fail 4 - Perform an authority search with sorting enabled, it will fail 5 - Apply patch 6 - Reindex, deleting current indexes perl /home/vagrant/kohaclone/misc/search_tools/rebuild_elastic_search.pl -d -v 7 - Repeat 3&4, they should succeed Signed-off-by: Srdjan <srdjan@catalyst.net.nz> Signed-off-by: Jonathan Druart <jonathan.druart@bugs.koha-community.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master CC| |kyle@bywatersolutions.com --- Comment #13 from Kyle M Hall <kyle@bywatersolutions.com> --- Pushed to master for 17.05, thanks Nick! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de --- Comment #14 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Is enh correct here or should this not be a bug? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |normal --- Comment #15 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Sort is broken, it's a bug. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 --- Comment #16 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Ok, missed the dependency on bug 17255 earlier - should we still push this to stable? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=18189 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|Pushed to Master |RESOLVED --- Comment #17 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- 3 months - still no answer here. Closing. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org