[Bug 23521] New: ES 6 - limit terms with many word can make the search inaccurate
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Bug ID: 23521 Summary: ES 6 - limit terms with many word can make the search inaccurate Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: alex.arnaud@biblibre.com That issue seems to happen only with elasticsearch 6. So make this BZ depends on ES 6 Omnibus. Refining a search on author lead to the following query: "query_string": { "query": "(_record:*) AND author:Dillinger Girl" } This query returns a document with "Dillinger Escaplan" as author and "Girl" in the title => Ko The fix should be something like: "query_string": { "query": "(_record:*) AND author:(Dillinger Girl)" } That query returns only documents with "Dillinger Girl" as author => Ok -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|ES 6 - limit terms with |ES 6 - limit terms with |many word can make the |many words can make the |search inaccurate |search inaccurate -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |alex.arnaud@biblibre.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |20196 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20196 [Bug 20196] [Omnibus] Prepare Koha to ElasticSearch6 - ES6 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 --- Comment #1 from Alex Arnaud <alex.arnaud@biblibre.com> --- Created attachment 92498 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=92498&action=edit Bug 23521: Put parentheses arround limit terms Test plan: - Use Elasticsearch 6 (you'll need Bug 18969), - create a biblio (#1) with "Dillinger Girl" in author and what you want in title, - create another biblio (#2) with the word "girl" in the title and "Dillinger Escaplan" as author - reindex - search * and refine on "Dillinger Girl" - Ko => Biblio #1 and #2 appear - Apply this patch, - search * and refine on "Dillinger Girl" - Ok => anly biblio #1 appears - use Elasticsearch 5 again - check for no search regression -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Alex Arnaud <alex.arnaud@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Michal Denar <black23@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |black23@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 e.betemps <e.betemps@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |e.betemps@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #2 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- See doc : https://www.elastic.co/guide/en/elasticsearch/reference/6.8/query-dsl-query-... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 --- Comment #3 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- (In reply to Fridolin SOMERS from comment #2)
See doc :
https://www.elastic.co/guide/en/elasticsearch/reference/6.8/query-dsl-query- string-query.html
You see an example of query containing parenthesis ofr terms with many words -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 sandboxes@biblibre.com <sandboxes@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off CC| |sandboxes@biblibre.com --- Comment #4 from sandboxes@biblibre.com <sandboxes@biblibre.com> --- Patch tested with a sandbox, by Séverine QUEUNE <severine.queune@bulac.fr> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 sandboxes@biblibre.com <sandboxes@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #92498|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 --- Comment #5 from sandboxes@biblibre.com <sandboxes@biblibre.com> --- Created attachment 93741 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=93741&action=edit Bug 23521: Put parentheses arround limit terms Test plan: - Use Elasticsearch 6 (you'll need Bug 18969), - create a biblio (#1) with "Dillinger Girl" in author and what you want in title, - create another biblio (#2) with the word "girl" in the title and "Dillinger Escaplan" as author - reindex - search * and refine on "Dillinger Girl" - Ko => Biblio #1 and #2 appear - Apply this patch, - search * and refine on "Dillinger Girl" - Ok => anly biblio #1 appears - use Elasticsearch 5 again - check for no search regression Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr> Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #93741|0 |1 is obsolete| | --- Comment #6 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 94771 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=94771&action=edit Bug 23521: Put parentheses arround limit terms Test plan: - Use Elasticsearch 6 (you'll need Bug 18969), - create a biblio (#1) with "Dillinger Girl" in author and what you want in title, - create another biblio (#2) with the word "girl" in the title and "Dillinger Escaplan" as author - reindex - search * and refine on "Dillinger Girl" - Ko => Biblio #1 and #2 appear - Apply this patch, - search * and refine on "Dillinger Girl" - Ok => anly biblio #1 appears - use Elasticsearch 5 again - check for no search regression Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 --- Comment #7 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 94772 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=94772&action=edit Bug 23521: (follow-up) Add quotes Not only should terms from facets/limits be grouped, but order should probably matter We build a different facet for "Dillinger Girl" or "Girl Dillinger" The parens are possibly overkill now, but I think it makes it very clear and does not hurt To test: 1 - Follow original plan 2 - Create a new record with 'Girl Dillinger' as author 3 - Search for 'Dill*' 4 - You get all three records (and maybe others that match) 5 - Limit by 'Girl Dillinger' - you get two records 6 - Same for 'Dillinger Girl' 7 - Apply patch 8 - Limits/facets for 'Dillinger Girl' and 'Girl Dillinger' now match a isngle record -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ere.maijala@helsinki.fi --- Comment #8 from Nick Clemens <nick@bywatersolutions.com> --- Alex, your patches work well, but I took things a bit further, can you check my work? Added Ere to CC for another opinion. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Ere Maijala <ere.maijala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #94771|0 |1 is obsolete| | --- Comment #9 from Ere Maijala <ere.maijala@helsinki.fi> --- Created attachment 94799 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=94799&action=edit Bug 23521: Put parentheses arround limit terms Test plan: - Use Elasticsearch 6 (you'll need Bug 18969), - create a biblio (#1) with "Dillinger Girl" in author and what you want in title, - create another biblio (#2) with the word "girl" in the title and "Dillinger Escaplan" as author - reindex - search * and refine on "Dillinger Girl" - Ko => Biblio #1 and #2 appear - Apply this patch, - search * and refine on "Dillinger Girl" - Ok => anly biblio #1 appears - use Elasticsearch 5 again - check for no search regression Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr> Signed-off-by: Séverine QUEUNE <severine.queune@bulac.fr> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Ere Maijala <ere.maijala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #94772|0 |1 is obsolete| | --- Comment #10 from Ere Maijala <ere.maijala@helsinki.fi> --- Created attachment 94800 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=94800&action=edit Bug 23521: (follow-up) Add quotes Not only should terms from facets/limits be grouped, but order should probably matter We build a different facet for "Dillinger Girl" or "Girl Dillinger" The parens are possibly overkill now, but I think it makes it very clear and does not hurt To test: 1 - Follow original plan 2 - Create a new record with 'Girl Dillinger' as author 3 - Search for 'Dill*' 4 - You get all three records (and maybe others that match) 5 - Limit by 'Girl Dillinger' - you get two records 6 - Same for 'Dillinger Girl' 7 - Apply patch 8 - Limits/facets for 'Dillinger Girl' and 'Girl Dillinger' now match a isngle record Signed-off-by: Ere Maijala <ere.maijala@helsinki.fi> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 --- Comment #11 from Ere Maijala <ere.maijala@helsinki.fi> --- Makes sense to me. I don't particularly like the slightly ambiguous syntax used with limits (I'd interpret 'author:Something Other' as 'author:(Something) AND Other', but since it works with Zebra, we need to support that. A helper function for splitting a string to field and term would be nice since we do the same regexp now in at least two places, but it's still clear enough that it works for me. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@ptfs-europe | |.com Status|Signed Off |Passed QA --- Comment #12 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- With comments (and subsequent signoffs) from all ES experts already I'm going to go for PQA on this one. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |20.05.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 --- Comment #13 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work everyone! Pushed to master for 20.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Bug 23521 depends on bug 20196, which changed state. Bug 20196 Summary: [Omnibus] Prepare Koha to ElasticSearch6 - ES6 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=20196 What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Joy Nelson <joy@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|20.05.00 |20.05.00, 19.11.05 released in| | CC| |joy@bywatersolutions.com Status|Pushed to master |Pushed to stable --- Comment #14 from Joy Nelson <joy@bywatersolutions.com> --- Backported to 19.11.x branch for 19.11.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com --- Comment #15 from Lucas Gass <lucas@bywatersolutions.com> --- missing dependencies for 19.05.x, not backporting -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23521 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |25882 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25882 [Bug 25882] Elasticsearch - Advanced search itemtype limits are being double quoted -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org