[Bug 28610] New: Elasticsearch 7 - hits.total is now an object
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Bug ID: 28610 Summary: Elasticsearch 7 - hits.total is now an object Change sponsored?: --- Product: Koha Version: unspecified Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: kevin.carnes@ub.lu.se In Elasticsearch 7 hits.total is now an object (https://www.elastic.co/guide/en/elasticsearch/reference/7.0/breaking-changes...). The total is now in $result->{hits}->{total}->{value} instead of $result->{hits}->{total}. There is also $result->{hits}->{total}->{relation} which specifies if the value is exact or if it's a lower bound. So it's possible to say "Your search returned > 10000 results." You can always get an exact total by adding a track_total_hits parameter set to true when using the Elasticsearch search method (https://www.elastic.co/guide/en/elasticsearch/reference/7.0/breaking-changes...). -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Kevin Carnes <kevin.carnes@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |25439 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 [Bug 25439] [Omnibus] Prepare Koha to ElasticSearch 7 - ES7 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Kevin Carnes <kevin.carnes@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- Version|unspecified |master -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 --- Comment #1 from Kevin Carnes <kevin.carnes@ub.lu.se> --- Created attachment 130291 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=130291&action=edit Bug 28610: Elasticsearch 7 - hits.total is now an object In Elasticsearch 7 hits.total is now an object which is not always an exact value. You can always get an exact total by adding a track_total_hits parameter set to true when using the Elasticsearch search method To test: 1) Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Search.t 2) If you observe an error about types, apply patch for bug 25669 3) Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Search.t 4) Observe that tests with count fail 5) Apply patch 6) Observe that tests with count pass 7) Sign off Sponsored-by: Lund University Library -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Kevin Carnes <kevin.carnes@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Kevin Carnes <kevin.carnes@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #130291|0 |1 is obsolete| | --- Comment #2 from Kevin Carnes <kevin.carnes@ub.lu.se> --- Created attachment 130343 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=130343&action=edit Bug 28610: Elasticsearch 7 - hits.total is now an object In Elasticsearch 7 hits.total is now an object which is not always an exact value. You can always get an exact total by adding a track_total_hits parameter set to true when using the Elasticsearch search method To test: 1) Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Search.t 2) If you observe an error about types, apply patch for bug 25669 3) Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Search.t 4) Observe that tests with count fail 5) Apply patch 6) Observe that tests with count pass 7) Sign off Sponsored-by: Lund University Library -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #130343|0 |1 is obsolete| | --- Comment #3 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 130844 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=130844&action=edit Bug 28610: Elasticsearch 7 - hits.total is now an object In Elasticsearch 7 hits.total is now an object which is not always an exact value. You can always get an exact total by adding a track_total_hits parameter set to true when using the Elasticsearch search method To test: 1) Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Search.t 2) If you observe an error about types, apply patch for bug 25669 3) Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Search.t 4) Observe that tests with count fail 5) Apply patch 6) Observe that tests with count pass 7) Sign off Sponsored-by: Lund University Library Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |victor@tuxayo.net Status|Needs Signoff |Signed Off --- Comment #4 from Victor Grousset/tuxayo <victor@tuxayo.net> --- It work on ES 6 and ES 7 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart+koha@gmail. | |com, | |nick@bywatersolutions.com Severity|enhancement |normal Assignee|koha-bugs@lists.koha-commun |kevin.carnes@ub.lu.se |ity.org | --- Comment #5 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- + if (version->parse($self->{es_version}) >= version->parse('7.0.0')) { It's going to be mess very quickly if we are starting to add conditions on the version. Cannot we avoid that? -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 --- Comment #6 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- I've removed the condition, and the tests are passing under ES6. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #7 from Fridolin Somers <fridolin.somers@biblibre.com> --- (In reply to Jonathan Druart from comment #5)
+ if (version->parse($self->{es_version}) >= version->parse('7.0.0')) {
It's going to be mess very quickly if we are starting to add conditions on the version. Cannot we avoid that? +1
This may impact a lot search performance. Or we should add a cache on parsed ES version. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Kevin Carnes <kevin.carnes@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #130844|0 |1 is obsolete| | --- Comment #8 from Kevin Carnes <kevin.carnes@ub.lu.se> --- Created attachment 131008 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131008&action=edit Bug 28610: Elasticsearch 7 - hits.total is now an object In Elasticsearch 7 hits.total is now an object which is not always an exact value. You can always get an exact total by adding a track_total_hits parameter set to true when using the Elasticsearch search method To test: 1) Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Search.t 2) If you observe an error about types, apply patch for bug 25669 3) Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Search.t 4) Observe that tests with count fail 5) Apply patch 6) Observe that tests with count pass 7) Sign off Sponsored-by: Lund University Library -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Kevin Carnes <kevin.carnes@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Needs Signoff --- Comment #9 from Kevin Carnes <kevin.carnes@ub.lu.se> --- (In reply to Jonathan Druart from comment #6)
I've removed the condition, and the tests are passing under ES6.
Great, ES6 supports track_total_hits. That makes it much easier. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 --- Comment #10 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to Kevin Carnes from comment #9)
(In reply to Jonathan Druart from comment #6)
I've removed the condition, and the tests are passing under ES6.
Great, ES6 supports track_total_hits. That makes it much easier.
Not sure someone told you yet, but great to see these updates for Elasticsearch 7 going on - thx Kevin! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |julian.maurice@biblibre.com --- Comment #11 from Julian Maurice <julian.maurice@biblibre.com> --- Shouldn't 'track_total_hits' option be set in the 'count' subroutine too ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Kevin Carnes <kevin.carnes@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #131008|0 |1 is obsolete| | --- Comment #12 from Kevin Carnes <kevin.carnes@ub.lu.se> --- Created attachment 131436 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131436&action=edit Bug 28610: Elasticsearch 7 - hits.total is now an object In Elasticsearch 7 hits.total is now an object which is not always an exact value. You can always get an exact total by adding a track_total_hits parameter set to true when using the Elasticsearch search method To test: 1) Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Search.t 2) If you observe an error about types, apply patch for bug 25669 3) Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Search.t 4) Observe that tests with count fail 5) Apply patch 6) Observe that tests with count pass 7) Sign off Sponsored-by: Lund University Library -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 --- Comment #13 from Kevin Carnes <kevin.carnes@ub.lu.se> --- (In reply to Julian Maurice from comment #11)
Shouldn't 'track_total_hits' option be set in the 'count' subroutine too ?
Yes, thank you for catching that. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 --- Comment #14 from Julian Maurice <julian.maurice@biblibre.com> --- Tested with bug 25669 applied. It works great with ES 6 and ES 7. It looks like the version of Search::Elasticsearch (6.x or 7.x) doesn't matter too much. I tried both and everything went well. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #131436|0 |1 is obsolete| | --- Comment #15 from Julian Maurice <julian.maurice@biblibre.com> --- Created attachment 131542 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131542&action=edit Bug 28610: Elasticsearch 7 - hits.total is now an object In Elasticsearch 7 hits.total is now an object which is not always an exact value. You can always get an exact total by adding a track_total_hits parameter set to true when using the Elasticsearch search method To test: 1) Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Search.t 2) If you observe an error about types, apply patch for bug 25669 3) Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Search.t 4) Observe that tests with count fail 5) Apply patch 6) Observe that tests with count pass 7) Sign off Sponsored-by: Lund University Library Signed-off-by: Julian Maurice <julian.maurice@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|normal |critical -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 --- Comment #16 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- -requires 'Search::Elasticsearch', '5.01'; +requires 'Search::Elasticsearch', '6.00'; Why this change in this patch? We certainly should upgrade our minimum version, but it should be done on its own bug report. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 --- Comment #17 from Kevin Carnes <kevin.carnes@ub.lu.se> --- (In reply to Jonathan Druart from comment #16)
-requires 'Search::Elasticsearch', '5.01'; +requires 'Search::Elasticsearch', '6.00';
Why this change in this patch? We certainly should upgrade our minimum version, but it should be done on its own bug report.
6.00 is the minimum version that has track_total_hits in its API for search (look for track_total_hits in https://metacpan.org/release/DRTECH/Search-Elasticsearch-6.00/source/lib/Sea... and https://metacpan.org/release/DRTECH/Search-Elasticsearch-5.02/source/lib/Sea...) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |release-notes-needed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 --- Comment #18 from Fridolin Somers <fridolin.somers@biblibre.com> --- As Release Manager, I'm OK to change min version here. As long as we tell it in release notes. It is already ES 6 in https://wiki.koha-community.org/wiki/System_requirements_and_recommendations -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This is one of the changes release notes| |to have Koha compatible | |with ElasticSearch 7. This | |one also causes the full | |end of compatibility with | |ElasticSearch 5. Users are | |advised to upgrade as soon | |as possible to | |ElasticSearch 7 since | |version 5 and 6 are not | |supported anymore by their | |developers. --- Comment #19 from Victor Grousset/tuxayo <victor@tuxayo.net> --- It has already been dropped officially with Bug 27252 But actually with syspref ElasticsearchCrossFields off, ES 5 still worked [1] so this time will be the real end of support of ES 5. Hopefully not much instances use it as it's EOL since 2019-03-11 https://www.elastic.co/support/eol I gave a go at release notes. Tell me if info is missing or if it's not the right writing style. [1] https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27770#c9 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 --- Comment #20 from Fridolin Somers <fridolin.somers@biblibre.com> --- Release notes looks great. Thanks a lot. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|release-notes-needed | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 --- Comment #21 from Julian Maurice <julian.maurice@biblibre.com> --- Just tested this with Elasticsearch 8.1.1 (with bug 25669) and it seems to work well :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #131542|0 |1 is obsolete| | --- Comment #22 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 132389 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=132389&action=edit Bug 28610: Elasticsearch 7 - hits.total is now an object In Elasticsearch 7 hits.total is now an object which is not always an exact value. You can always get an exact total by adding a track_total_hits parameter set to true when using the Elasticsearch search method To test: 1) Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Search.t 2) If you observe an error about types, apply patch for bug 25669 3) Run prove t/db_dependent/Koha/SearchEngine/Elasticsearch/Search.t 4) Observe that tests with count fail 5) Apply patch 6) Observe that tests with count pass 7) Sign off Sponsored-by: Lund University Library Signed-off-by: Julian Maurice <julian.maurice@biblibre.com> Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |22.05.00 released in| | Status|Passed QA |Pushed to master --- Comment #23 from Fridolin Somers <fridolin.somers@biblibre.com> --- Pushed to master for 22.05, thanks to everybody involved 🦄 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|22.05.00 |22.05.00,21.11.05 released in| | CC| |kyle@bywatersolutions.com Status|Pushed to master |Pushed to stable --- Comment #24 from Kyle M Hall <kyle@bywatersolutions.com> --- Pushed to 21.11.x for 21.11.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mtj@kohaaloha.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |RESOLVED Resolution|--- |FIXED --- Comment #25 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Backported: Pushed to 21.05.x branch for 21.05.17 Nothing to document (not yet full support for ES 7), marking resolved. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|22.05.00,21.11.05 |22.05.00,21.11.05,21.05.17 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |REOPENED Resolution|FIXED |--- --- Comment #26 from Mason James <mtj@kohaaloha.com> --- (In reply to Victor Grousset/tuxayo from comment #25)
Backported: Pushed to 21.05.x branch for 21.05.17 Nothing to document (not yet full support for ES 7), marking resolved.
hi Victor can you push this for ES7 support, (i think it was missed) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|REOPENED |RESOLVED --- Comment #27 from Victor Grousset/tuxayo <victor@tuxayo.net> ---
(i think it was missed)
Indeed! :o Thanks for catching that. Pushed now. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org