[Bug 25439] New: [Omnibus] Prepare Koha to ElasticSearch 7 - ES7
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Bug ID: 25439 Summary: [Omnibus] Prepare Koha to ElasticSearch 7 - ES7 Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: victor@tuxayo.net ES6 will be end of life when the version 8 will come out[1]. there is no alpha of version 8 so we have at least half a year[2] before that happens. [1] https://www.elastic.co/support/eol [2] https://discuss.elastic.co/t/elasticsearch-8-release-date/202759/2 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=20196 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |22520 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22520 [Bug 22520] Be Elastic compliant 7.x and 8.x (_doc) -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |25668 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25668 [Bug 25668] ElasticSearch 6 deprecation: the default number of shards will change in version 7 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |25669 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25669 [Bug 25669] ElasticSearch 6 deprecation: [types removal] Specifying types in put mapping requests is deprecated (incompatible with 7) -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |25684 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25684 [Bug 25684] ElasticSearch 7 start crash: at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #1 from Fridolin Somers <fridolin.somers@biblibre.com> --- Elasticsearch 6 is going to end of support at the end of this year : https://www.elastic.co/fr/support/eol So we should work on compatibility with 7 (before it is also EOL ^^) -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 --- Comment #2 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Already? :o Is it because it's written 6.8.x EOL date 2020-11-20 Maintained until 8.0.0 ? There are no signs of ES 8 alpha https://github.com/elastic/elasticsearch/tags -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 --- Comment #3 from Fridolin Somers <fridolin.somers@biblibre.com> --- (In reply to Victor Grousset/tuxayo from comment #2)
Already? :o Is it because it's written 6.8.x EOL date 2020-11-20 Maintained until 8.0.0 ?
Yep -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Amit Gupta <amitddng135@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |amit.gupta@informaticsgloba | |l.com, | |amitddng135@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |27043 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27043 [Bug 27043] Add to number_of_replicas and number_of_shards to index config -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Bug 25439 depends on bug 25668, which changed state. Bug 25668 Summary: ElasticSearch 6 deprecation: the default number of shards will change in version 7 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25668 What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |DUPLICATE -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |27770 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27770 [Bug 27770] ES: Deprecated aggregation order key [_term] used, replaced by [_key]" -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |27771 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27771 [Bug 27771] ES: The parameter include_type_name should be explicitly specified in put mapping requests to prepare for 7.0. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Bug 25439 depends on bug 27043, which changed state. Bug 27043 Summary: Add to number_of_replicas and number_of_shards to index config https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27043 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to oldoldstable |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 claire.hernandez@biblibre.com <claire.hernandez@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |claire.hernandez@biblibre.c | |om -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Kevin Carnes <kevin.carnes@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |28610 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 [Bug 28610] Elasticsearch 7 - hits.total is now an object -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 --- Comment #4 from Victor Grousset/tuxayo <victor@tuxayo.net> --- ES8 is coming. So is ES6 actual EOL. Is turns out the EOL ES page is wrong. As 6.8.x still gets updates. But the 1st ES8 alpha has been published [2]. So we should expect ES6 to EOL in around 6 months [3]. https://github.com/elastic/elasticsearch/tags https://discuss.elastic.co/t/elasticsearch-8-release-date/202759/2 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 --- Comment #5 from Fridolin Somers <fridolin.somers@biblibre.com> --- FYI here is the upgrade doc : https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrad... -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 --- Comment #6 from Fridolin Somers <fridolin.somers@biblibre.com> --- Actually https://www.elastic.co/fr/support/eol says : 6.8.x EOL on 2022-02-08 If you are in 6 < 6.8, first upgrade to 6.8. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 --- Comment #7 from Victor Grousset/tuxayo <victor@tuxayo.net> ---
Actually https://www.elastic.co/fr/support/eol says : 6.8.x EOL on 2022-02-08
great, they recently fixed the page :D Before it basically said that 6.8.x was EOL. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 --- Comment #8 from Victor Grousset/tuxayo <victor@tuxayo.net> --- ES 8 is out since a week. So Koha only works with the EOLed ES 6 ^^" The very good news is that there have been multiple patches submitted to tickets. So check them out. Now with bug 25684 we can test Koha with ES 7 and see how it actually works. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Bug 25439 depends on bug 27771, which changed state. Bug 27771 Summary: ES: The parameter include_type_name should be explicitly specified in put mapping requests to prepare for 7.0. https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27771 What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |DUPLICATE -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kevin.carnes@ub.lu.se --- Comment #9 from Victor Grousset/tuxayo <victor@tuxayo.net> --- With Bug 25669, Bug 25684, Bug 27770 and Bug 28610 applied the tests pass[1] on ES7 and searching works and cataloguing indexes correctly a new or modified title. So that would be it? :D Anything else to test? Or these bugs reaching master and being backported will be enough to claim ES7 support? [1] minus some local failures I also have on master without any patch -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 --- Comment #10 from Fridolin Somers <fridolin.somers@biblibre.com> --- Bug 22520 is not a mandatory dependency ? -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 --- Comment #11 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Right, I don't understand what is it about. But it looks like something not covered by tests and basic workflow. I'll ask there. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Bug 25439 depends on bug 22520, which changed state. Bug 22520 Summary: Be Elastic compliant 7.x and 8.x (_doc) https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22520 What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |INVALID -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Bug 25439 depends on bug 27770, which changed state. Bug 27770 Summary: ES: Deprecated aggregation order key [_term] used, replaced by [_key] https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27770 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |critical -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 --- Comment #12 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Switched to critical (and the dependencies not yet backported) to be sure it's not skipped for backports. Because it's really bad to not have a supported ES implementation anymore. If something like the recent log4j critical security vulnerabilities happens again, ES7 will be the only fix. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Bug 25439 depends on bug 25684, which changed state. Bug 25684 Summary: ElasticSearch 7 start crash: at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25684 What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |RESOLVED Resolution|--- |MOVED -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mtj@kohaaloha.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Bug 25439 depends on bug 28610, which changed state. Bug 28610 Summary: Elasticsearch 7 - hits.total is now an object https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |31499 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31499 [Bug 31499] Add libhttp-tiny-perl 0.076 dependency for ES7 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Bug 25439 depends on bug 28610, which changed state. Bug 28610 Summary: Elasticsearch 7 - hits.total is now an object https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |REOPENED Resolution|FIXED |--- -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Bug 25439 depends on bug 28610, which changed state. Bug 28610 Summary: Elasticsearch 7 - hits.total is now an object https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28610 What |Removed |Added ---------------------------------------------------------------------------- Status|REOPENED |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Bug 25439 depends on bug 31499, which changed state. Bug 31499 Summary: Add libhttp-tiny-perl 0.076 dependency for ES7 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31499 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart+koha@gmail. | |com --- Comment #13 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- What's the status of this? All the bug reports linked have been pushed. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 --- Comment #14 from David Cook <dcook@prosentient.com.au> --- (In reply to Jonathan Druart from comment #13)
What's the status of this? All the bug reports linked have been pushed.
Victor, do you have an update on this one? -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=33353 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 --- Comment #15 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Hi :) There is enough real world usage to claim that Koha (down to 21.05.x) supports ES 7 - Lund University since at least 2022-02 - LMS Cloud on 150 Koha instance since 2022-06, just learned this at the hackfest. OpenSearch 1.x passes also the tests and is a drop in replacement of ES 7.x so we can be confident enough to advertise support for it also. So unless someone thinks of a problem I'll update the system requirements wikipage with that. --- Finally finished the discussion with Mason about how what to say about all this and ES 6 being EOL and the ES license change and OpenSearch fork. Is that ok if in the wikipage we provide support and license info for each Koha version like this: - ES 6.x: supported, but EOL since 10 Feb 2022 so no more update even in case of security issues - ES 7.x: supported, some phrase to tell that it's not really libre/open source anymore (SSPL license) and it's complicated blablabla - OS 1.x: supported, drop in replacement for ElasticSearch, still unambiguously libre/open source (Apache License 2.0) For those unsure what to pick, recommend OpenSearch 1.x so we don't push people towards a not really libre dependency. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|NEW |RESOLVED --- Comment #16 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Looks like this is done then, closing. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |33626 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33626 [Bug 33626] compare_es_to_db.pl does not work with Search::Elasticsearch 7.0 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 --- Comment #17 from Victor Grousset/tuxayo <victor@tuxayo.net> ---
So unless someone thinks of a problem I'll update the system requirements wikipage with that.
Finally done. For ES 7 and OS 1 support. No recommendation for now, I'll have to email the 2nd half for comment 15 to koha-devel so we settle on something. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Bug 25439 depends on bug 33626, which changed state. Bug 33626 Summary: compare_es_to_db.pl does not work with Search::Elasticsearch 7.0 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33626 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to oldstable |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25439 Bug 25439 depends on bug 25669, which changed state. Bug 25669 Summary: ElasticSearch 6: [types removal] Specifying types in put mapping requests is deprecated (incompatible with 7) https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25669 What |Removed |Added ---------------------------------------------------------------------------- Status|Needs documenting |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org