Well, if ES doesn't need to be replaced, adding Solr support would still be an interesting experiment. ;) Indeed there are more reasons for clustering. I was thinking more of the traditional ES default of 5 shards per index. In any case, when you start thinking about HA and LB, there's always some added complexity, and while requiring Zookeeper with Solr is another puzzle piece compared to ES's way, I've found it to be a very small part of the whole deal. And if you want HA, you'll probably want to go for 3 nodes or another odd number to avoid any split brain issues. With Solr you can easily have two beefy nodes running Zookeeper+Solr alongside a smaller one running only Zookeeper. What I've found more of an issue with both ES and Solr is the documentation or lack of it. While most of the time all the API calls, configuration etc. has been documented, the core concepts and why you'd want to use them may not be. Solr's has been improving, but there are still some features that I have trouble understanding properly (e.g. autoscaling stuff). --Ere dcook@prosentient.com.au kirjoitti 19.1.2021 klo 0.58:
At this point, I don't think that we need to give up on Elasticsearch. After more review and reflection, the licence change doesn't affect the Koha Community, since the Koha Community doesn't modify or redistribute Elasticsearch source code. And all the recommended installation methods have used the compiled Elasticsearch that is licensed with the Elastic License.
I've never used SolrCloud, so that's interesting to hear those details!
As for clustering, I would say that there are more reasons to cluster than sharding/partitioning like high availability and load balancing. For instance, I maintain a number of clusters that can autoscale up/down based on demand and losing X nodes won't cause my services to fail. In terms of cloud computing, you can typically get 2 smaller instances for the price of 1 larger instance. If the workload can support it, I'd prefer a 2 node cluster to 1 standalone server. But... I would say that is a more advanced configuration. Most libraries probably would only need 1 standalone server and a bit of downtime might not be the end of the world.
I imagine that you're right about the largest amount of work with ES has been abstracting the search not to assume Zebra everywhere. In my other projects, I try to keep things modular, as you never know when you're going to have to replace a component with something new...
David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia
Office: 02 9212 0899 Online: 02 8005 0595
-----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Ere Maijala Sent: Tuesday, 19 January 2021 6:27 AM To: koha-devel@lists.koha-community.org Subject: Re: [Koha-devel] Elasticsearch changing licences
I'd say that you're right in that Solr is the only viable alternative. Which, from my point of view, is actually quite thrilling since I feel much more at home with Solr. That said, I find it just a little bit annoying how often it seems that something that happens outside your core of work forces you to do a lot of extra work.
I've yet to encounter an ILS with enough records to need clustering. Our discovery interface uses Solr with a 6-shard SolrCloud instance, but then it's 80+ million records. So I believe this won't be an issue. Solr also support sharding without SolrCloud, but it's still somewhat complex and not automatic like with ES. Though I read somewhere that ES is also moving to default to a single shard.
While both are Lucene-based, they're quite different. Or rather, the approaches they take are quite different. ES to me feels more like "make it easy to get started, but beware, there will be obstacles", while Solr is more like "here's what you need to consider before getting started, and it may feel daunting, but you'll be much more informed in the long run".
Luckily, I think that with Koha the largest amount of work with ES has been in getting to the point where it's not assuming Zebra everywhere. This would make the task of introducing Solr much more reasonable.
--Ere
dcook@prosentient.com.au kirjoitti 18.1.2021 klo 3.08:
I wonder that too. In the past, Solr didn't have cluster options, but it seems like it might have SolrCloud now for that? I think it uses Apache ZooKeeper which adds overhead and complexity. Elasticsearch's clustering is a lot more self-contained if I recall correctly. But then clustering would only be needed for more advanced implementations...
But Solr is probably the only viable alternative to Elasticsearch at this point?
David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia
Office: 02 9212 0899 Online: 02 8005 0595
-----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Joonas Kylmälä Sent: Friday, 15 January 2021 9:07 PM To: koha-devel@lists.koha-community.org Subject: Re: [Koha-devel] Elasticsearch changing licences
Ḧi,
yeah, just heard about this. We might want to start looking into some community developed solution, maybe solr?
Joonas
On 15/01/2021 07:22, dcook@prosentient.com.au wrote:
Hi all,
Have folk heard that Elastic is changing the Elasticsearch licence?
Depending on your sources and your beliefs, this may or may not matter, but certainly something to be aware of.
Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : https://www.koha-community.org/ git : https://git.koha-community.org/ bugs : https://bugs.koha-community.org/
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : https://www.koha-community.org/ git : https://git.koha-community.org/ bugs : https://bugs.koha-community.org/
-- Ere Maijala Kansalliskirjasto / The National Library of Finland _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : https://www.koha-community.org/ git : https://git.koha-community.org/ bugs : https://bugs.koha-community.org/
-- Ere Maijala Kansalliskirjasto / The National Library of Finland