[Koha-devel] Deploying Koha at scale

Fri Jun 12 02:58:32 CEST 2020

Thanks, Mengu (and Ere)!

It sounds like Zebra is really the most challenging aspect of scaling right
now. Here's hoping that Elasticsearch is or will be stable enough to be used
as the main search engine. 

Mengu: What do you do about cronjobs for a Koha installation running over
multiple servers? Disable them on all but 1 VM for that Koha installation?

Actually, now that I think about it, Koha Plugins could be a challenge,
although once I improve the tooling for installing plugins, it should be
manage plugins at scale.

Thinking about RabbitMQ. that should be OK if each instance has a worker and
the RabbitMQ servers are clustered (or at the very least centralized).

Mengu: Is your Memcached clustered?

Mengu: Do you do any log shipping, say with rsyslog?

Sorry for all the questions. I like to think about as many details as I can
heh.

David Cook

Systems Librarian

Prosentient Systems

72/330 Wattle St

Ultimo, NSW 2007

Australia

Office: 02 9212 0899

Online: 02 8005 0595

From: Koha-devel <koha-devel-bounces at lists.koha-community.org> On Behalf Of
Mengu Yazicioglu
Sent: Friday, 12 June 2020 5:12 AM
To: koha-devel at lists.koha-community.org
Subject: Re: [Koha-devel] Deploying Koha at scale

Hi,

I'd like to share some experiences of our installations and what we have
done and we haven't.

We've used many servers for apache part, and galera cluster for Mariadb
part. We used nginx for load balancing and I shared a new entry in the wiki
about our conf:
https://wiki.koha-community.org/wiki/Nginx_conf_sample_for_load_balancing_wi
th_many_servers

The main problem always occured from Zebra site as you said David. Indexing
for more millions records was horrible of course.

After indexing it was easy to copy data and create new server while working
in a huge VM environment. Not only indexing data but also with many requests
from users also used a lot of resources of hardwares. Since Zebra part was
the weak point, we preferred to power database servers and change a lot of
code just to search directly from database instead of using Zebra. This gave
us quite big flexibility in our installation.

In some problems, if we need to do re-indexing Zebra, we used nigths of the
day to do that and used some extra servers for indexing and when finished we
copied the data.

I personally spend a lot of time for Zebra things and I really hate its
structure. I hope Elastic search will be good for cluster environments.

We also thought Dockerized environment for a scale for large installations
but we could not find necessary money and time to finish.

Best regards

Mengu

Devinim Ltd.

On 11.06.2020 11:03, Ere Maijala wrote:

Hi,

This is a welcome discussion, thanks for starting it!

A couple of quick comments:

1. If you're running Elasticsearch and Z3950 Responder, I can't really
see a need to run Zebra at all anymore. Is there something that it's
really needed for?

3. Z3950 Responder is pretty lightweight, and it'll be the search index
doing most of the work. But I understand the need from fault tolerance
point of view. You could use YAZ Proxy or Metaproxy as the load balancer
for multiple Z3950 Responders. However, that's still a single point
managing the sessions.

--Ere

dcook at prosentient.com.au <mailto:dcook at prosentient.com.au>  kirjoitti
11.6.2020 klo 10.52:

Hi all,

For a long time, I've been thinking about how Koha could scale better, and
I'm wondering what other people are thinking about this topic. I think Mengu
is probably the best expert on this topic that we have on Earth, so
especially curious about his input. 

It's relatively easy to add more Apache web servers, more MySQL nodes, and
more Elasticsearch nodes. but the same might not be said for other parts of
Koha like the following:

1. Zebra
   a. In theory, you could load balance queries to a cluster of Zebra
listeners
   b. But indexing updates to many Zebra databases could be tricky (I
don't know if Zebra's database can support many readers). I suppose you
could alter rebuild_zebra.pl to update multiple databases in a variety of
different ways. I imagine Mengu could speak to this, as I recall him talking
once about needing to do work to make Zebra scale?

2. I'm not familiar enough with the SIP server to say
   a. Off the top of my head, I think a TCP load balancer would be good
enough here

3. Z3950 Responder? 
   a. I'm not familiar with this daemon

4. What about cronjobs? Only 1 instance should execute the cronjobs.
   a. I suppose a person could disable cronjobs on all Koha nodes
except for 1? It wouldn't be very robust in terms of availability but it
would technically work most of the time. 

Any other thoughts?

David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia

Office: 02 9212 0899
Online: 02 8005 0595

_______________________________________________
Koha-devel mailing list
Koha-devel at lists.koha-community.org
<mailto:Koha-devel at lists.koha-community.org> 
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

_______________________________________________
Koha-devel mailing list
Koha-devel at lists.koha-community.org
<mailto:Koha-devel at lists.koha-community.org> 
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20200612/c10179cf/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 484 bytes
Desc: not available
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20200612/c10179cf/attachment-0001.sig>