[Koha-bugs] [Bug 27078] Starman hanging in 3-node Koha cluster when 1 node goes offline.

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27078

--- Comment #6 from Christian McDonald <rcmcdonald91 at gmail.com> ---
Well Koha doesn't really know it is part of a cluster, does it? It still talks
to a single MariaDB instance (at localhost, just like a standalone instance),
and the same can be applied to ES (again a single localhost:9200 ES node could
be configured on each, just like a standalone deployment). 

Apache isn't aware it's part of a cluster. Plack/Starman aren't aware either.
Really Koha is still "standalone" from the perspective of Koha itself, it just
so happens that the DB, file-system and index are distributed.

So what I mean about taking nodes offline is that, when all nodes are online
and all services running, everything is very performant. As I would expect.
However, when one node goes offline, the Koha application itself becomes very
slow to respond to page loads...lots of spinning browser tabs waiting for a
response...but it will eventually respond. My question is, why? Again, Koha as
an application isn't aware that it is part of a cluster, it doesn't know that
it's database, file system and index are replicated under-the-hood.

Here's what's weird. Like I said, when all nodes and their services are online,
everything is fine. However, say if I "systemctl stop" Maria, elastic, Apache,
and memcached on a single node (say on Node A), everything still is fine when
connecting to either of the remaining nodes (Nodes B and C). However, if I then
power down Node A (remember, all it's koha-related services had been stopped)
or pull node A's network connection, node B and C become very slow to serve
page requests. Again, this isn't because of some performance degradation of
Maria (3 nodes can withstand 1 node offline), Elastic (again, one node offline
is okay). 

Also, when Apache is nuked, HAProxy immediately stops sending clients to that
node. So in that regard, once Apache is hosed, that node isn't even going to
get client requests.


What can I do to monitor performance, or poke into Plack/Starman? The reason
why I have a hunce Plack/Starman are involved here is because when I disable
Plack on all nodes, the loss of a single node doesn't impact the performance of
the remaining nodes... granted with Plack disabled they are all noticably
slower, which is expected

-- 
You are receiving this mail because:
You are watching all bug changes.
You are the assignee for the bug.