[Koha-devel] Memory usage by Koha daemons (rebuild_zebra.pl, SIPServer.pm, etc)

[dcook at prosentient.com.au]

Hi all,

 

I was thinking about memory usage by Koha daemons in light of a koha
listserv post about Koha resource requirements.

 

I notice that rebuild_zebra.pl uses about 170MB when idle. While that's not
staggering on its own, that does add up. That's 1.7GB for 10 processes,
3.4GB for 20 processes, 5.1GB for 30 processes, and so on. You get the idea.


 

That reminds me that in early 2018 on our openSUSE Koha server, I actually
wrote a Koha indexer daemon that only uses 12MB of memory when idle. It
periodically checks the databases for every Koha instance on that server and
launches rebuild_zebra.pl as necessary (in non-daemon mode) and uses
advisory file locking to avoid running duplicate index jobs for an index.
The memory footprint is tiny. I can't recall the exact number of instances
that covered but a conservative estimate would be 60. That's 1 process using
12MB vs 60 processes collectively using 10.2GB of RAM. 

 

I've opened https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27267
to explore this further. In fact, as I was writing comments there, I
realized that an alternative would be to keep 1 process per Koha instance,
but to make that a very lightweight process which just checks if there is
Zebra indexing to do, and to launch rebuild_zebra.pl if there is. I suppose
there pros and cons to both centralized and decentralized models. We already
use decentralized so may as well stay that way and just optimize it.

 

I think that I could actually use some paid work time to do this too, so if
people are happy with this last approach, I'd go ahead and do it.

 

--

 

I notice that SIPServer.pm also uses about 165MB of RAM when it's idling.
That also adds up. However, I don't know how much of that is due to
Net::Server::PreFork and how much of it is our Koha code. I'd have to
explore that more, and since I don't run tonnes of SIP servers, it's not a
priority for me.

 

David Cook

Software Engineer

Prosentient Systems

Suite 7.03

6a Glen St

Milsons Point NSW 2061

Australia

 

Office: 02 9212 0899

Online: 02 8005 0595

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20201218/44a99e65/attachment.htm>