[Koha-bugs] [Bug 22417] Delegate background jobs execution

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417

--- Comment #51 from David Cook <dcook at prosentient.com.au> ---
(In reply to David Cook from comment #50)
> We use RabbitMQ for some other projects, and I think I'm going to start
> looking at using it with some of our local Koha extensions. 
> 

With my own code, I've added a local tool to tools-home.pl, and I have that add
a JSON message to RabbitMQ with Net::Stomp.

I have a background worker using Net::Stomp to get the JSON message, do the
long-running task, and then email a user with the result. 

It does the bare minimum of what I need it to do, and I'll be looking to roll
it out soon as an experimental feature to select Koha instances.

Both the web app and daemon worker rely on configuration in koha-conf.xml to
know not only the RabbitMQ server address, username, and password, but also
what queues to use. (I rather use a YAML file for this, but koha-conf.xml
already has caching support, so it's so easy to use.)

Things to think about:
- Installing RabbitMQ and plugins are easy, but you need enable the stomp
plugin and create virtual hosts as the RabbitMQ administrator... which is only
possible if they're running on the same machine
- Technically, all Koha instances could use the root vhost, and we could prefix
the queue names. (I'm going to be running into this issue when I roll out my
experimental work more widely.)
- Flow control for the background worker. Should it do 1 task at a time, or
should it fork off workers? If it forks workers, you can easily exhaust your
system's resources if you don't have any artificial limits, but you can also do
more work. (Alternatively, we could create the facility to run X background
workers, so sysadmins would determine the limits.)
- How best to store the results of background tasks? Should it be in 1 central
store, or should it be more task specific?

Most of those are sysadmin type questions except for the last one. For the last
one, I'm thinking task specific might be best, especially when it comes to
background tasks done by Koha Plugins.

-- 
You are receiving this mail because:
You are watching all bug changes.