[Koha-bugs] [Bug 22417] Delegate background jobs execution

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=22417

--- Comment #4 from David Cook <dcook at prosentient.com.au> ---
(In reply to Jonathan Druart from comment #1)

> Discussions:
> <snip>
> 2. Even if we do not go for RabbitMQ I think we could start moving the code
> from pl to Koha::BackgroundJob::*pm
> It will be "easy" (nothing is easy, right?) to replace it with something
> else.

I totally agree; it would nice to be as message queue implementation agnostic
as possible.


> Going further:
> 1. Implement Koha::BackgroundJob for the different background jobs
> 2. Provide a koha-worker script to consume to jobs
> 3. Rewrite the task scheduler with Koha::BackgroundJob (1993)

It's been a couple years since I looked at RabbitMQ. Is there a way to
cancel/remove/delete a message once it is enqueued? I think this would be
useful for a task scheduler. I built that into the POE-based task scheduler I
built for https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10662.
(Actually, I also built-in the ability for in-progress jobs to be canceled in
the event a big batch job was started and a person realized it was a mistake.)


> 4. Provide a configuration file to prioritize or postpone jobs (for instance
> no batch modification on Wednesday between 9-17) - Or maybe we won't need
> that (?)

Fridolin mentioned something like this to me when I was talking about
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10662 with him at
Kohacon18 in Portland. 

He pointed out that for a multi-tenant server you might not want all the
different Koha instances on the server to run their background jobs at the same
time due to restraints like performance, rate-limiting, connection limits, etc. 

Off the top of my head, I don't have a solution for this one though, although
in the case of https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10662
I use separate task schedulers for every Koha instance on the server. 

In this case, we could potentially use 1 RabbitMQ server, and X number of
workers... although we'd have to have a way of setting the KOHA_CONF for each
task. 

That might be the best way to handle multi-tenant scenarios. 

Alternatively, I think we'd need to have unique queues for each Koha instance
and workers for those unique queues. Requires a bit of thought in any case. 

> 5. What would be your needs?
> 

Ideally, it would be nice to replace my home-made task scheduler in
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10662 with this one.
I like that mine can add/start/pause/stop/remove tasks, but I haven't seen a
mainstream task scheduler that is capable of doing that, so that might be too
difficult to do here.

-- 
You are receiving this mail because:
You are watching all bug changes.