[Koha-bugs] [Bug 32573] background_jobs_worker.pl should ACK a message before it forks and runs the job

[bugzilla-daemon at bugs.koha-community.org]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32573

--- Comment #12 from David Cook <dcook at prosentient.com.au> ---
(In reply to Tomás Cohen Arazi from comment #11)
> (In reply to David Cook from comment #8)
> > 
> > RabbitMQ isn't the wrong tool for the job
> 
> The question is: are we better with just polling? What do we lose?

For starters, I'd refer people to Jonathan's comment from July:
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30654#c17

Database polling has a number of shortcomings. It's slower and less efficient.
It's not easily scalable. You have to implement all the messaging and
concurrent processing logic from scratch. It's a naïve method of queue
processing that people resort to because they understand relational databases
and don't understand message queues.

I think that one could argue that Koha is allowed to be slow, inefficient, and
not scalable. The amount of background processing that Koha needs to do is
going to be minor in comparison to many other systems. 

So I imagine that Koha could get along just fine with a bespoke database
polling background job processing system. In hindsight, we probably should've
done that originally. Perhaps we were too ambitious in our desire to use modern
tools and standards. 

But I think that just using database polling would be a backwards step for the
project overall at this point. Koha's usage of RabbitMQ is improving. We devs
are collectively improving. 

I think we really only have 2 remaining problems:
1. How to handle jobs that run longer than the RabbitMQ default 30 minute ack
timeout (I think the "ack_late" for some jobs is a good compromise to handle
this - alternatively Koha implementors can tune their own RabbitMQ
installations if they find the default is too low)
2. How to have multiple workers when using Koha packages (this is easy with
RabbitMQ, but the existing database polling failover makes this significantly
harder, but it's a solvable problem)

-- 
You are receiving this mail because:
You are watching all bug changes.