https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32573 --- Comment #8 from David Cook <dcook@prosentient.com.au> --- (In reply to Kyle M Hall from comment #5)
Considering batch record modifications are not atomic, I don't think that's necessarily a compelling argument.
I think that shows that batch modifications are problematically designed. Consider my comment on bug 32278 that talks about how having item/record numbers saved into tracked batches. We could then process those batches in an atomic fashion.
I can't see any scenario where either: A) We should ack before processing. We are stymying Koha's ability to do background processing by running extrememly long processes in serial with no way to fork multiple background jobs at one.
B) RabbitMQ/STOMP is the wrong tool for the job and we should remove it's use from Koha.
Actually, I can revise A a bit. We can fork and give the existing rabbit connection to the child process, and have the parent start a new connection to pass off again. That mains we can parallel process without acking first. It doesn't solve the 30 minutes time limit ( which is why I'm going to continue advocating for acking first ) but it does improve the situation greatly.
From day one, I've advocated for multiple worker processes. Frequently, you start multiple worker processes independently of the web app. I have other applications where I run the worker processes in standalone containers and I can start as many as I want to do the work based on the workload of that application (based on the computational power I have available of course). I also put them together in pipelines which means I can process lots of data very quickly. Those apps are way more data heavy than Koha. I still ACK after I process my RabbitMQ messages.
In Koha, we shouldn't be having extremely long running processes. We don't do any 1 thing that is really computationally that heavy; we're not running complex image manipulation on high resolution medical images with terabytes worth of data. We're just writing inefficient suboptimal code. RabbitMQ isn't the wrong tool for the job, but I think the Koha community doesn't have the knowledge/experience to know how to use it (well). And I'll include myself in that camp. I think that I've probably used RabbitMQ more than anyone else in the Koha community, and I'd still consider myself a RabbitMQ newbie. -- You are receiving this mail because: You are watching all bug changes.